I was invited to give a talk at the Growth Marketers | San Francisco Meetup by Joshua Fechter, Head of Growth at UpOut. I met Josh at the Growth Marketing Conference earlier this year. Definitely an event to attend if you’re on the marketing side of your company. Also, if you don’t know about Josh, then you should come to one of the his meetups because there are very few people I’ve met in growth that automates and experiments at the scale that he does with only 1 person; respect. Oh, also there’s free pizza and beers! Win win!
For the talk, I specifically chose to tackle churn for PaaS companies, because there hasn’t been much written about it as compared to SaaS metrics. And since a lot of companies are trending towards a platform business model, we should definitely have more resources.
Below are the slides and dialogue (almost verbatim).
Hi Everyone! Thanks for coming out tonight. I chose this topic specifically because it’s something that we’ve worked hard to track at Plivo and unfortunately there hasn’t been that much written about it because most of what we read about churn & LTV comes from the SaaS world. And if we took the exact SaaS churn calculations and applied it to PaaS, then we’d be very mislead and tonight I’ll illustrate why that is.
First off, a little about myself and why I’m semi-qualified to talk about this. I’m the first full-time marketing hire at Plivo, which means that I wear a lot of hats. And being a very lean team means that we can make decisions quickly and adjust our strategy based on data and results.
This also means that, not only do I spend a lot of time on implementation, content, and PR amongst other things, I also spend a lot of time with our co-founders strategizing for growth. And if you add all those percentages together… it’s 310%! Which is pretty normal when you’re going through the type of growth we are experiencing.
In the nearly 3 years I’ve been at Plivo, we’ve experimented with a ton of stuff; too many to name here. We were lucky enough that some of those experiments paid off and we became profitable 2 years ago. Today, we continue to grow rapidly. And lot of this growth has to do with our super low churn rates. And tonight, I’ll share with you how we got to those conclusions (without giving away too many secrets).
So in this talk, I’ll go over how Platforms differ from the SaaS model and how we modified certain things in the SaaS churn model to account for unique customer segments.
So briefly, why is churn so important?
Churn is now synonymous with startup success. Most of us understand how negative churn can “silently kill your business“. Veterans in the space like Andrew Chen and David Skok have written extensively about why measuring churn is one of the most important things you can for your business.
To put it plainly, if you’re spending more money on customer acquisition than you can retain, then you have a serious problem. And in an conservative VC environment that makes it increasingly more difficult to raise money, revenue and profit are your best friends.
To paint you a picture of how severe a problem like churn is. In 2004, Netflix shareholders filled a class action lawsuit over its “improper calculation” of churn rates. The shareholders argued that Netflix used a churn calculation that produced artificially lower churn rates than what was actually occurring.
The most interesting result of the lawsuit was that the judge actually threw out the case; ruling that there was no single industry-wide definition of churn rate.
This just shows how significant differences in churn rate calculations can occur even for public companies. This should be alarming because if your model is flawed, you could be making detrimental underestimates of your churn rate. It’s just not as simple as taking the the percentage of participants who discontinue to use your service and divide it by the average number of total participants during a given period of time.
So in the following slides I’m going to show you why this may be the case and how you can modify calculations to fit your business model.
A lot of the churn calculations you’ve read online come from the Software as a service Business (SaaS) business model. SaaS models have been around for a long time and they typically have a tiered pricing structure like this one. I’m sure everyone has seen this before; where you’re trying to get the customer incentivized to pick the “BEST” option.
The essence of a SaaS model is that the pricing tiers are based on monthly recurring revenue (MRR). Here, customer churn can be easily calculated based on monthly cohorts and when you look at it on an individual customer level, it looks something like this…
Each tier has pretty much the same structure as this one. Every month, you’re collecting a fixed amount of money from each customer. And depending on your structure, some tiers can even have contracts, which adds to predictability of MRR, which we and investors love. Contracts are great for adding predictive revenue for SaaS models because then you know exactly when your customers will renew or churn.
And because there is such a fixed cadence of revenue, we can easily look at the customer pool at an aggregate monthly level…
Since revenue is collected every month on the same renewal date, you can easily setup a monthly cohort chart and see where customers are dropping off in each succeeding month. From here we’d be able to calculate percentage of churn, revenue churn, and most importantly life time value.
I’m not going to go over exactly how you calculate churn for SaaS because there are a lot of resources out there.
What I want to focus on are the flaws of applying the SaaS concept of churn directly to the PaaS business model.
But first, we must understand the limitations of this SaaS churn model.
When churn models like this one use monthly cohorts, it fundamentally relies on the assumption that no customer can churn within the first month. This makes a lot of sense for SaaS because the monthly pricing model ensures that all customers pay for the month up front. Therefore, when you take a snapshot at the beginning of the month and then divide that by the total number of churned customers, you don’t have to worry about anyone within this cohort churning during the month that they signed up.
But this is not the case when you’re dealing with platforms as a business or PaaS businesses.
What is platform as a business?
PaaS used to be reserved for software that’s a layer below SaaS. It’s a type of business that replaces the complexity of building and maintaining a specific infrastructure. For example, Amazon Web Service (AWS), IBM SmartCloud, RedHat’s OpenShift, Google App Engine all fall under this traditional definition of SaaS.
Today, PaaS is used to loosely describe software that replaces a component of your application that you don’t have to build and maintain.
Take Uber as an example…
Uber claims that they are a platform and rightfully so. Not only is Uber extending into other delivery businesses, they have opened up their API for third-party applications. Any application can now request a ride for their users without opening the Uber app; replacing the need to build the rideshare functionality yourself. Uber started as a taxi replacement, but it’s long term mission is to create reliable transportation everywhere. A much bigger picture than fixing the taxi problem alone.
What does this mean?
It means that revenue will get rather hard to predict for PaaS business models. Though I’m sure Uber has amazing data scientists that make it a breeze.
But think about it from an user’s perspective. As a rider, I can take an Uber for any distance at any time of day as many times a month as I’d like. Contrary to the SaaS business model, there is no fixed amount that I need to adhere to. And as businesses move towards the PaaS model, we see this type of fluid revenue more often. This is true not only for Uber, but also for platforms like Airbnb and of course Plivo.
So, being a bit of a geek, I mapped out my spend on Uber to see if there was a pattern.
This is my monthly Uber spend over the last year since march. It looks to be some what consistent, with most months above $80. Side note: this was actually pretty surprising because I use Lyft much more often. Gasp! How much am I spending on rideshare?!
At an aggregate level, there is probably a solid average that accounts for a significant percentage of their riders. And as long as your commute and lifestyle don’t change. Your spending behavior can be relatively consistent.
Take Airbnb as another example…
One could argue that Airbnb doesn’t fall under PaaS because it’s not really replacing any type of infrastructure or function. But Airbnb’s revenue looks more like a PaaS company than anything else. This is because hosts can set their price to anything they want and Airbnb takes a percentage of hosting fees. This means that Airbnb’s revenue can be even less predictable.
This is my Airbnb lifetime spend since I joined in 2012. Just look at that gap!
After my initial purchase, you’d probably consider me a churned customer. But in reality, Airbnb has always been on my evaluation list every time I make travel plans. It just happens not to work out most of the time because I find a better deal elsewhere. Side note: Airbnb should really look into incentivizing return customers.
Even though I’m only one data point, you can see how churn can be really unpredictable for PaaS companies. There’s not too much pattern to my spend except some seasonality clustered towards major holidays.
Then, how do we control for these variances in customer lifecycle and deal size?
I get asked about churn and revenue a lot at Plivo. Even though our MRR is consistent and growing, as a true platform provider, we don’t have tiered offerings or contracts like SaaS that make our churn rates straight forward.
Let me illustrate why calculating churn can be so difficult for PaaS.
If you’re running a platform. Your total revenue at the customer level may look more scattered than this.
There’s no real trend. Nothing that shows any specific pattern. And after pulling customer spends over a 12 month period, you can see that they are all over the place…then you try everything you can think of to connect the dots and it totally doesn’t work…
Obviously, you also attempt to model this out other ways, with a straight forward monthly churn analysis, but that’s where the SaaS churn model starts to break down.
So what do you do?
Most start by trying to control for the variance they have in their customer pool.
At first, you may have tried the standard churn approach, taking on a monthly cohort to standardize your calculations. So you sum total customer revenue per month and make your churn calculations that way. There’s nothing wrong with this approach, except (EVERYTHING! j/k) that you have to keep in mind that you also need to control for the variance across the entire group of customers you are averaging.
For example at Plivo, we have customers that pay us $25 a month and some that pay us 6 figures a month. And because there is so much variance within your pool of customers, taking the average is just not enough. Remember, this is also under the assumption that your customers don’t churn within the same month, which may not be the case.
Second, you may have also looked at increasing or decreasing your cohort length to control for the variance in customer life cycles. But as a PaaS model, customers can churn within days or stay with you for years. This means that one broad calculation probably isn’t going to cut it.
It’s a rare occurrence to have so much variance of customers within a single product but this is completely normal for PaaS. This is actually the nature of a self-service platform business. Essentially, the customer base of a single product can come from different personas.
So now what?
We have to work our way backwards to get to the basics of customer segmentation. This is not an intuitive step, but it’s essential.
Remember the SaaS tiered pricing model?
Well, the reason why churn is easier to calculate is that customers self-select the package they want. Therefore, you can be sure that everyone in your “Awesome $2000 package” group has the same persona and therefore likely to have very similar churn rates too. This allows you to not only calculate churn at an aggregate company level, but you can also calculate churn at each of the tiered pricing levels, which can help you understand how each of your product lines is doing.
For PaaS models, customer segmentation is not this clean cut. Unlike SaaS models, where customer segmentation is done at the top of the funnel, PaaS segmentation has to be done ad hoc. This is especially true when you have a self-serve platform like Uber, Airbnb, or Plivo.
You could try to bend your business to fit neatly into a SaaS model, but if it won’t work if tiered pricing and annual contracts are not the norm for your industry. From experience, a better approach is ad hoc customer segmentation because it’s the least disruptive to your customer experience.
This approach requires you to analyze your customer base and segment customers by a criteria that makes sense for your company. Typically, this will be revenue, but it can also be use case, region, company size, etc.
After you analyze your customer base, patterns behind different customer sets start to emerge. For a platform like Plivo, we can see distinct personas within a single product line based on revenue. Here are some pretty graphs to show you how your customer segmentation by revenue could look like.
After some analysis, personas within your platform customer base will start to emerge. And typically, each persona will gravitate towards a life cycle trend as well. Here’s an infographic showing how customer life cycles can differ within each customer segment.
At Plivo, we’ve found that, our largest customers have average churn rates of over 3 years and most of our customers from the very beginning are still with us today.
To sum it up, we can apply this methodology when we’re analyzing new data for the first time.
- Research: What ever you’re trying to model, it’s very likely that it has been solved before. Though, you may have to borrow ideas from other industries, so keep an open mind.
- Apply out of the box models: You don’t need to reinvent the wheel, but do understand its limitations. Because just like the SaaS business model, it may not work for you. So be cautious about how the model applies to your business and anticipate how it can skew your data.
- Modify the model to fit your business: Even though I’ve now illustrated how we calculated churn at Plivo. Don’t just take my word for it. Instead, develop a strong baseline based on your own data, match that against an industry average and see if it makes sense. Remember, even industry averages can be misleading, so be sure to run multiple comparisons.
- Forecast and let data speak for itself: If your model works, you should be able to use it to predict your revenue, 1 month, 3 months, or 6 months out to a certain degree. Try to select an interval that makes sense for your business and be weary that long test intervals can also lead to larger discrepancies. That is, one degree off from Pluto can mean light years of difference. And why not put money on those projections? Seriously, get an office pool going, may the best model win.
- Audit your model regularly: Even if your model works within acceptable tolerances, don’t just set it and forget it. As your business grows, your customer base can be changing too, so audit your model regularly.
And that’s it. Here’s how to find me.