a show dedicated to the best practices for building and growing app businesses. We sit down with the entrepreneurs, investors, and builders behind the most successful apps in the world to learn from their successes and failures. SubClub is brought to you by Revenue Cat. Thousands of the world's best apps trust Revenue Cat to power in-app purchases manage customers, and grow revenue across iOS, Android, and the web. You can learn more at revenuecat.com. Let's get into the show.
And my guest today is Dan Layfield. After helping Code Academy scale ARR from 10 million to 55 million, and working as a product manager on Uber ETH, Dan now consults with startups looking to increase subscription revenue and blogs about his learnings On the podcast, I talk with Dan about estimating the ROI of product changes before building them, calculating your subscription app's growth ceiling, and why you shouldn't make assumptions about what is and isn't working in other apps.
Hey, Dan, thanks so much for joining me on the podcast today. Yeah. Hey, what's up, David? Thanks for having me. So you've been blogging up a storm the last couple of years. I've linked to your posts a ton in the sub club newsletter. I've tweeted out links to your blogs, posted about them on LinkedIn and commented on LinkedIn. So I wanted to have you on to talk more in depth about some of these things that you've been writing about.
Of course, you've had a ton of great experience at Code Academy and worked with consulting clients who you've helped with monetization. And so I want to dig into some of these writings that you've done the past couple of years. And one of the more interesting topics that you've kind of touched on here and there. is this idea of focusing on the ROI of projects.
This is a really tough one. So first of all, like, just tell me what you mean by that, like high level overview, and then we can kind of dig into the specifics of how you actually determine the ROI of a project. Yeah, sure. I'd say it's the thing you kind of learn the hard way as a product manager. You know, I was at Code Academy for five years. I saw us grow from about 10 million ARR to about 55 million ARR. You know, obviously there's a ton of factors in that, just not the growth team's work.
but i think it's a lesson you learn as kind of like a young product manager that you can in theory do lots of ideas but when you're responsible for moving metrics you need some concept of like the headroom your projects have You can't always 100% forecast it accurately. You're on a team with six to eight engineers and a designer and a product manager. Your two-week sprints probably cost somewhere in the area of $40,000 to $50,000.
So it's, again, I kind of always have to keep like a mental math back to what's the likely output of the things we're doing on. And like your company literally will not be profitable ever if the output of your sprints doesn't exceed the cost of your. I think that concept like makes sense to people. It's a really hard one to implement in practice. In my experience, it.
not productive to try to 100 forecast the impact of every product change you're going to make But I think the way I would think about it is if you were to kind of divide up the work you're going to do in a quarter into the buckets of kind of like technical debt clear down, small stuff and bugs that just kind of has to happen and big swings.
big swings are typically where things get expensive where you're implementing like giant new features or you're like taking a big shot i think those projects definitely need headroom analysis you need like a concept of what the upside could be for all the money you're going to put into them through development. You know, a lot of folks go into these big projects. I'm in the middle of one right now with my inside project weather app. You know, we've spent the last five months on a big swing.
But honestly, I didn't put dollar numbers to that. The hardest thing on the big swings is that you also don't know how long they're actually going to take. It's really tough to estimate those things. Let's start with the big swings and we'll kind of work backward from there. When thinking about those big swings, how do you think about putting numbers to the potential for a big swing? Across my career, it's always helped me to do the basic math.
So it's, you know, we have an LTV of like $100. We think this feature is going to increase early lifecycle usage and therefore lower churn and therefore increase LTV.
i still think it's good to chart out the basic assumptions of how many people do we sign up for month how many people activate on what screen do you think people would see this feature what percent might click on it and like of that what percent might adopt so that's pretty simplistic math it's not going to show you necessarily if you're right but it will show you if your assumptions are wrong That's a great way to think about it too, because then you're actually having to force yourself to.
look at the numbers. And we'll get into this later with some of your churn-related math. But if you're only bringing in 100 people a day or 500 people a day, you do have to understand that there is a certain amount of headroom that any one feature can get adopted. And then part of that equation, I would imagine, is also will this lead to greater conversion? Do we have a hypothesis that this feature will actually cause more people to convert?
And then will this feature cause more people to retain? How do you think about all those factors as well? I guess I think mostly about where you want to initially focus. So I think like a mistake I see in a lot of companies and I definitely made a ton of times earlier in my career is like when you were in a company or if you were a founder working on stuff. You feel really productive, kicking a lot of stuff off in parallel.
Because if you have a project tracker, it gets a ton of lines that show a bunch of stuff in progress and the team feels really busy. But the easiest thing to lose sight of when you're working on a product is like projects in progress don't do anything for your business. They only start to do something in your business when they go out the door. so if you have big projects kind of like chunking them up into milestones so you ship things incrementally and start to slowly test assumptions
goes a long way. I've never been at a company where the first milestone going out is something people feel comfortable with. It's almost always like hackier than you want it to be, or it's not the full experience. and i think where that can go wrong is i think when i'm going back to my point on focus
I think you should pick points in your life cycle or monetization or product to try to deliberately increase the effectiveness of. And then you just stay focused there long enough to take a couple shots to see if you can actually improve it. Across most of my career, like the biggest wins we found were always like the third or fourth swing at something. Because if you ship something, you realize it doesn't work. And if you move on immediately then, unless you were completely off base.
It doesn't work. Ideally, you have enough tracking setup where you can like see signs of light. somewhere or you're running an A-B test and it like... the variant didn't get totally crushed by the control. You realize you're like kind of onto something. because i think when you move on too quickly it's like it takes your engineers a little bit of time to learn that area of the code base
So there's certain like knowledge debt that they pay down to figure out how to work there. Same with the designers, same with the founders, et cetera. How do you think about some of these big swings impacting top of funnel? For example, my side project weather app, the big thing we've been stuck on is a feature that my hypothesis is that one, I'll be able to get some press around it because it's very innovative.
Two, it's a kind of feature that I think will do really well in marketing. So I think this is a thing we can finally, you know, spin up some TikTok videos, advertise it on Meta. It's a kind of feature that I think. will get people's attention. Now, did I do anything to validate those assumptions? No. And this is where it's easy to get stuck. But how do you think about top of funnel in relation to these big swings? Yeah, you can take big swings anywhere. I think...
The thing I, at least I've always tried to optimize for is like cost of learning something versus payoff. If you think something could go viral or get picked up or be splashy, it's like, how hacky are you comfortable being building something that can validate that?
without fully releasing it across lots of different types of entrepreneurship. Like, can you pre-sell access to something is a classic way of validating an early stage assumption, or could you just use mock-ups and test that on TikTok? And not get like stuck behind the whole development cost. Everybody has a little bit of a different comfort level in terms of like how growth hacky you want to be.
to validate something if you go too far the product feels spammy the product feels wildly disconnected from the actual experience everybody's seen mobile gaming ads where the actual game looks nothing like the advertisement So I think there's definitely like a balance to strike there. The funniest thing in mobile gaming though is that once people saw all those fake ads getting so much attention, they went and started building the games that were actually the experience that the ads portrayed.
And from what I understand, some of those have done relatively well. But yeah, I think it's a great concept. I mean, the tricky thing there is how much do you tip your hand to your future product? I mean, have you ever... done those kind of hacky validations? And how do you get comfortable around not getting ripped off? I mean, I think that's a big concern for a lot of folks, myself included, that if I show this publicly, people will just rip it off and then beat me to the punt.
I think that's relatively illogical, but maybe there is some truth to that. Like, how do you balance those kind of illogical fears versus... Maybe there is some truth to that. Across the companies that I've worked for or consulted with, the ones that do the best are the ones that are the fastest. and like know the direction they want to move in. So to me, it's like that is the most important thing.
So companies that like I've seen struggle more, like they spread themselves too thin too early or they take too long trying to build something and validate the concept. When I was at Code Academy, we were the leader in that little industry of learning to code at the time, which was big at the time. We saw companies copy stuff that we were doing that I knew didn't work. And we just like never cleaned it up because we moved on to something else.
But like there's a ton of other learn to code tools that duplicated features we built. that like we saw didn't work or we saw be net neutral or like ineffective. And they copied it assuming that we knew what we were doing. But in reality, like we were also testing something that we saw not work. And then we just like left it live because we got distracted and we moved on to other stuff.
That's hilarious. So then the answer there is maybe you do that validation at the point at which you know you can deliver on it rapidly enough to not worry about that competition. And then that's really just a fundamental skill of a business, especially these days with.
AI assisted development and other stuff, the pace of development is just moving so quickly and the pace of being able to copy something is moving so quickly. So you just need to be ahead of that as a business or you're not going to stick around. Yeah, definitely. I think like the point on what. to copy is to me like a nuanced one. It's like, I think you, the founder or the product team or whoever you are,
you should be opinionated on like the types of features you should build. And you should not look to competitors unless you're like woefully under the baseline. on what you should build next i think once you're building a future you should look at the best practices and reinvent as little of the wheel as so if you're building like a document finder like look at google docs look at sharepoint look at finder in mac look at all the little features those things have and those are probably the
whether you should have a document finder is kind of up to you. But it's like once you're committed to a feature set, just steal all the best practices from the people who are the best at it. Might not be your competitors, might be people who do that. Yeah. It's a really good point too, that you never know what is actually working and what's not working. When you look at things from the outside, I see this a ton in the app industry.
People tweeting, oh, this app is making 200K a month. Copy these five steps. And then for some of those actually know the folks who build the app. And I know, you know, four of the five points that this growth hacker made on Twitter is like, that's not why the business is being successful.
Those four things are completely irrelevant and aren't what's fundamentally driving the business. It's this other thing that you just can't even see, even though you can look now across TikTok and Meta and other platforms to see which ads. are being shown a ton, you don't always even know whether those ads that are shown a ton are actually profitable or not. And then even if they are actually profitable for that app.
would it then actually be profitable for your app? I think people lean a little too much on this, on copying the wrong things without really understanding the deeper context behind them. Exactly. I feel like that's why I kind of write my blog in the... style that i do i get the appeal of looking at other companies and like their tools now where you can observe their ab
And it's like, see, they ship one thing and then another thing, and they went with the other thing. And it's like, it's unclear from the outside. If that's true, like if they calculate that correctly, if they went with something that wasn't actually the winner, it's also, it's really unclear if that would apply to you, the company. So to me, it's more important to like, look at the theory behind why these things work. and care less about the exact implementation.
We've chased a few rabbits here, but I do want to circle back to the other couple of things you talked about in this kind of ROI equation. One of them is the small things. And how do you think about paying back the technical debt, making small changes? And the ROI of these smaller things along the way versus the big swings, which we've talked about. Yeah, I think about it kind of like an investment portfolio where you need a certain percentage in each.
So if you're just starting out, you probably don't have a lot of technical data. There's probably still like a layer of technical cleanup that has to happen or shuttering old features or updating libraries and scripts, et cetera. Maybe you only need like 10% of your quarterly development time going to that, but you need some percent.
or the product starts to get stale and technically really hard to manage. If you're at a giant company like Microsoft, they could be paying off 50% of their roadmap with technical debt items. I think it depends a lot on the face of life. I think in the kind of like... smaller optimizations bucket i think it's really easy for teams to get lost pursuing just the bigger things and forgetting about basic product quality
especially in certain areas like purchase flows, onboarding. There's certain parts of the application where 100% of your users go through it, and they go through it at times where they're not really sure if they're staying with their product. To me, those things should be polished to a mirror. So it's a thing I've talked about a couple of places, but one of the highest ROI projects we ever did at Code Academy is we rewrote all of the checkout page error.
so like if you plugged in a payment and it didn't work making sure you got a message back that she told you what to do and didn't just say like invalid payment method. Please contact your bank. Tell them what's wrong. Tell them what to fix it. If it doesn't work, send them PayPal. PayPal doesn't work, send them back to a card. It literally took like two days.
of downloading stuff, stack ranking the errors, rewriting it, re-merging it back to the code base. And like maybe that, let's say very conservatively lifted check on page conversion, like 1%, but a hundred percent of your new revenue goes through the check. So it's really like you lift the whole business's revenue 1%. So for a subscription app, that would be then making sure you're onboarding through paywall.
just incredibly dialed in. How do you think about forming a hypothesis around which things need work? I would imagine tracking... each page in the onboarding, getting good analytics around where people are dropping off, why they're dropping off and kind of finding the rough edges. Because I mean, this is, again, something I personally struggle with, something I see so many people struggling with.
is that you're trying to build this perfect product, but you're perfecting areas that don't matter. So how do you find the places that actually matter? So that this 25 to 50% of your bug fixes and technical debt and smaller projects are still impactful projects versus just polishing something that doesn't matter. I think it's always a mix of art.
So I think I always want the highest level of analytics setup I can get to. I think it's tempting to just view funnels, especially early stage funnels, as you look at the place where the biggest drop off. And you like start to intuitively work there. But inherently, like some of the parts of the funnel will have a big drop off. I prefer to work bottom up. So go from like the highest intent people back up.
It's really hard to convince people of stuff. So you should start with the people who are already convinced. In purchase flows, it's always the people who try to click payment and it doesn't work for some reason that's like under your control. So if you're selling through an app store, this matters less. But if you're selling a web checkout page, this matters a lot more. and go through areas of like bugs, confusion, dead end. I think a really good surveying technique is right after purchase.
You get kind of like one shot at asking an open ended question of like, was there anything about the purchase experience that didn't work for you? Was it confusing? or like is there any questions you had about this product before purchase that you couldn't find an answer to And it's like, you'll never get a flood of information through that, but you'll get like two, three, four, five things a week that start to like build your intuition around like what could be wrong in the funnel.
i think another thing that helps a ton is just like have people who've never seen it before go through it with fresh eyes so these could be your friends these can be usertesting.com people like you the person building this have stared at this thing for so long But frequently when I consult with clients, it's like I can see really obvious things because I've never seen these flows before.
But the person who built it like knows this is version three. And like this part looks weird because this was done between version two and version three. And there was like a really good reason for that at the time. But to a fresh customer, they don't know.
If you're five people and everybody's kind of focused and working together, that looks really different. But if you're a 50 or 100 person company, I saw a tweet, I think just this morning saying, if you're the CEO of a SaaS company, you should.
at least quarterly because teams underneath you are changing things rapidly enough that you don't know what's going on. So maybe even part of your... practice as a larger company should be having the CEO having another team go through the onboarding and looking for that in addition to people with zero context outside of the company looking at that on a regular basis. Yeah, exactly. I think a classic place to look for things to fix is the lines between teams.
So like if one team runs the ad and the other team manages the landing pages, there's no guarantee there's any continuity between the ads and the landing pages. It's like a classic place you find low hanging fruit of like. team a owns this step team b owns this step team c owns this step
No one's paying attention to the user gap, to the gaps and experience between those things because each team gets kind of focused on their area totally naturally. Yeah. And the app stores, it's really easy for those kinds of things to happen as well. If your screenshots. weren't changed for the last six months and don't even mention the feature that you now know is like one of your higher converting features you know that needs to be maybe screenshot one or two in your app store page if
you've built this new feature and think that that's what's driving a lot of your conversions, but it's easy to lose sight of that full funnel experience. Or like you said, at bigger app, It's maybe that the ASO team is working so independently from the monetization team that they don't even realize that this new feature or some change or some A-B test that won.
is driving a ton that they could pull up into the screenshots and other marketing copy and things like that. Yeah, exactly. I think when you're a product person or designer and engineer working on one feature set.
Whenever you do customer interviews, you want to ask them about that feature set, but to the user, it's all just one experience. Everything from your email marketing to your ads, to your app store page, to the product itself, to the things they hear about you, just kind of like one blob of opinion.
So to them, it's like, it's all one experience, even though like, I'm really curious about how, when I was at Uber, like, how did they like the ranking of the homepage? But like, they don't think like that. They think of, could I find food? Did it show up on time? And was it good? Even that journey is across like thousands and thousands of people work on all that stuff together. yeah it's crazy the next thing i want to talk about is you most recently wrote a post about
this idea of having a growth ceiling. And that's the fact that depending on the number of people you have coming in and your churn rate, you can actually calculate. how far you can grow based on that. And tell me about this concept and then we'll go through how that works in practice. Yeah, definitely. There's a couple of like really good rules of thumb to know within subscription products.
Probably the handiest is like one divided by your monthly churn rate as a decimal point is your average month. So if you have like a 20% month over month churn, that's 0.2. 1 divided by 0.2 is 5. So your average use will be around for five months. So you obviously see the power in that equation of reducing churn, like dramatically increases the length of retention. Another implication of that is every time you cut your in half, you double that number.
So like if you divide one by 0.1, you get 10. So just, it's not easy to reduce churn that much, but reducing churn is super, super powerful. An equation that we didn't even understand at Code Academy at the time is if you take the number of users that you're acquiring per month.
and divide that by your month per month churn rate, you get the ceiling of users that you can keep at that level. So if you have 500 users coming in per month and you're churning 10% of them per month, like 500 divided by 0.1 is 500. So 5,000, the number in which the inputs will equal the outputs. So unless you drop churn or you pick up the acquisition, your growth ceiling, basically, you'll never be able to grow above 5,000 users.
So this is something that I commonly see in subscription products. Before you're at that threshold, your user-based numbers will just keep increasing. And it's tempting to think that your product work is going really, really well. But we should really keep an eye on is churn the numbers that are upstream of churn and your acquisition numbers. Because if your growth ceiling is 5,000 people and you're at 3,000 people, you're just going to kind of drift up.
from a subscriber-based number regardless of your actions assuming churn and acquisition So it's easy to assume it's going really, really well, but you have to be very precise with what you're tracking. Yeah, I see this all the time. And we've talked about on the podcast before, there does seem to be a ceiling for a lot of subscription apps.
million to 10 million range, sometimes 10 to 20 million range where you kind of hit that. And it's kind of cool that you're mathematically describing the ceiling but it does seem to be that without some new growth lever without some big improvement in churn you just start churning out
the same number of users you're bringing in and you just kind of hit the ceiling. So you're growing, growing, growing, and it's looking good and you're drifting up. Maybe even you're not drifting up, but you're growing really rapidly, but then you just kind of hit this ceiling.
And equating it to churn and being able to calculate, I think is really powerful. We'll put a link to this blog post in the show notes. But how do you think about that for annual? Because it's not quite as easy to do the math on that. Yeah, I mean, the month-over-month churn rate is really like a blended average of all of your churn across monthly and annual plans. I think the way I think about that for...
I guess in most subscription businesses is you should look at this cohorted. So for every like month of users that sign up, what is their like month zero, one, two, three, four, five, six, et cetera, like churn rate. And you should look at that across both of your plans.
i think the best way of getting ahead of churn is not basing retention off of payments it's basing off of some sort of core action in the product so if you're slack for example and someone signs up for a monthly plan They might send zero messages for three months and just forget you're charging them $19.99, but they're probably churning eventually.
So if you really want an average retention at a product, figure out what the core action is. So in your weather app, it should be, I expect you to come in and check the weather three times a week or four times a week or one time a week or something like that. Because that will always like improving core activation and the core habit will always improve. Once someone is in the mental space of training, there are things you can do, but it's hard.
There's a couple mitigating steps you can take, but it's not nearly as effective as fixing it upstream. I want to dive a little bit more into churn mitigation. And you've written a lot about that. I think that's kind of one of the focuses of your blog is churn and retention. But before we dive into that, I did want to ask.
How do you factor in the kind of long tail? Because I've talked to enough apps in my role here. I've done office hours for years and then having my own apps. And then the thing is. Most apps, retention doesn't go to zero. It goes to some... percentage of that initial cohort. So if you're looking at, let's say year one, you acquire 10,000 subscribers and 60% of those churns, so you have 4,000 subscribers. But then by year two,
You're not turning another 60% of subscribers. You're maybe only turning 30% of those subscribers. And so like looking at my own app, I still have. 10% of monthly subscribers, and I think maybe even a little higher of annual subscribers from eight years ago, but that, you know, 10%, 8%. that's not a really fat tail. A lot of the apps that I've talked to that are doing really well, I'm building a fantastic business. that line flatlines more like 20%, 25%.
where, you know, you turn a higher rate the first year, a lower rate the second year. And by the third or fourth year, you kind of have Eric Crowley has this idea of tourists versus locals. True, it's like you're going to have people who come and check out your app and it's not a good fit or it's a good fit for a short amount of time. But then they move on. And then you have the locals being the people who your solution becomes part of their daily, monthly, weekly life.
And they're just going to stick around for years, maybe decades. So how do you factor in that long-term retention in this equation of what your growth ceiling is? I take the month over month average, factoring all the plans to calculate your just month over month churn rate. Again, month over month churn rate is not as helpful as knowing the cohorted and plan based churn.
but i think if you have a retention curve that like flattens at some point it's really good you always want it to flatten higher um but if you don't have it flattened ever if those curves go to zero it's really really hard to grow your user base and you're going to have a higher monthly So you can't stack a big user base.
If a hundred percent of your people after month seven go to zero, because you'll have to acquire so many people to get that. And you always have a bucket that takes a little while, but a hundred percent of the things you pour into it leaks out. i mean all subscription products are leaky buckets but you want that leak as small as possible I think when I think about why that happens,
To me, the least discussed part of churn management is the underlying use case you solve for. So like your retention length will be really dictated by how long the user has the problem you're solving for and like how much they need your solution to do it. so if you think of like cell phone plans like i don't know how long i've been on google fi but probably like eight years nine years like if they don't screw anything up i'll probably be with them 25 years
Because I have a daily need for their product and the workarounds and backups to them, like none of them are compelling to me. And I realistically need to sell. So my guess is like cell phone retention plans year over year retain 80% of people, like 90% of people, like maybe minus some switching, maybe I'm off, but you'd have really, really high retention. Contrast that with like meditation.
Where like meditation is like a thing you don't really need an app for. You'll probably either like learn it and like it or realize it's not for you. Like relatively quickly. So the only way you can scale like Headspace or Comm is you need millions of people. or you need a plan structure that locks them in. I think we're seeing that a lot with AI apps right now, is that Studio Ghibli converting your avatar and pictures and stuff.
OpenAI just blew up with that, which I don't think they even expected. But Sam Altman's been tweeting about their servers melting and adding a million users in an hour. That may be the fastest product. growth you know they were starting from a massive base i mean the chat gpt app is already massive and they already have a massive subscriber base And then to be adding a million users in an hour was just insane. So they had this viral moment. But to your point, if that's the primary use case,
People are going to churn out because they're not going to be converting their photos on a weekly, daily basis. And you need other use cases for that to be a sustaining subscription. For OpenAI, I mean, you would hope that people get in and start seeing the utility and using it for a multitude of things.
But I think there's a lot of AI apps that blow up and then there's not that kind of sustaining use case. Yeah, I think you kind of hit the nail on the head of like, if you don't have a sustaining use case, one, it might not be a good fit for a subscription product.
Or it might be not a good fit for a subscription product for most people. So you can see kind of like AI headshot generation being great for like... PR people or photographers or like heads of HR as a company who want everyone in their company to have the same headshot format or whatever.
So there might be like a subset of personas where those things work for. But if you don't tackle like a long-term recurring use case and provide value at like a cadence that exceeds what someone wants and therefore they're willing to pay for it, it's really, really tough to build a subscription. I think if you were to bucket the types of subscriptions that people pay for, you have the really long-term stuff like mortgages, financing products, loans, rent, cell phones, healthcare.
We're like those problems never go away effectively. And you can build giant monopolies in those spaces. You then have the short to medium term use cases of like dating, fitness, dieting, language learning, gaming. We're like, those are things people.
stick with for a couple months a subset of them will stick with them for a long time but really that's because they like your product a lot and they don't switch but the underlying like need fades away Although in gaming, it has been interesting to see Fortnite and some of these other companies, the way they've created this kind of sustaining use case is that entertainment is a long-term use case.
But people get bored and then go try other MMOs or other ways to get that entertainment. And the way some of these big game companies have been solving for that is with season passes and things like that, where the product itself... is changing enough to bring people back in on a regular basis with these. new characters, new game modes, new collaborations with big IP and things like that.
So even if your use case is short term, the perfect example with ShopGPT is like, OK, a bunch of people came in to convert their photo because it looked really cute. But then there are a ton of other use cases for chat GPT. And so then, you know, as they're able to expose. users to those other use cases and get them deeper into the product, there are those sustaining use cases that will be used on a more regular basis. And those are the kind of things that even if...
start out of the gate with a explosive viral success. Like this is what you should be looking for is that next season of Fortnite kind of thing where you keep building the use case or keeping it fresh enough to keep people coming back. Definitely. I think the recipe that works in the shorter term use cases is you need massive audience size. Like a Surefire recipe to die as a subscription company is short life cycle, small.
where it's like you just won't be able to acquire enough people and this thing will never grow I think kind of where you hit them with the Fortnite season pass. is what i would call like a companion subscription product so like uber eats has one uber has one of these doordash has one of these lyft has one of these where you're in a long-term use case but there's like a high ability to switch between providers
So they end up building these kind of like past products really as a retention tool. So like if I'm paying Uber Eats, I forget what their membership product is, but call it like $9.99 a month in the U.S. I'm way less likely to use DoorDash. And because Uber makes its money through the core business, you have a lot of flexibility in terms of how you price the subscription.
Yeah, totally. In that blog post on churn, you go through five kind of key areas of what drives churn. And so you already covered that first one of how long will users have the problem you solve. But the next one is.
how strong is your product market fit? Yeah, it's a great question. I think, like you said, the first thing that will dictate overall retention is how long do people want to be solving the problem that you ultimately solve? The second one is like, how good are you at solving that problem? Which I would call product market fit.
To me, there's two kind of ways of measuring product market fit. There's the classic kind of Sean Ellis hack and growth survey, where you survey a bunch of users and you ask what percentage couldn't live without the product. And you want 40% or higher, the people that couldn't live without the product.
i think that's really good for earlier stage companies when it's tough to know your retention numbers and you can survey people relatively quickly get data back relatively quick I think the better way of measuring retention is like trying to chart out the percentage of a cohort that keeps doing the same action. like month over month so for your app it'd be like what percent come back and check the weather x times per week and if you graph that like you're saying you probably see it flattened
Which means like for whatever segment of that user base that is, like you have product market fit. So the other thing that I keep in mind is there's degrees of product.
So there's kind of like this works for me and I like it, but if given a better alternative, I'll switch. Or there's this thing is the best thing I've ever used for this. And I think the way of looking at that is if you can... as best you can segment by persona or some concept or persona, you should ideally see like good retention for one of them, which means you're kind of on to something. Yeah, and using my weather app as an example, it kind of hits your earlier point about use case duration.
Even though it's a ridiculously competitive space, even though it's really hard, one of the reasons I keep at it is that weather is one of those indefinite continual use cases where people will check the weather for probably the rest of their life. And then for me, thinking through this, as you were saying that... you know, the fact that my long-term retention is in those like high single digits, maybe low double digits of eight to 10 to 12%.
that's probably a good sign for me of product market fit that I have some level of product market fit, but not great product market fit. If I had great product market fit because the use case is... with great product market fit should be landing closer to 20%, maybe even higher than 20% in that kind of long-term retention. And that's maybe a thing to look at for me to watch over time and for people listening to go look at your...
long-term cohort retention. We actually, in the revenue cat dashboard, this is where I always look at, of course, but we have that cohort of retention where you can see across all time. you know, what's that retention curve look like? And then you can go and cohort it and look at different years, different date durations and look at, is that improving over time or is that getting worse over time? You know, did a new feature change that in any meaningful way?
But yeah, I hadn't thought of that until you said that, that that long-term cohort is a very direct sign of the level of product market fit that you have. And if you're an early exchange company, like you might be at a long-term use case and it might take 14 months. to like see those curves flatten and if you're only at month six of building an app like that's tough but i think if you look at each cohort coming in you should see the early stage months like stack up to be slightly higher
That will depend on like where you get users from. Like it's a very common thing that companies will go viral and you'll see a flood of people come in, but they're not really your persona. And like that cohort retains terribly. So even though it's a good for the metrics overall, the question is always like of the people you're either paying to acquire.
building SEO content for acquire, using word-of-mouth funnels to acquire? Are they the right type of person? You always care about the long-term LTV. of the monthly cohorts that you retain. This goes back to our earlier discussion as well about not knowing what is and isn't successful for other companies. So when you see an app go like insanely viral,
And you're super jealous because you're still, you know, just running meta ads to get people into the app or whatever. And you're like, oh, my app could only go viral. Half the time, probably more than half the time, really isn't a great way to acquire users unless... You have those other things stacked up and we're going to keep going through them, but you don't get to see those numbers. You only get to see the TikTok that does, you know, 10 million views.
you don't get to see how many of those actually convert into paying users, how many of those paying users stay subscribed, and was that actually a good cohort? And the answer is often it isn't. So another thing where you can look at the numbers, but like completely misunderstand what's actually going on behind the scenes.
Yeah, you care about stacking up a user base that will pay you for a long time and pay you well. As best you can, you should be guiding product development and growth of the earliest stage indicators. So like onboarding success, activation success, certain personas, like segmenting audiences, and then feeding that back into whatever acquisition machine. So if you acquire a thousand users a month, but
15% of them are, let's say, coffee baristas. And coffee baristas want to know the weather for XYZ reason. And they're the best fit. There's nothing wrong with other types of people coming towards the application. But if you're going to spend money and time, you should focus on where you're going to get the longest term. Totally. And that actually brings us to the next point in your blog post.
is that how well you activate users is a huge part of retention. So I say after you have like a level of product market fit, the big question in activation is like how many people experience the value of the product? so it's like the classic milestones within product development to track this are kind of like sign up how many people register set up how many people like take the steps necessary to receive value from the product so that could be
setting up your profile or turning on location services so you can see the weather or something like that. Then there's kind of like the first aha moments of what percentage of people. sea value. So like I was going to go on a hike and your app told me it was going to rain and I didn't go on a hike or I thought it was going to be cloudy today and you notified me of weather. And that's like a great early, happy experience for users. And then first 30 days activation, then long-term retention.
As best you can, especially the early stages, you want to make sure people are experiencing like the win on the application. as fast as possible. I think if you look at any classic benchmark reports, especially around trials, like people make trial, whether I will or will not start a trial experience, like almost in session zero. probably within the first like 10 minutes on the application, maybe quicker.
So it's like being really clear in what you do and guiding people to do that in the first session is really, really helpful. There's things you can do around like lifecycle marketing that they don't in the first session. There's ways of guiding them back. But going back to headroom analysis. really good email campaigns have like a 30% open rate and like a net 2% click the rate. So it's like once you lose them on a product, it's tough.
The next one is payment processing. And for subscription apps, you know, more heavily reliant on the app stores, maybe that's less of an issue, but that does kind of encapsulate that entire conversion event of like, how many people are you actually getting to convert? How do you think about that? Yeah, definitely less of a problem for app store based.
So even though the 30% fee that Apple and Android put on applications, you do get a lot out of the box for that. You get a high converting checkout page. It handles currencies for you. It's really good at payment processing. It's really good at payment retries.
You do do a lot of stuff. If you build off the web, this is more your problem. So the number one thing I'm always looking at are the two things in payment processing rates is after someone clicks pay on your checkout page, what percentage of time do you collect them? So like almost no one tracks this the first time, but it's the highest intent people you can possibly get. a certain amount of those fails will be valid. Like their card won't have a balance.
But there's a surprising amount that if you look into the details, you can try to find ways of optimizing for. A classic one is like U.S.-based companies who charge around the world, and they still only use Stripe or U.S.-based gateway. So there's regions like India where like the US-based payment system is not as effective. Same with China. The bigger you are, the more kind of like headroom there is.
in like starting to set up secondary payment providers or getting a subscription manager. For app-based stuff and for smaller companies, this doesn't apply as much at all. But I think the philosophy of it takes so much effort to get someone to try to click pay. lose as many as few of those people as possible the second one is recurring payment success rate
So the classic things to do here is the classic dunning and email notifications when payments fail. I think, again, the App Store handles most of this stuff for you. I think in the App Store, you still might be able to detect this and customize some email.
to get people to update their payment method i've always seen again it's not a massive uplift but there's value in spending like two days copywriting those because this is going to impact a hundred percent of your people who fail payments and the lower your churn rate gets the more likely most of your churn is payment so like if you get your turn date all the way down to two percent fifty percent might be painted base churn
And for apps, a lot of apps are getting really good at when people click your CTA. And this is always kind of a tricky thing with the app stores and honestly, any CTA.
is that you can get more people to click a button that more obtuse when it's just start free trial and the actual price is hidden somewhere in the fine print you can juice the number of people who are going to click your cta but then when the app store sheet pops up and tells you the actual terms and the actual price, you know, how many people are clicking your CTA and then dismissing the payment sheet and not completing that.
And that's probably a multifactorial thing of the less clear you made the pricing on your paywall. probably the higher rate of people who are going to go ahead and cancel the actual payment because they didn't know what they were signing up for.
And then it's always kind of a balance of like, how do you balance how many people are clicking on it versus how many people are actually starting a free trial? I've actually been experimenting in my app and I haven't seen many people do this, but when people... click the CTA, and then cancel the payment.
I actually pop a little modal now that gives them a non-recurring free trials, a reverse free trial. So I offer a reverse free trial to customers where they get seven days completely free that doesn't renew. And I haven't seen a ton of success with that. So I'm not going to say this is like a magic bullet, but I think those.
kind of experiments or what you're talking about is that when somebody clicks your cta that is high signal again maybe lower signal if your cta is confusing or not clear on the price and things like that but it's something you want to be experimenting with and that's a really great place to experiment a lot of apps now When you dismiss the payment sheet, they'll offer a discount immediately and kind of try and win you back that way immediately.
And that can be incredibly effective. I mean, a lot of apps are doing that now. Although I have heard Apple has been hassling some developers about that, that it's kind of a form of price discrimination. Which is understandable if you're just randomly giving people 50% off because they dismiss a payment sheet.
you know, that's maybe not the best customer experience, but it can be incredibly effective. And so it's kind of goes to what you're saying is that that payment section, whether on the web or in the app is a place to... spend a lot of time and be thinking a lot about. The last one on your list is how good are you at winning people back once they leave? So you kind of already touched on it that this is actually really hard and maybe that's why it's at the bottom of your list.
But what are some of the strategies you've seen be successful here? And then... So realistically, what do you think people can expect? And then kind of back to the whole ROI discussion, maybe the ROI on this is so low that especially for an earlier stage app, when the numbers are just low, it's just not a place to invest a lot into.
If you look at really big subscription companies like the New York Times is one, like Clear is another one, the U.S. travel-based app. In their cancellation flows, they're always taking every shot they can to try to win you back, regardless of what you click on the surf.
So if you click like it's too expensive, they'll give you a temporary two month discount. Zoom does this too. If you click on like, you know, I had a negative experience, they'll connect you with customer support. If you click on like, you know, I'm too busy right now, they'll let you pause the subscription. I'd say this tactic set. he's not the cheapest to implement because typically it involves like bespoke logic in your payment process
So if you're web-based and you use Stripe, I don't think Stripe has any out-of-the-box features that help you here. So you have to custom manage all this logic. But there's a reason all the giant companies do it. I've implemented this a couple of times across a couple of companies. I'd say the big three that I've seen work is pause, like discount and kind of like connect with support. Pause works the best if it's a temporary hack.
so we implemented this code academy like people don't want to learn to code every day of the week unlimited they go through like periods of learning so if you let them pause and let them come back when they're less busy they're more likely to retain in the long term
Like if my mortgage offered pause, I don't think anyone would use it because you're like, you need your mortgage or you don't. Like the underlying need doesn't really go away. Same with discounts. I'd say the two discount sad strategies that I see work is like.
temporary discount so we'll keep you in the same plan structure for three months at 50% off and then we'll auto readjust you back or like will drop you down to a lower like hidden tier of the product so when you go to at least when i went to cancel zoom i think i was paying
15 bucks a month for zoom and they offered me like a seven dollar a month plan but it's only five calls the calls can be longer than 40 minutes or something like that so it's instead of losing you completely they let you shift down i think the risk here is that you should keep an eye on the percentage of your users that go for those discounts like most what i would call like growth tactics they get widely known
So years and years ago, it became known in the internet that if you went and added a bunch of stuff to a checkout cart and then didn't purchase out, most of those companies would then just email you a coupon code to bring you back. And that like works until most of people know to do that behavior and just everybody recoups the discount. I think this tactic set is going to go in the same direction. That said, when I've implemented this, I've seen between like a 10 and 20% drop in.
Depends a little bit on the use case, but like it adds up to be material. There's a reason all the giant companies do this. It's not the world's cheapest tactics to implement. but like definitely, you know, rounds the sharp edges off your churn number. The complexity piece is a tough one. I mean, you know, we don't talk about revenue a ton on the podcast, but this is something we're trying to solve right now, specifically with our customer center product where.
you can actually drop a button into your settings or wherever you want to put it of like manage subscription or whatever. And then when people go to cancel, you can initiate a survey. And so by building out this tooling, we are trying to remove some of that complexity. That's maybe a point for look for the easy wins in those flows and look for tooling and other things that can help you get those easy wins. Versus spinning up a whole project where you're spending months on this.
as we discussed earlier in the podcast, where there's so much other low-hanging fruit that you should be focusing on, that this is maybe not where you should do a three-month massive project to build out this really big custom internal tool until you're further along.
where the number of people you get back will be enough of a meaningful impact to justify that three months of building and whatnot. Yeah, definitely. I think it's another lesson from 10 plus years in product now. It's like the complexity tax is real. It's like the North star of your development should be protecting like future velocity.
So it's like you can do things now that make things incrementally more effective, but you have to manage this code base forever. So a classic tactic that every subscription company eventually does is geo-based pricing.
So you realize like the Nordic countries have the highest willingness to pay along with like Western Europe and the US and Canada. And, you know, there's a second tier of companies that can pay less. And then there's like a third tier of companies that will pay the least. And you start to like localize price level. in addition to currency and like what appears on the page. That tactic almost always works in making your total user base produce more cash.
but also you have to then manage 10 price packages forever. So every time you list pricing in your ads or in email or in your customer support docs, you now have to manage this complexity forever. So it's like, it's effective, but... you should squeeze, in my opinion, the easier tactics out first. But, like, we did it at Code Academy. It's effective. But, like, you kind of can't unring that bell.
And now when you do price changes, you have 10 price tiers to think about. And currencies change in fluctuation to each other. So you're probably pegging the FX rates initially. But then like, how do you manage the change? It's just another thing you have to pay attention to. Yeah, there's so many things, you know, as an indie, I mean, I've been very fortunate just working on the app stores of so much of this complexity is abstracted away by the app stores.
But I haven't done regional pricing yet. And part of it is exactly what you're saying. And this is where... It's tough to manage that complexity over time. Even price testing on the app stores, you end up with a bunch of different SKUs and you can different... end up with different subscription groups and it can just become a huge mess and you got to factor that into the the roi calculations yeah how much complexity does this add
to the business long-term, even pricing tiers. We've talked about it a lot on the podcast. It's great to introduce a new pricing tier. But that's a lot of complexity that now you have to support forever. Multiple tiers, what features get unlocked, what features don't. But it's worth doing. It's just figuring out the right time to do it in the stage of your business and everything. Yeah, exactly.
Code Academy introduced a tier above our normal tier after I'd left, I think. So I think we were already like north of 50 million a year before we hit a second tier. You can build as many pricing tiers as like you can make great profits. But like the distance between good and great is way longer than people think.
So it's like your monetization systems as a whole are really just like a measure of how much of the value your product produces that you capture in money. So it's like typically the best thing is you make your product as strong as possible. Then you figure out how much you want to monetize. but i think it's really easy when you start making money to make the monetization system the goal in itself and lose track of like the you just collect a percentage of the value people feel in your
And there's definitely tactics and ways to make that more effective. But if the product isn't increasing in value, it's really tough to raise your price. Well, I think that's a great place to wrap up. It all does ultimately come back to product. You have to build something that's actually delivering value for folks. And then especially as a subscription app, you have to create products that deliver value over time or people just.
aren't going to stick around. Well, anything else you wanted to share as we wrap up, we will share links to your blog and we'll share links to the two specific blog posts that we referenced today. But yeah, anything else you want to share as we wrap up? I think that's the main thing. Like, you know, I think in summary, like we learned a ton of hard lessons at Code Academy. It took us five plus years to figure all these things out that we just talked about in an hour.
I think, you know, what I'm trying to do is write as many of them down as I can on my blog at subscriptionindex.com. I'm in a place where I'm like not really scaling the consulting business because I'm kind of at capacity. So I'm literally like writing down everything that I see to be effective. on the blog so hopefully people going through this journey now don't have to you know take all the bumps and bruises that we did.
Awesome. Well, thanks for sharing some of those learnings today. And it was fun to get the more nuanced take. I mean, you can read those blog posts in the past, re-read them leading up to this recording. It's great to get the kind of behind the scenes. I read those in like 10 minutes.
And so, you know, maybe 15, 20 minutes total reading of the two blog posts, maybe less actually, but we talked about it for over an hour. So there's just so much to it. So thank you for coming on the podcast and sharing the more in-depth from the blog post. Yeah, of course. Happy to. Thanks so much for listening. If you have a minute, please leave a review in your favorite podcast player. You can also stop by chat.subclub.com to join our private community. Welcome to the Sub Club Podcast.
a show dedicated to the best practices for building and growing app businesses. We sit down with the entrepreneurs, investors, and builders behind the most successful apps in the world to learn from their successes and failures. SubClub is brought to you by Revenue Cat. Thousands of the world's best apps trust RevenueCat to power in-app purchases manage customers, and grow revenue across iOS, Android, and the web. You can learn more at revenuecat.com. Let's get into the show.