Weekly-ish Update (June 30, 2025): Preparing to launch private podcasts, marketing stuff, huge bill from Google

All right. So welcome to the, I don't know if we can call it a weekly update at this point. It's been two and a half weeks since the last one. Weekly-ish. I guess welcome to our update. Let's just call it an update, a short update. traveling right now probably that's also one of the reasons we'll probably do weekly once you're back

That's true. Yeah, that's true. Yeah, and I'm like really short on time because the cleaners will come to Airbnb very soon. So yeah, let's get going. And I think we should start with the thing that we worked on together last Saturday, because that is... Big one. Yeah, I don't even remember what happened in the last two weeks because what happened in the last two days has been basically consuming both of us full time.

Yeah. You want to talk about it quickly? So we're recording this on Monday morning, right? Monday the 30th.

On Saturday, because I'm on the East Coast right now, and I think around like 8.30, 8.45 or so, I got up and I look at my email and I'm seeing that we've breached our budget. on the Firebase, Google Firebase. I'm like, what's going on? I just go to metrics, and I'm seeing a spike that looks like a nightmare spike. So it's like our costs.

that are normally at like very, very low numbers. And there is like, I don't know, 50 fold increase in costs. And the total number is something like $2,000 in just last day. And I'm like, oh my. God, I should not swear. And I'll tell you why later. But yeah, but I wanted to swear now, but I didn't. Because it was like that emotional reaction. So and I ping you on WhatsApp and it's like six o'clock your time. And somehow you happened.

to be on your phone on Saturday at 6 o'clock. So we started looking at it together, and we realized that something is happening in our backend. We're just accumulating costs. Like crazy, literally like hundreds of dollars per hour. I guess long story short, we were right post-mortem about what happened. It took us, I think, a couple of hours to identify what the problem was and then stop the problem. So we stopped accumulating costs.

I would say we stopped the problem almost immediately and then identified like what the heck was happening. We get the timeline. I think we detected the issue at like 6.15 Pacific. And we stopped at about 7.25 because that's when we see the impact stopping. Stop. Yeah. So in the typical sort of Amazon fashion, we documented everything. Then we shared the doc with Google. Now Google also asks us for more information so they can take the charges off the bill, which is really great of them.

Yeah, a public post more than was coming. I think people ask for it after your LinkedIn post. They're like, tell us what happened, tell us more. Yeah, so we'll do that. And I don't think they want it in this sort of shut and friday way. I think they actually want to understand what happened to... to us so that you can avoid that. Alright, so what's up with the app?

On the app side over the last two and a half weeks, we are preparing to graduate the private podcast from the early beta that we had, which had a limit of one private podcast only. And so now I've added some sort of like transcript quotas and stuff like that. We'll document all this before we release the next version. So those things are in the beta. The unfollow or deleted episodes, like if I do podcaster. deletes an episode.

from a podcast that UX, or if you unfollow like a private podcast, but you have episodes from it in your listen later, you can no longer access it. The UX for those is much more informative and better. minor fixes for UI in descriptions and all that so that's all in the app right now cool you're on my side I have not done much in terms of user-facing. So we added chapters to transcripts on the web app. So now if the episode has chapters, then they are interspersed with the...

Transcript that is pretty nice and user facing one user told us already. It's nice. I mean, it took like a couple of hours to do for the rest of my time, because I'm traveling, so I wasn't as productive as usual. But also I've been dealing with a bunch of like ops stuff, like we are having issues where we're seeing that some users

In particular, Arnab, because he's in Canada. Sometimes the web app is broken. The style sheets are broken. And we have a theory that this happens because of caching across regions. So we probably have to do something about that. To begin working on this, we added monitoring, so we can see when this kind of thing happens. And we can address this in a more data-driven way, as opposed to just jumping into the project. And then they also removed cookies from Google Analytics tracking.

I never liked those cookie disclaimers. And I feel like we're not really gaining much by collecting more data about users. So we were like, yeah. Screw it. Let's just get rid of them. The reason why we don't show the cookie banner is because we don't collect cookies, which is a really nice way to say this. This is how it should be. This is how it should be, yeah.

Then also I've done a bunch of work on marketing. We've created assets for our brand, like our logo and like a badge to listen on Metacast and stuff like that. So that other... Sites can link to us more easily. And we've been working with James Cridland from PodNews. It's a newsletter for podcasters. He also has a website. He added links to us because now we have those easy ways to link to us.

We also got added to a repository of app links and app badges, you know, stuff like that. Quick teaser. This is what led to the GCP bill craziness. It's not the problem. But this is what basically surfaced the bug that was there in our site. Yeah, because we got out of the pod news, it triggered the volume that triggered the condition that, you know, I don't know about James' sense of humor. If he was like our friend, I would just say, yeah,

It's all your fault. James, in case you're listening to this, it's not your fault, but it actually helped us identify the issue that we had. Actually, imagine if this happened at a larger scale. It could have been tens of thousands of dollars. So, yeah.

Yeah, we also are working with Transistor and Buzzsprout to add our icon to their websites. It'll probably take a while, but basically that's how we spread awareness. And so it's not a very interesting work, but it's necessary work. Yeah, also like... I wrote the blog post for the previous version. That was what, the V121. Also, every once in a while we send emails to...

users who have signed up with email addresses. So we send welcome emails to people who just recently joined. We send emails to people who churned. And we also send emails to people who seem to be making good use of the app. The content of the email is different, but basically we ask for feedback. And none of this is automated right now. It's all like you sending it. Yes. Like a template, but still, this is what you're sending it manually.

Yeah, it's runbook and mail merge and Gmail. It's kind of nice to, like we received a few responses back now and they're very sort of, actually pretty long, thoughtful emails, which is really cool. All right, so what's up next? All right, so I think we still have to... get over the hump of the google cloud bill they have asked some more questions so we need to do that in order to like basically take this off our bill i suspect it will probably

take a few more calendar days but in terms of work i think most of the work on our side is done it's now just digging up some data that they're asking for and yeah for the app i think 1.22 we'll try to release it I would say maybe early next week, maybe this week, we'll see. Private podcast. So I don't know if we'll make them non-beta, but at least you'll be able to follow. If you're a premium customer, I'm forgetting up to 10.

private podcasts if you're a free user up to three private podcasts we'll document more about like exactly what is included but yeah we're preparing that build and release hopefully later this week or early next week yeah And then for you, Ilya, for the website and stuff? Yeah, so when we release the 1.22, I'm going to write up about that, the usual release stuff.

Then I'm working on the banner. Basically, it's like, you know, if you're old enough and you use the internet in the late 90s, like the websites used to have those banners with ads in their websites. We did something like that in our website. podcast and episode pages that tell people that podcasts are better when you download the mobile app. Because kind of frankly speaking, right, the only reason we have the web app is so we can get more users for the...

for the mobile app right now, right? Because it's not a full fledged app, it's more like a website. And we want to convert that to users more. So very pragmatically, we are doing work to add that. ad, if you like, and kind of instrumentation for that so we can drive more app downloads and grow more. I also hope to, because we are probably not going to record an update next week, so hopefully in the next couple of weeks I'm going to...

add the transcripts provided by creator to our backend. So we can start using the creator provided transcripts and not just AI transcripts like we do now. And there's a bunch of ops work that needs to be done, like those issues that we report from the web app, they seem to be a little bit noisy. So we need to figure out what's going on there, debug them. We also need to add some more monitoring to other parts of the stack that usually we mentioned about caching. So we'll see how that behaves.

in monitoring and maybe we'll have to address that. And because of the incident, our analytics became unusable because of just so many events going into our BigQuery. tables, so we have to drop them and recreate them so that we reduce the data size. We can query the Yeah, we are talking about the BigQuery views, right? The raw data we can still thankfully query, but it's an insane volume of data in there now. It is very expensive right now to query it.

Yeah. And I think we're also paying for the storage now. I think we should probably drop the tables today. Maybe after we get the build resolution from Google so we still have the data. But yeah, that... It's the first time I see that, where BigQuery just craps out. It just tells us, I can't query these data sets. It's too large. Out of memory. I'm like, wow. And we aren't even... What do super big companies do?

They get on a call with their enterprise TAM or something and the TAM immediately fixes them. Or maybe there is some setting that you have to set to allow for the memory sessions more expensive. Yeah. All right. So, and to conclude this, let's do the recommendations. Yeah. Let's do podcast recommendations. You've got 30 seconds. Start the timer. I got 30. Okay. So Ilya has been talking about Josh Wade skin and from, from.

place to I heard so I finally bunker down and i'm going through his book art of learning it's very inspiring so far and then the second one i'll say is dennis taylor we had on our episode three early on is my favorite like sci-fi author Bobbyverse. He has a new novella called Flybutt Out on Audible, so I'm listening to that. All right, Ilya, your timer starts now. Okay, so I listened to the My First Million podcast for the first time.

Because I do want to make my first million ASAP. Yeah, it was a great podcast, actually. I listened to it in the car with the rest of the family on a road trip. It was an interview with Chris Korner. It was so fun actually to listen to the whole thing with my kid. My kid really liked it. I'm so grateful that they didn't swear. That's why I promise to not swear in podcasts anymore.

Maybe you can flesh out this part of your recommendation in the next time we talk or in the larger episode that we have. Yeah, yeah. Yeah, I guess the point is kids do get exposed to podcasts, like a secondhand exposure, right? And that's why, like, if you don't have to swear, don't. And like, it didn't really occur to me until I was in a situation where I'm like, well, thanks God, they didn't drop any F-bombs in there. Right, right. All right. See you soon. Bye.