Kaizen! NOT a pipe dream (Friends)

Jerod Santo:

We are kaizening once again. Kaizen 15, and Gerhard, you will be pleasantly surprised... This is Friends \#50. This will be our fiftieth episode of Changelog & Friends.

Gerhard Lazu:

Wow... Look at that. No way.

Jerod Santo:

I know you like round numbers and milestones...

Gerhard Lazu:

I do, I do.

Jerod Santo:

Yeah. So you're gonna love this. No random integer here... Nice, round 50. So here we are, Kaizen \#15, Friends \#50, Adam's here, Jerod's here, Gerhard's here.

Gerhard Lazu:

So good to be back. I was really looking forward to this one. We have a couple of things in store... And I'm not bigging it up... You will be impressed. \[laughter\] By degree.

Jerod Santo:

Alright, the gauntlet has been thrown. We'll see. Let's see if you impress us this time around. Maybe I'll impress you guys.

Gerhard Lazu:

I know you will.

Jerod Santo:

Okay. That's easy then.

Gerhard Lazu:

Yeah. You've already impressed me, Jerod.

Jerod Santo:

Oh, good.

Gerhard Lazu:

I have been paying attention to that repository. You have impressed me. \[laughter\]

Jerod Santo:

Thank you.

Gerhard Lazu:

So good to see you, Adam, as well.

Adam Stacoviak:

I'm glad to be here, of course. I just love the ride. I love the ride.

Gerhard Lazu:

Cool. By the way, just a heads up, we will be talking about bidets towards the end. So heads up. It'll be very short, but there will be a short bidet conversation.

Adam Stacoviak:

Oh, you have thoughts on bidets as well.

Jerod Santo:

Oh, bidets. I thought you were talking about birthdays; like, you shortened it to B-days.

Gerhard Lazu:

No, no, no.

Jerod Santo:

Oh, is that how you pronounce, bidets?

Gerhard Lazu:

It's just how I pronounce it.

Jerod Santo:

Alright.

Gerhard Lazu:

I'm glad that you know what it means.

Jerod Santo:

Okay. So you listened to our friend's episode with Kelsey Hightower, I assume...

Gerhard Lazu:

Yeah, I really enjoyed it.

Jerod Santo:

Alright. So I'm ready to learn, because I've never used one, I've never even seen one in the flesh... So let's save that to the end, because we have more important stuff to talk about. Alright, take us on a ride, Gerhard.

Gerhard Lazu:

We will start somewhere, which is I think -- I think it's related to Jerod, I think. But maybe it's Adam as well. So as you pointed out, Jerod, I do like round numbers. I do like dates. I pay attention to those things. So one of you will know what happened on June 21st, two years ago. Something happened this day, two years ago, and it's related to Ship It show.

Jerod Santo:

Was it your last episode, episode 90?

Gerhard Lazu:

No.

Jerod Santo:

No, that was in January.

Adam Stacoviak:

You're stumping me.

Jerod Santo:

I don't know. Related to Ship It show in June... Was it the original ship of Ship It?

Gerhard Lazu:

So anyone with a terminal can verify this.

Jerod Santo:

Oh, gosh...

Gerhard Lazu:

You can do whois shipit.show.

Jerod Santo:

Oh, we acquired the domain maybe.

Gerhard Lazu:

We created it. 21st of June, 2022. That's when ShipIt got --

Jerod Santo:

I don't know how you know that. How do you know that?

Gerhard Lazu:

Just random. I was paying attention to things.

Jerod Santo:

Do you have like a list of dates that you put in your diary, and you're like "June 21st is now important to me, because shipit.show..."

Gerhard Lazu:

No, I just stumbled across it. I just stumbled across it. That's how it happened. But I thought it was very interesting that on this day, two years ago, shipit.show became a thing. So the actual domain.

Jerod Santo:

Gotcha.

Gerhard Lazu:

So that was nice.

Jerod Santo:

That is nice.

Gerhard Lazu:

Cool. So Kaizen 15.

Jerod Santo:

Tell everybody, so we're all on the same page, for those who haven't listened to Kaizens 1 through 14, what it is that we're doing here with Kaizen, why we're doing it, and then we'll get into it.

Gerhard Lazu:

Okay. So Kaizen stands for continuous improvement, and that's an abbreviation that many are familiar with, but change for the better. I think that's what the translation is closest to. So the idea of kaizen started three years ago, almost three years ago, by the way; and we will get to that. Three years ago, I think it was summer. The first Kaizen ever, Kaizen \#1. Ship It show episode 10. Shipit.show/10 and you'll see it. And it started with Fastly, it was summer, and it's when half the internet went down. That's what happened three years ago. \[00:05:58.06\] So the idea behind Kaizens was to improve things about the Changelog setup, on a regular basis, and then talk about it. So the pace was every 10 Ship It episodes, and it just happened to be every two and a half months. And for the past three years, every two and a half months that's what we have been doing. And it's been a lot of fun.

Adam Stacoviak:

I did a little digging into this extended version of the meaning of this. It says, as we know, Kaizen is a Japanese term; it means improvement, or as you'd mentioned, Gerhard, change for the better. And it says "In the context of business, particularly manufacturing management, it refers to a philosophy or practice that focuses on continuous improvement of processes, products and services." And here's the part I like. It says "The concept of kaizen involves all employees, from top to bottom, all of management, to the assembly line workers, and it encourages a culture of sustained, incremental improvements." I think that encapsulates a bit more, because it involves everyone. It's not like you, or Jerod, or just me... It's a collective, and it's this culture, which I think is interesting, because Kaizen has -- you know, most of the pillars, Jerod, of our business are built by you and I, Jerod. And then here comes Gerhard, with Ship It, and all the things he's been with us for the past - I don't even know how long, but pretty much forever... And he brings in this new pillar called Kaizen. And I think this is very much Gerhard-born, but adopted by all... And I'm really thankful, because Kaizen has become part of my own personal DNA as well. And I like that.

Gerhard Lazu:

That makes me so happy. I'm really glad that three years after we started, we just add another break. You're right, it does include everyone. We've always done this as all three of us. We've never done it like separately. So from the beginning, this has been the secret formula, and there's been a lot of navel gazing. \[laughter\]

Jerod Santo:

That's a fact... Which is something that I generally am allergic to, and I try to avoid.

Gerhard Lazu:

But actually, do you know what it means? Do you know what navel gazing means?

Jerod Santo:

Well, it's a stare at your own bellybutton.

Gerhard Lazu:

It's the contemplation of one's navel as an aid to meditation. So you're contemplating the basic principles of the cosmos and the human nature. Check Wikipedia. Omphaloskepsis. It's Greek. That's where it comes from. Navel gazing. \[00:08:28.29\]

Jerod Santo:

Jerod here, in the editing room. You know, I thought omphaloskepsis sounded familiar when Gerhard said that, but I couldn't quite put my finger on it. Now that I've had more time to think about it, omphaloskepsis was our word for round three in the \#define game show we played on episode 25. Maybe you recall it, "Game theory, dude." I'll link to that chapter from this chapter in case you want to jump over there and listen to it before returning to our navel gazing. Okay, back to the gazing.

Gerhard Lazu:

And there's lots of statues of Greek dudes, very muscular dudes, great-looking, gazing at their navels.

Jerod Santo:

Yeah. Similar to us.

Gerhard Lazu:

Similar to us, exactly. With more hair. They have more hair.

Adam Stacoviak:

They do have more hair.

Gerhard Lazu:

Yeah. So I think Kaizen is a better name. I think -- yeah.

Jerod Santo:

I like Kaizen.

Gerhard Lazu:

I quite like that.

Jerod Santo:

Yes, I do as well.

Adam Stacoviak:

Let's put a new spin on navel gazing though for me, because it's generally been like a frowned-upon term...

Jerod Santo:

Well, it's something that you do for yourself, by yourself. And so navel gazing I think in public is where it becomes frowned upon. You can stare at your own navel all you want, and if it helps you with your thought process and your self judgment and all that kind of stuff, go for it. But navel gazing is a private activity. And when we do it in public on the airwaves, at a certain point you're like "Let's stop staring at our own navels." But insofar as it's continuously improving our navels...

Gerhard Lazu:

Better mine than someone else's... That would be weird. \[laughter\]

Jerod Santo:

I would have to agree that yes, navel gazing outward is worse than inward. So at least we aren't going that far. But I do agree that Kaizen is a much better way of framing it, so it feels okay.

Adam Stacoviak:

\[00:10:08.00\] Here's maybe a better twist on it. If you're gonna navel gaze, turn it into a product. Like, this is a product. Kaizen is a product. We produce something as a result of navel gazing, and we're doing that, but we're doing it in the manner, in the culture of Kaizen, continuous improvement, improvement for change, change for the better... But at the same time, we've turned this sometimes habit or circumstance of just wanting to popularize the things you're doing, and not be overly self-promotional. And we've always been a little guarded with being overly self-promotional. But I think that there's been so much value in all the Kaizens, in all this navel-gazing, Kaizen, change for the better stuff that I think I now see a positive style and a positive light to what is typically determined as navel gazing.

Gerhard Lazu:

Yeah. No, I do think that we've been sharing and promoting a lot of the work that the community has been doing from the open source, we've been promoting projects, we've been helping some of our partners improve themselves... Whether they wanted to or not; we had some of those as well.

Adam Stacoviak:

That's right.

Gerhard Lazu:

The idea is we've always been outward-looking, and we try to make everyone that integrates with us better in some shape, or form. And that makes me really happy.

Jerod Santo:

Well, while we're navel-gazing navel gazing - we've gone up a level, or down a level, depending on which perspective you have - I want to talk about continuous improvement versus continuous change. Because while we bikeshed and talk about the details, I think there is a culture of change for change's sake, which we could fall into, and perhaps have, and sometimes you don't know what is improvement and what's actually steps in the wrong direction... So I think we should ask ourselves, as we move along, are we just continuously changing for change's sake? Or are we actually improving? and that's part of the iteration process, it's also part of the judgment, the retrospective, how is it going, was this a change for the better...? Similar distinction I make between productivity and effectiveness. I think that you can produce a bunch, but really, what you want is to be effective in whatever you're doing. And sometimes you're just producing. And it's like, well, cars produce as well, carbon dioxide, or whatever... So not always the best thing. So instead of trying to be productive, we should try to be effective. And I think instead of trying to continuously change, we should make sure that we are continuously improving... Which I readily admit you can't know that until you make the change, and then you can judge it. But something maybe as a Northstar as we move forward is like "Is this change actually for the better, or just for the sake of trying something?" Alright, let's stop navel-gazing the navel gazing. Let's start getting to the standard navel gazing now.

Gerhard Lazu:

Yeah. So I have a couple of items on my list, and there's like three buckets. So what can you expect? You've just joined, or maybe you've skipped forward through the navel gazing part...

Jerod Santo:

Yeah, chapter break...

Gerhard Lazu:

And then you just came and landed here... So what can you expect in this episode? The first thing, I'd like to give kudos to some people in the open source community that did some great work. I would like to share some fly.io love. There's a lot of fly.io love to go around. We'll get to that in a minute. And also, I want to talk about turning pipe dreams into reality. And that's an important one. So, rules for today... We'll share the background story. We have lots of stories to share between three of us. We will share those stories.

Jerod Santo:

Okay.

Gerhard Lazu:

No peeing in pools.

Adam Stacoviak:

Gosh.

Gerhard Lazu:

Remember the last one? No peein in pools, okay? \[laughter\]

Jerod Santo:

Alright...

Gerhard Lazu:

And most importantly, we want to celebrate the wins. We had quite a few. Navel gazing is allowed, by the way; you can do it, it's okay. And you can't see mine, so it's all good.

Jerod Santo:

That's right.

Gerhard Lazu:

So the first thing, the first person that I would like to congratulate is you, Jerod.

Jerod Santo:

Me?

Gerhard Lazu:

Yes.

Jerod Santo:

Well, let me have it. Why? What did I do?

Gerhard Lazu:

You made the first PR in nine months.

Jerod Santo:

\[laughs\]

Gerhard Lazu:

I noticed that.

Jerod Santo:

\[00:14:10.26\] I did. I did that kind of for you, I did that kind of for me... I was like "It's been a while..." I normally just push straight up on the master branch. But I thought "This is a noteworthy change. I should open a PR, and Gerhard's gonna love this."

Gerhard Lazu:

I did. Oh, man. I loved it so much.

Jerod Santo:

Kinda pandering...

Gerhard Lazu:

So 510. Pull request 510. If you go and open up GitHub, our GitHub, changelog.com, "News redesign." Oh, yes, baby. That was amazing. By the way - "baby", I'm only using it as an endearment. The fact that it took nine months has nothing to do with it, okay? This is not your baby. It's a labor of love. It's a labor of love. It's passion, commitment, but it's not a baby. So anyone with this thought, I want to discourage it, okay?

Jerod Santo:

Okay, good. Yes.

Gerhard Lazu:

So I would love us to talk a little bit about it, and I would like to start -- anything you want to share, any of the background stories, any of the things that you want to share about the news redesign, Jerod?

Jerod Santo:

Yeah, absolutely. So the background story is that we had a newsletter for many years called Changelog Weekly, which you would subscribe to on Changelog.com/weekly. We also have a nightly newsletter, which is still going, which is an automated scraper of the most starred repos on GitHub, which is increasingly becoming simply AI, crypto, and malware. So it's getting to be less useful than it used to be... But that one goes out weekly, it went out weekly, and that's why he named it Weekly. Right, Adam? We had a hard time coming up with a better name for that, because well, we already had Nightly, and Weekly, and... And it was what it was. And then we started the Changelog News podcast as a flavor of the Changelog... Which was quite a success, a hit. People liked it, we like to do it, so we kept doing it, and we decided to streamline the newsletter, and the podcast, send them out at the exact same time... They were already complimentary; let's just make them more efficiently complimentary. And so Changelog Weekly became Changelog News, which was just both a podcast and a newsletter. People who are listening to this pretty much already know that. The technical side of that was that the podcast became its own podcast in our system, and had a podcast page just like all the other ones, just like Ship It's page, just like JS Party's page, etc. But it had this newsletter component that wasn't very well featured, basically. So when you'd go to changelog.com/news, you would find the Changelog News podcast, and there'd be a subscribe button with newslettery things... But we weren't featuring the newsletter at all. And so this redesign was an effort to have a different page that lives on changelog.com/news, that serves both the podcast and the newsletter better, in terms of people understanding what we're doing there, being able to quickly read the most recent issue... You know, give it a better splash page. Because lots of people land on that page, and we want them to know what it is. And it's more than just a podcast. So that's the backstory. A couple of shout-outs. Of course, Cody Peterson, who worked with me on the design and the frontend implementation of the new splash page. Shout-out to me for coding it up and opening up an epic PR, which I don't normally do... And to a handful of folks who decided to send us kind words to use on the page. We have a lot of people who enjoy the podcast, enjoy the newsletter, and they tell us this privately via email and stuff, and so I reached back out to everybody who reads it and said "Hey, if you would give us a quote, we'd like to put some of those on the homepage." And so shout-outs to Mary, to Justin, to Chris, to Maros, and to \[unintelligible 00:17:57.09\] whose quotes were awesome. And we got more than those, so thank you to everybody who sent one in. Those were the five that we selected to feature on the homepage. And that's pretty cool. \[00:18:15.23\] Being able to hear from and see some of the people who take value in the newsletter I think is powerful when you're trying to decide -- like, two things you want to know. A, do people like this? And then B, what does it look like? Show me a recent issue. And I think with this redesign that we've accomplished that. So that's that.

Gerhard Lazu:

Very nice.

Jerod Santo:

Coded it up, rolled it out... It's there, it's ready.

Gerhard Lazu:

So I really love the emails. It's one of the few newsletters which I read. Whenever you send the newsletter, Jerod, I really enjoy it. I have clicked on the View in Browser button, and I noticed -- I think until about 90, like issue 90, it had previous design. That was \[unintelligible 00:18:58.29\] side hustle expose. That was the last issue, which had the new design. And then, 91, it was the new one. So all you have to do is literally drop the email part, and you'll see the previous design. So news/94, /89, so on and so forth.

Jerod Santo:

Right.

Gerhard Lazu:

And the reason why I said 89 is because maybe you've done it by design, maybe you knew what you were doing in this case...

Jerod Santo:

What did I do?

Gerhard Lazu:

...the previous issues have also improved. So when you ship the new redesign, all the previous ones look amazing.

Jerod Santo:

Oh, thank you. We did it on purpose. We did them all.

Gerhard Lazu:

You've done it on purpose. \[unintelligible 00:19:36.15\]

Jerod Santo:

Yeah, we did it all.

Gerhard Lazu:

If you haven't seen that link in the email, click it and check it out. Otherwise, changelog.com/news/90.

Jerod Santo:

Yup.

Gerhard Lazu:

So... I have a question, and I'll go first. Okay?

Jerod Santo:

Okay. That always gives us time to think.

Gerhard Lazu:

Exactly. What do you like most about the new redesign, about the News redesign? What do you like the most? I will go first, and you'll understand why in a minute. So my favorite feature of the new redesign is the 27 more reasons to subscribe.

Jerod Santo:

Oh, you like that part?

Gerhard Lazu:

I love that. In particular reason 15. \[laughter\]

Jerod Santo:

Okay, which says...?

Gerhard Lazu:

Who else takes the time to list 27 reasons to subscribe?

Jerod Santo:

Right?!

Adam Stacoviak:

That's right.

Jerod Santo:

I thought somebody in the world would get a laugh at that, and I'm happy to hear that was you.

Gerhard Lazu:

I did. That was so good.

Adam Stacoviak:

That shows you read all 15, or all 27, because you got to 15. Yeah.

Jerod Santo:

Did you make it all the way through?

Gerhard Lazu:

I did.

Jerod Santo:

Okay.

Gerhard Lazu:

And I improved it. And I'd like to show you how. \[laughter\]

Jerod Santo:

Okay.

Gerhard Lazu:

28.

Jerod Santo:

So are there 28 reasons now?

Gerhard Lazu:

Well, let me share my screen... I will obviously narrate this, and you'll understand why. Okay, so what I'm sharing right now is Changelog.com/news/99. So if you go in a browser, you can see this. "27 more reasons to subscribe", love it. Reason 15, amazing. So how would I improve it?

Jerod Santo:

I don't know. Oh, 29 more reasons to subscribe... Okay. I like this.

Gerhard Lazu:

29 is important to me, right? It's only my birth date.

Jerod Santo:

It's a very important number. Now, did you add two reasons, or...?

Gerhard Lazu:

I did.

Jerod Santo:

Okay.

Adam Stacoviak:

Oh, my gosh. Scroll... Scroll...

Gerhard Lazu:

Yes! He noticed it! \[laughs\] So what I did -- hang on, let me just make this a little bit bigger... What is different about this list?

Adam Stacoviak:

Well, it's got zeros in front of it. Or did it have zeros already?

Gerhard Lazu:

No, it did.

Adam Stacoviak:

Okay. My bad. Well, it's a --

Gerhard Lazu:

It's a pyramid scheme. It's using a pyramid scheme.

Jerod Santo:

I like that.

Gerhard Lazu:

I reordered them.

Jerod Santo:

I like that, because I've actually done that in the newsletter before. Have you noticed?

Gerhard Lazu:

I know you did. That's why I paid attention to that.

Jerod Santo:

Okay, I love that. Yes, it gets longer and longer as you go down. So you've reordered them.

Gerhard Lazu:

I did. So I'm going to submit a pull request, and you let me know if you like it, okay?

Jerod Santo:

I will accept this.

Gerhard Lazu:

\[00:22:05.19\] Cool. Okay. And for everyone else, there'll be a new pull request. I don't know the number, but by the time this comes out, I'm sure we'll have a link.

Jerod Santo:

So funny story about this particular feature... This was almost entirely Cody's idea, because I'm like a) I want social proof; I want people's quotes on there. And then b) we do a few things differently, and they're all minor; it's like minor reasons. But I want to have somewhere a list of the stuff that we do, like we don't track your clicks, blah, blah, blah. And he's like "Well, what if you just did way too many of them, and just made some up?" And he actually stubbed it out with 36 reasons... And I took it as a personal challenge to come up with 36 reasons why you should subscribe. And in the mid '20s I'm like "Cody, this is too hard, dude. I cannot come up with -- he's like "I did not expect you to come up with 36 reasons. I just made up a number." And so I finally made it to 27, and left it there. But I love that you've improved it, you've added two. I will go on record to say we are accepting pull requests on reasons to subscribe. So if you have more reasons, I would love to get that number as high as humanly possible.

Gerhard Lazu:

99. 99 reasons.

Jerod Santo:

Yeah, I mean... You have to subscribe if you read 99 reasons, don't you?

Gerhard Lazu:

Yeah.

Jerod Santo:

Excellent job here. I will merge this.

Gerhard Lazu:

Thank you. Thank you. **Break**: \[00:23:21.09\]

Gerhard Lazu:

Now, I want to show you a really cool feature. So I've used the previous improvement that we shipped last in the last Kaizen, which is I'm connecting to the Neon from my local environment. Look at this. So I've done this, I've created a branch, I think it was a few days ago. I can't remember. And obviously, Master has moved since, right? So if I'm going to close this, I'm going to go to Home, localhost 4000, and if we go to the podcast "Securing GitHub" 2024.06.19, right?

Jerod Santo:

Right.

Gerhard Lazu:

I'm sure there's a new one... Let's just go back to the main one.

Jerod Santo:

Yeah, there is.

Gerhard Lazu:

Let's see what it is. What's the last one? So "Polypandemonium."

Jerod Santo:

Yes. JS Party episode \#327.

Gerhard Lazu:

Cool. So Neon - I have the console open. I'm going to click on my branch, and I'm going to say "Reset from parent." This is me syncing the data. Yes, reset.

Jerod Santo:

This is your dev branch. Yes.

Gerhard Lazu:

Okay? Done.

Jerod Santo:

That was fast.

Gerhard Lazu:

Two seconds. I come back...

Adam Stacoviak:

So fast.

Gerhard Lazu:

...I refresh...

Adam Stacoviak:

Polypane.

Gerhard Lazu:

Boom.

Jerod Santo:

Bam. That's cool.

Gerhard Lazu:

Cool. So that was it. Alright. That made me very, very happy. Very happy. The whole thing, really cool. So over to you. What do you like the most? I'm going to stop screen-sharing. What do you like the most about the news redesign?

Jerod Santo:

Well, you kind of stole mine, Gerhard. I was gonna say the 27 reasons...

Gerhard Lazu:

It can be the same. It's okay. Now they're 29. That's good.

Jerod Santo:

That was definitely my personal favorite part, because it's just kind of zany, and something you don't see all the time... And probably that most people wouldn't even notice, but it's there. And so yeah, I'll pile on... I'll say the pyramidical structure of 29 reasons to subscribe is definitely my favorite part. Adam, yourself?

Adam Stacoviak:

I think it's just as simple as having a much better landing page after literally just so long. I mean, so long I can't even tell you how many years to have a better place to send somebody to subscribe, that showcases the content, and the reasons, and the ability, and the social proof, all in one place. This divided design was like -- I know when we first talked about our newsletter, early on, before it got into code, or dev, or really like some fleshed out design, we were talking through like "Okay, the newsletter could be on the right, and the left side can be static, and you can easily scroll on the right." And it matches the brand, because the left side can be dark, and the right side could be the content... And it's obviously massaged and morphed since then, but the basic idea was really good. But then having a place now to just call home for Changelog News, which is Changelog.com/news... I really wish we had .news, but whatever. We won't get upset about that.

Jerod Santo:

That's my -- I mean, continuous improvement... I'm still holding out.

Adam Stacoviak:

Let's kaizen that at some point.

Jerod Santo:

I mean, Gerhard knows the exact date that we bought shipit.show, right?

Adam Stacoviak:

Yeah...

Jerod Santo:

When we get Changelog.news, and I can stop saying .com/news... I mean, that's gonna be a huge win. And someone out there who holds that domain and needs negotiating power better not listen to this. Because when I talk to them, I'm totally gonna play it cool, like "Hey, do you care about this domain?" I mean, just \[unintelligible 00:31:35.01\] Happy to pay some money for it, just don't want to get raked over the coals for it... But man, we've got to have changelog.news. It's just so much cooler.

Adam Stacoviak:

Yeah. And really too, I think my other favorite thing about this is that it didn't -- the existing newsletter design didn't have to change really at all, if any, to fit into this new design. So the original email design got pulled into this, and so I think that's kind of nice that we can literally incrementally improve Kaizen, this newsletter, to be - beyond newsletter, a decent design, and good design, and great content - to a full-on landing page that doesn't have to change a lot; it just sort of added to it. It was additive, versus total morphed. So I like literally how iterative we got to this point to have this kind of page.

Jerod Santo:

\[00:32:26.15\] And one more note on the newsletter redesign... So for those who haven't seen it, as Adam described, on the left hand side it's dark, and it's really representing what you're looking at, and has the reasons to subscribe, the buttons and stuff... And on the right hand side it's the most recent issue, so you're literally reading what it looks like. And in light mode, the right hand side has a white background. Now, in dark mode they're both kind of dark, and there's a border in between... But the idea behind this, kind of the metagame is really -- the podcast/newsletter is a hybrid. It's two things in one. It's the only podcast that is a newsletter, and it's the only newsletter that's also a podcast that we know of, in this space... And so you kind of have this two-faced thing going on, where there's a duality to the content, and there's a duality to the design... Which I think is kind of cool. Probably that gets lost on most people, but for those of us who put it together, we just thought that was kind of neat. Now, I don't want to directly reference Two-Face, because I realize he's a villain, and it's probably not the best association... But you know, it's got that duality thing going on, which is pretty cool.

Gerhard Lazu:

It took me a while to realize that there is a button in the top right corner, you can press Play.

Jerod Santo:

That's the one thing that we probably should continue to fix, and make better is like you want to be able to read it and you also wanna be able to listen to it. And yes, there's a Play button in the right hand corner, and as you scroll the newsletter - like, that's sticky; it's not gonna scroll with it, so it's always right there. But I'm guessing - and I don't look at stats very often, but I'm guessing less people are listening to it on the web than it used to be... It used to be our standard episode player, and now it's kind of a unique thing. You also can't skip chapters, and stuff. I think when I'm wanting to scan an episode and see if I want to subscribe, I kind of want to click through it and maybe like scrub it, and none of those tools are there, because we kept it very stark... But definitely ways we could make it better, and make it more obvious that yes, you can also listen to this right now, while you're staring at it.

Adam Stacoviak:

And you almost want like a player of sorts, but then that's too much. Like, does it really require that? Because the listenership isn't there on the web to like put that dev behind it. And I think design-wise it could stand out a little bit more. Like, the Play button could be more Play-buttony, besides like literally a Play button... I don't know, it's great design. It's just -- there could be some tweaks and turns to let it stand out a little bit.

Jerod Santo:

Some sort of affordance that draws attention to that Play button. Maybe like has some sort of a sheen to it, or like something that just draws your eye over there and it makes it seem -- maybe even the word Play under, so you can just say "Oh, I can play this", because we all know what a Play button looks like, but they kind of blend into the background when we're reading... So yeah, we could definitely make it better.

Gerhard Lazu:

I really like how we are improving Jerod's pool. This is amazing.

Jerod Santo:

\[unintelligible 00:35:02.08\]

Gerhard Lazu:

I would like us to keep going... No, we are improving it, for sure.

Jerod Santo:

I appreciate it. I appreciate it.

Adam Stacoviak:

I guess a question for you, Gerhard, is you showed off Neon there, and you showed off how you can reset to - what was it, parent branch? Is that what the --

Gerhard Lazu:

Yeah.

Jerod Santo:

Okay.

Gerhard Lazu:

So I have a branch, and I was just basically synchronizing it with all the data that's in the parent branch.

Adam Stacoviak:

Well, why did you show that off?

Gerhard Lazu:

Because I was using it locally, and obviously the two diverge, so I have my own branch, literally my own branch from the database, that I can always synchronize, basically catch up with Master --

Adam Stacoviak:

Dev\_gerhard, or something like that, right? Doesn't that give you like a default --

Gerhard Lazu:

It doesn't have my name yet, but it has a date.

Adam Stacoviak:

It doesn't? Okay, gotcha.

Gerhard Lazu:

Yeah.

Jerod Santo:

So what happens if you make local changes? Maybe you go in there and delete some episodes while you're doing development.

Gerhard Lazu:

It's on my branch. It will not affect the production.

Jerod Santo:

Right. But then when you hit sync, it's not going to merge changes, it's going to just --

Gerhard Lazu:

it's recess. It's one way.

Jerod Santo:

It gets you back to where you were.

Gerhard Lazu:

It's one way, Yeah.

Jerod Santo:

That's what you want.

Gerhard Lazu:

Exactly.

Jerod Santo:

\[00:36:00.01\] I'm still a little tentative of those things, because I have -- I guess anxiety is the word. You know, I played basketball early in the mornings, and one time I got halfway to the gym and realized I didn't have my socks... Which pretty much ruins it, right? And it's a 20-minute drive. So I'm either coming home, or I'm missing that day... And ever since then, I've had like "Do you have your socks?" anxiety. And to the point where I put them in my bag, I put my bag next to me, I'll start to drive my car, and I'll stop -- I won't stop, but I'll think to myself "Do you have your socks?" And I know intellectually that I have my socks, because I literally --

Adam Stacoviak:

Oh, man...

Jerod Santo:

But I'll still just check. I'm like "Well, I'm just gonna check... Oh yeah, there they are." And I have that a little bit when you're like just having this production branch, and then your dev branch, and you're doing stuff and you're like "Am I sure I'm not connected to the production branch?" You know?

Adam Stacoviak:

Yeah.

Jerod Santo:

It's a little anxiety-ridden for me still. And probably because I'm just not used to it. Or if I get bit by it one time, I'll probably never want to use it again... But the value is there. So it's interesting to me, but I'm still kind of just a little bit like -- I don't want to accidentally screw something up.

Gerhard Lazu:

Yeah. Well, there are backups. The whole point is like if we did make a mistake, is the system resilient enough that we can recover from it? And how quickly can we recover from it?

Jerod Santo:

Sure.

Gerhard Lazu:

Right now, let's say we delete something in production... Well, do we have a backup we can go back to? Yes, we do. How much are we going to miss? Not that much.

Jerod Santo:

Yeah, I guess the things that make me nervous are accidentally sending emails to people, or deleting mp3s off of a CDN, or... There's stuff that is recoverable, but a bunch of work, and embarrassing, and stuff. So that's just standard -- I think all developers have at least concerns in the back of their head as they're doing stuff, and I'm just one of them.

Gerhard Lazu:

That's why a well-designed system really helps. For example CDN, you only upload. You don't allow the keys to -- the keys that you used to upload are not able to delete, for example. You have a separate set of keys. For this branch --

Jerod Santo:

You're just telling me what we should do, right?

Gerhard Lazu:

Well, yes, exactly. \[laughs\] Very subtly. Very gently, sliding it into the conversation.

Jerod Santo:

I'm taking notes... Yes, that's really smart. \[laughs\]

Adam Stacoviak:

Yeah, that's interesting though that you have keys that unlock, but not keys to lock. That's kinda like the concept. I have a key in my hand. If the door is unlocked, I can lock it. But I can't unlock it with the same key.

Jerod Santo:

And actually, we can't delete mp3. We could overwrite them with other mp3s... But it is write-only in that sense, not delete. So that was just an example of things I think about.

Adam Stacoviak:

The reason why I asked you about why you show that off, because I think the last - either last Kaizen or the Kaizen before, Jerod, you shared apprehension of Neon in the dev branches space... Because we contemplated latency, we contemplated the costs... And there's a lot of things that I've learned since then. I had a great conversation, literally yesterday, with a fella named Bryan Clark, who is the VP of product at Neon. And so we were talking at length, and this will come to an ad spot near you, of course, but we like to do really cool ad spots for our content. But I learned a lot about this, that the database spins up in 500 milliseconds or less, so literally half a second... That if you walk away from your machine, you're in development mode on your local branch, doing development on a branch, that the database spins down to zero after five minutes of inactivity. So there's no long-term polling. So it spins down to zero. The cost is basically nothing, because you're in dev mode; unless you're creating data... Then and only then would you pay more for space. So it's really just time on the CPU, the diffing there, and then obviously the whole thing that Gerhard just showed off, which is why I asked this question, was this reset to parent, so that you can suck in more data, so that you can actually have prod-like or literally prod data in dev mode with just the slight cost of a little bit of latency. Because it's literally not local, so there's not going to be -- it cannot compare to local.

Jerod Santo:

Right.

Adam Stacoviak:

\[00:39:55.25\] But then Brian did also divulge some potential future stuff that I'm gonna mention here, because it's not an ad spot, and it's cool. They're going to work on the ability to go offline with your database, and still be Neon-like. Neon, locally dev branch, all the fun things you want, with Neon local dev branches, but go offline. Go on a plane, go on a train, whatever it might be. That's future coming. I'm kind of like future-spelling for them, so don't get too upset, Neon, please... But this is cool tech, they're working on it, and they have a plan to continuously improve this developer workflow with a database... Which has not been there really, at large, ever... And it's on our favorite database, Postgres. So that's a beautiful thing.

Jerod Santo:

Well, that leads me to what I asked Gerhard for last time. Didn't I ask you for something like that as a gift?

Gerhard Lazu:

Yes, you did. Fresh data... Something about booty... No, no, no. \[laughter\]

Jerod Santo:

Yes, I do recall something about that...

Gerhard Lazu:

So yeah. Okay. So yes, you did. I have something else for you, though...

Jerod Santo:

Okay. It's not ready. \[unintelligible 00:40:53.16\]

Adam Stacoviak:

Is it called more booty, or double booty, or \[unintelligible 00:40:55.10\]

Gerhard Lazu:

It's called something -- it's called Pipe Dream.

Jerod Santo:

Oh, it's called a pipe dream.

Gerhard Lazu:

Cool.

Jerod Santo:

Let's hear it.

Gerhard Lazu:

So we're not ready for that just yet. We still need to stay in the news redesign camp for a little bit longer...

Jerod Santo:

Oh, we do. Okay.

Gerhard Lazu:

...because I was going to share a couple of things that I think would make it better.

Jerod Santo:

Okay.

Gerhard Lazu:

So first of all, there's obviously that player. We talked about that a fair bit. The second thing which I wish was there, and I don't know where you stand on this, but just a very faint background on the sponsored segments. Right now they all like blur... And I think it would be very helpful to see what is sponsored and what isn't without -- basically, just like by skimming. Because it just puts me in the right mode, and it feels like it's more honest that way. So sponsored sections, just like a fainter gray, or a faint whatever... Like, it has to be tiny. It's not an ad, but it's slightly different. Also the shoutouts. I think if the shoutout section would be slightly different, it would make it easier to digest. Because now it's not like one long newsletter. It has parts to it. I would quite like that.

Jerod Santo:

What do you mean by the shoutouts? Are you talking about when I give Changelog++ shoutouts?

Gerhard Lazu:

Quote of the week.

Jerod Santo:

Oh, quote of the week.

Gerhard Lazu:

Yeah. I think also like a different background color... Because those I think \[unintelligible 00:42:12.27\] And for example, I've seen quotes, I wanted to go back to them, and I didn't quite remember where they were, so I had to basically start reading to get to "Oh, here's the quote that I was looking for."

Jerod Santo:

Right.

Gerhard Lazu:

So... Tiny things. Tiny things.

Jerod Santo:

Yeah. I mean, I'm not against any of those things. Some of this is my desire to write the newsletter on my own terms, which is basically in raw Markdown, without any sort of additional tooling. And so the only like special formatting that we do is on the sponsored posts. That's an h4, I think, when we give a thanks to 1Password for sponsoring Changelog News... And we just style that differently, literally because it's like just the only h4 or h5, I can't remember which one, that I actually use. And everything else is H2, and H3, and blockquotes... And it's like literally just raw Markdown, which helps me actually author in a timely and enjoyable fashion. And so there's no special -- there's no special anything. So there's no way that -- like, we don't know semantically that a certain thing is sponsored, except for that it has that little h 5 or h4 under it. Now, we could probably do some fancy footwork and detect that, and like just based on the way I write it... So I could get more formal on certain things, but I don't do any classes, any tagging, anything. The quote of the week is literally me making an h3 that says "Quote of the week", and then a blockquote. And then that's it. Like, that's what the quote of the week is. So in our system, there's no fancy formatting, because it's so free-flowing. And that allows me to be more, I think, creative and experimental with what I'm doing. Quote of the week is not always there, which is I think is funny. There's not always a quote of the week, there's not always a video of the week... So it's not like I'm saying "I need a quote for this week", and I plug it into the quote of the week block. So it's more a result of how I write, versus an editorial decision. \[00:44:14.02\] And we definitely used to have -- the sponsored section I think was more called out than it is currently, and I think we could do some coding and just follow the same syntax that says "When you find a section that has this particular element in it, then change the style", or something. So I'm not against it. It was just like we wanted it to be as free-flowing as possible, so that I could continually put them out every week.

Gerhard Lazu:

Yeah.

Adam Stacoviak:

Your perspective though, Gerhard... I understand that too, but do you feel -- I mean, this is important. Do you feel duped? What is your feeling whenever you feel like these things blend? Because it's not meant to be dubious, by any means... It's just meant to blend a bit more. So like Jerod had said, it's not something special, and then adding "Hey, thanks to 1Password for sponsoring Changelog News." That was last week's sponsor, I'm just reading that. And obviously in the audio it's called out, right? It's pretty clear in the audio... But it's less -- it blends more so with other content in the newsletter.

Gerhard Lazu:

So, I would like to have the option of quickly skipping over a section if I so choose. It's almost like "Okay, this is different. I know what this is. I can skip it, I can come back to it if I want to." But this is something different from the news items. And this is a sponsored news item, and this will talk about most likely the benefits of that tool, or how it can help me, or things like that. And I would like, again, to have that option of skimming over it quicker, if I so choose. Now, sometimes I'm in the mood of reading everything, and that's okay, but other days I'm just skimming for something; I remember seeing something and I want to go back, and I have to do quite a bit of reading to get there. The headings help, but it all like blends together. And also the quote of the week, I think they're good to shout out, and seeing that slightly different, because it just puts me like in another mindset. This is something that some person said, and maybe I'm interested in like digging deeper for this quote. Maybe there's an episode, and there most likely is an episode, so it pulls me in that direction. And being honest about that almost like interaction, I would find it more pleasing. Maybe it is just a personal preference, honestly.

Jerod Santo:

Yes, it's good feedback. No, I mean, I definitely think that there's times where you're reading a newsletter, and there's times when you're scanning the newsletter. And I think that while it's not difficult to see the things for sponsoring Changelog News with the -- we've put the cash bag emoji. I mean, most of our actual sections and breaks is like horizontal rules and emoji, right?

Gerhard Lazu:

Yeah.

Jerod Santo:

The quote of the week always has the exact same emoji to start it out. So I try to be consistent with those things, because I know they are visual cues for people. I think that we could probably make the sponsored posts slightly more prominent without having to rework my entire publishing workflow.

Adam Stacoviak:

Yeah, maybe always just putting the -- for example, to read the headline, "Strong, unique passwords for you and your team." That was the 1Password one from last week...

Jerod Santo:

Mm-hm.

Adam Stacoviak:

...maybe move the money bag up to be the emoji on the thing, versus the strong-arm thing, which I totally think is contextually correct. Maybe it's always the money bag emoji before it, that way it's like a visual cue. I don't know. Gerhard, would that be enough for you to keep it simple?

Jerod Santo:

Yeah, if the moneybag emoji is the sponsored news cue, that's one less emoji for me to select as well. I've gotta come up with which emoji to pick.

Adam Stacoviak:

That's right.

Gerhard Lazu:

I like it.

Adam Stacoviak:

Because then it's always leading with the money bag, and you can kind of spot the money bag, and the money bag, obviously... "Hey, this is sponsored" - and by the way, we do love 1Password - and that's cool. I think that that'd be a good --

Jerod Santo:

\[00:47:44.03\] Well, some people put like sponsored in parentheses in the title. We've considered doing that kind of stuff as well, and it just gets to be a little bit too noisy... But I think of it just like, the money bag -- because contextually, the money bag doesn't make sense until you know that "Oh, it's sponsored." And it's always just the money bag.

Gerhard Lazu:

I like that idea.

Jerod Santo:

And you can just be like "Skip the money bag if I want to."

Gerhard Lazu:

That could work. Or look for the money bag... If there's some money for me, let me check it out. \[laughs\]

Jerod Santo:

There you go. Well, it's funny, because -- who was I speaking to years ago, that kind of changed my perspective a little bit on Google ads? ...which - my perspective was skip the sponsored ones immediately. I do a Google search; I'm gonna skip right past the sponsored to the organic. Habitually, intuitively, easily. Give me the organic searches. And somebody who was older and wiser than me, but maybe a little bit less technically savvy, but definitely more business savvy, was like "I go straight to the sponsored ones." And I said, "Why?" And he goes "Because those people are serious about what they're doing, and they want to reach me." And I'm like "Well, that's interesting." So they're legitimate, because they've got money to spend on this exact thing I'm looking for.

Adam Stacoviak:

I do agree with that context, too.

Jerod Santo:

And I was like "That's actually interesting. Maybe I should click on those more often." Because at first I was like "Yeah, they're just trying to spend money to get here." It's like "Yeah, because they have a serious business that provides the exact thing that I'm looking for... So don't skip the sponsor, look at him." So maybe sometimes you look for the money bag, you know? I think - and I'm biased - that if a company is sponsoring Changelog News, they are a company that is worth paying attention to. I think so. And so look for the moneybag. It's a good point, Gerhard.

Adam Stacoviak:

I had this exact challenge, I suppose, on a call this week. I was trying to showcase how we showcase people in the newsletter. And I'm scrolling it, trying to find the sponsored ones, and I couldn't find, because it blended in so well. Then I was like "Hey, don't think we're trying to blend them in too much because I can't find it, but they just blend in. It's just natural." But I think the money bag being in front of it could be the easy icon...

Jerod Santo:

Totally.

Adam Stacoviak:

...because I would have been looking for it. So it's a win/win.

Jerod Santo:

We still put the thank you underneath it, you know?

Adam Stacoviak:

Yeah, exactly. Keep the same subheading.

Jerod Santo:

We're not trying to hide anything. Like, that's not the point at all. It never was. These are sponsored. That's what they're there for. We just wanted to do it in a way that's a) tasteful, and then b) allows me to write in a flowy manner, which is like throw in an h2 and an h4 and move on. But yeah, the moneybag for scanning -- I mean, I do like the emoji for scanning; I use them a lot myself, so I think that that would be a positive improvement. Quote of the week - not quite so sold on making that standout, because it's honestly just a thing I made up and do sometimes. It's not like a formal part. There are no real formal parts... Besides, there's going to be five or six main stories, and then the rest I just make up each week. Like, is there three smaller stories? Is there a list at the end that's shaped like a pyramid? You know, I just kind of roll with it, and I like that. But definitely good feedback. **Break**: \[00:50:45.16\]

Gerhard Lazu:

So I'd like to give a shoutout to Brandon Smith now. He's not involved with Elixir. Pull requests 513. He is in the list of contributors. It's a one-character change. He's my type of people. His comment is way, way longer than the actual change, and I love it, because it has all the context.

Jerod Santo:

Shout to -- is it Brandon?

Gerhard Lazu:

Brandon Smith, yes.

Jerod Santo:

Brandon.

Gerhard Lazu:

Brandon Smith.

Jerod Santo:

Yeah, good job, Brandon. He basically was just removing a hash sign - or as some people call it, an octothorpe - from our show titles, or something. Help me understand.

Gerhard Lazu:

Yup, it had to do with GitHub interpreting the hash sign and a number as a pull request for an issue. And obviously, we don't want that in the context of GitHub. So what he did, he basically fixed that. But if you check out pull request 513, there is a lot of detail. And again, I'm shouting that thing out specifically because that was a very nice contribution. In terms of why he made the change, not necessarily the change, which is also great, so thank you for that, but it's the detail around it. So more like that, please. I love them.

Jerod Santo:

Yeah. Small change, but he made the Merge button very easy for us to push, because all I had to do is read all the context, and I was like "Looks good to me. I like it. Merge button." Whereas if he had just opened a PR that just removed the octothorpe from our transcript titles or whatever that is, I'd be like "What's the point of this?" I'd think he was just trying to get a Hacktoberfest T-shirt, or something.

Gerhard Lazu:

In June.

Jerod Santo:

Yeah. Well, it's like Christmas in July. You know, Hacktoberfest in June.

Gerhard Lazu:

Yup.

Adam Stacoviak:

So I was launching a show on Wednesday, as we do, every Wednesday, and I logged into the admin, Jerod, and you know what I saw, right? I saw Changelog++, I saw all memberships, I saw all feeds in there... I think feeds has been in there for a bit, but Plus Plus is new... And we're excited, because we've been wanting to bring from Supercast over into changelog.com proper our Plus Plus stuff... Because it is better, it's not going anywhere, and we want to continue to Kaizen that as well. What's going on there?

Jerod Santo:

Right. Yes, we want to make Plus Plus even better, and one of the ways that we want to do that is by allowing our Plus Plus members to create custom feeds. So the number one request to anybody who signs up for Changelog.com/++ is "Why do I have to listen to all your episodes?", which - you don't have to listen to them all. But basically, the way that Supercast works, which is our platform that we've used since the initiation of this program to actually manage the entire process, is that we publish a single feed to them, and then they manage the individual people feeds that you subscribe to as a Plus Plus member. So we only get one to them. And so to them, we send over everything... Plus Plus versions, of course, which is the bonus content, extended episodes, ad-free. We send them those versions. And so as you subscribe, maybe you only listen to JS Party and you subscribed to changelog.com/++, and you're like "Wait, now I'm gonna get all your shows." And we just say "Yes, we're sorry about that. You can just delete the ones that you don't want." Maybe your app has a custom filter, where you can just filter out, just so you're getting JS Party... But like that's literally -- our hands are tied, because we don't control that platform. This is how we do it. \[01:00:06.23\] And we've wanted to fix that problem for a very long time, and at first I thought we had to just completely ditch SuperCast in order to do that, and then I started thinking how much work that would be, and how we want to eventually do that, and just have our own autonomy... But in the meantime, can we get halfway there as a stepping stone and give people what they want? And I determined "Sure we can." All you need to know is who's actually a Plus Plus subscriber and has an active account on our side of the fence. And then you allow them to have a feature and nobody else to have the feature. So I started off with building the feature called Custom Fees, where you can set up, exactly what it's called; log in, you can have more than one, you can pick which podcast is in it, you can pick your own album art, you can say how the titles are going to work, where it starts, etc. Just a few different things. And then you can subscribe to that. And I've been subscribed to one myself for the last six months, just to make sure it works every time... And it's working. So that's great. We're getting to where we're ready to roll that out to some folks... And so most recently, the next step is "Well, how do we know who has a Plus Plus membership?" And so that became a Supercast integration, which was actually a Stripe integration, because the cool thing about Supercast, which was one of the reasons why we went with them in the first place, is they say "We do not hold your customer base hostage. It's simply a Stripe integration. You own all your relationships; it's in Stripe." And so they're managing a Stripe account of ours, and now we're also not managing it, but we're syncing a Stripe account of ours. So that allows us to then suck in all the membership information into our database, and so now we know natively inside of Changelog.com who is a Plus Plus member and who is not... Which means now all we have to do is build and enable the feed building form, the UI for building and managing your feeds, and Plus Plus people will be able to build their own custom feeds. And so you could have a Go Time feed that's just Plus Plus for Go Time, you could have a JS Party feed, you could have "Pick my three favorite podcasts" of yours, or you can just say "Give me all of them." You can do all the things. And it's pretty cool. It works well. I'm excited to roll it out. We haven't rolled it out yet because I just got the membership information into the system... This week, Adam? Was it this week?

Adam Stacoviak:

This week, yeah.

Jerod Santo:

Yeah, this week. So that's been a lot of work behind the scenes.

Adam Stacoviak:

When I first went there it was empty, so... Now it's full. First time there it was empty. And I was like "Oh, this is in motion." And then later that day you pinged me, you were like "Hey, I added some stuff." I'm like "I saw that", when I was shipping a show. That's cool.

Jerod Santo:

Yeah, there's two parts to it. I mean, one is just a scheduled job that goes out, I think, every three hours, and just sucks in all subscription information from Stripe, and updates everything accordingly. And then the second thing is the WebHook system. So if you just sign up, we don't want to have to wait three hours, or even an hour... I mean, it doesn't take long., I could probably run it more often. It takes like 90 seconds to run, something like that. But I just set it up every three hours. And so the WebHook system - as soon as you subscribe, we know about it, and you can log in and actually manage that. One thing that's bugged me for years is you can sign into changelog.com as a Plus Plus subscriber and it still says "Sign up for Changelog Plus Plus", because we just didn't have any connection back. And so I've already changed that; it doesn't say that to you if you already are subscribed. And that's just bugged me for so long. So that's gone. But yeah, next up, and hopefully by next Kaizen, we will have custom feeds, the ability to create and manage them, and actually roll them out to our Plus Plus people... Even though you'll still sign up and do everything through SuperCast, basically, for now.

Adam Stacoviak:

And eventually we'll have a page like SuperCast, that will be a tier. We're still not sure; we've thought about several things. I think we spit-balled some ideas when we were in Seattle, about ways to do that...

Jerod Santo:

\[01:04:05.06\] Yeah, basically like a fully autonomous Plus Plus, without Supercast, would be the end goal. And the fact that we can now integrate with Stripe directly... We're halfway. We have a Stripe integration, it's just not the full loop. And so eventually we'll be able to close the loop. It requires us to build a bunch of pages, and emails, and... You know, many of you all listening are web devs, and so you know all the stuff you have to build... And so we have to build all that in order to completely jettison ourselves from SuperCast.

Adam Stacoviak:

The emails... There's a lot of different things that happen, like "Oh, your car didn't bill." The things that are all involved in managing subscriptions, essentially. We become a SaaS, but not really full-on SaaS. A lot of the mechanics that a SaaS --

Jerod Santo:

It's a PaaS. It's podcast as a service. \[laughs\]

Gerhard Lazu:

I like that. That is a great one.

Jerod Santo:

We should write that down, so we can use it in our marketing copy.

Adam Stacoviak:

Is it the PaaS out there already, or is it a different version?

Gerhard Lazu:

No, that's the one. Yeah.

Adam Stacoviak:

I get it. I get it.

Jerod Santo:

So yeah, custom feeds are there, and they'll be coming to you, Plus Plus people, very soon, with a trademark next to the Very Soon.

Gerhard Lazu:

Yeah. I'm excited about this specific feature myself, because whenever I -- I do have a Changelog Plus Plus subscription, and whenever I want to listen to a specific episode because I've seen it on the website, I have to wait for Overcast to download the others before I can go to the one that I want. And usually, maybe the one -- because there's the last recent, I think, one, or last ten, maybe the one that I want to listen is further down below. So I was actually going to disable the automatic download. But if you're saying that this is coming, this would be amazing. I can test it with Overcast specifically, to see if I can only see the episodes from my favorite shows. So that'll be very nice. Amazing.

Jerod Santo:

Yes. That's exactly how it works. Especially -- I mean, so many people sign up and they say "I just listen to Go Time", or "I just listen to the Changelog. Why do I get all these?" And we just don't have -- now we finally can say "Yeah, go create yourself a custom feed, and you can get exactly what you want, and nothing else."

Gerhard Lazu:

That's so cool. Super-cool. Speaking of things that we've talked about for ages, but haven't done... I have one to share. I'm going to screen share now, and I'm going to walk through as if you are here... So we're looking at pull request 516, Trace GitHub Actions Workflows.

Jerod Santo:

Okay.

Gerhard Lazu:

So what this does - whenever a workflow completes, the entire workflow trace gets sent to Honeycomb.

Jerod Santo:

Okay.

Gerhard Lazu:

So whenever GitHub Actions triggers, whatever happens in GitHub Actions, we can see that.

Jerod Santo:

You can see step by step each thing.

Gerhard Lazu:

So let me scroll down to see the picture... This is what it looks like. The image. All we see is like the duration; we have the Ship It workflow, and we can see whether it succeeded... And if we go a bit further down, we can see the various pipelines, because we have a bunch of pipelines within Ship It. \[unintelligible 01:06:56.02\] and some don't always run. But then what's even more exciting is that you can click on one run, and you can see details about how long different pipelines took, and what went in. So this specific one we have, we're looking at Dagger on Fly run. And by the way, these are the screenshots on the pull request; if you can open it up, you will see it. And it's the blue one that's highlighted. And we can see how long it took to set up the job, how long for example it took to connect to Fly, setting up the WireGuard tunnel... And then connecting to the Dagger engine, spinning it up, all of that. And we can compare these runs amongst themselves.

Adam Stacoviak:

Okay.

Gerhard Lazu:

So what I would like to do is go to the UI now, and click a little bit around. So we're going to Boards... We are in Honeycomb and we have three boards configured. GitHub Actions is the last one. We have Fastly content stats, and Fastly service stats. We won't be looking at those two, we'll just be looking at the GitHub Actions. So we click on that, we see all the workflows that ran in the last seven days by default. We can change this to say "Show me the workflows for the last 14 days." What do we see? What do we see, something that stands out just in these two graphs?

Adam Stacoviak:

\[01:08:15.02\] Duration got higher on the 19th-ish...

Gerhard Lazu:

Yes. 11 minutes. Exactly. Why did this workflow take 11 minutes? That's exactly it. Things like "Why did this one, take only 2.4 minutes, versus this one which took 11?" What was going on here? So click on that. Boom, you get there. Boom, you click on that, you click view trace... And within a few clicks, you go to the actual workflow. So we can see that there were some errors. We can see that this one took 11 minutes. Dagger on Kubernetes - that ran in 196 seconds. So that was good. But Dagger on Fly failed. And Dagger on Fly failed when it was actually running. 139 seconds; it was running for 139 seconds, then it failed. And then remember, always have two have everything. So then it fell back to GitHub, and that's the one that succeeded in 438 seconds, and eventually, the deploy went out. So this was -- we can see it was Jerod, the one that committed, so it was his commit. And if we scroll down here on the right hand side, we will be able to see a link, and we have to click on the actual Ship It. Yeah, that's the one. The actual run. GitHub Actions run. Go to, actually; that's what we want. We want to go to, we open it up, Dagger on Fly, boom, we click on this... We see the build test setup. What happened? Gen server call, await tar ball, timeout. So this was the timeout pulling down a package on this run. And because we tried it again, in a different workflow, it succeeded. What do you think?

Jerod Santo:

I think that's very cool. I think that's great for actually knowing why things are happening in GitHub Actions... Because I always wonder, "Why? Why is it not working?"

Gerhard Lazu:

Cool. So while I was looking around the whole Fly world, I came across this; I came across the blog "World Page Speed Test", by none other than Chris McCord, the creator of the Phoenix framework, which is what Changelog.com runs on. And again, he is our type of person, for sure. He tinkered for one afternoon, he took Elixir, Flame and Fly and introduced worldpagespeed.fly.dev. May 8th. So not that long ago. Very recent. And it's a great read. But we'll skip over that, because we're not interested right now, in this context, in the details. We want to see the finished result. Worldpagespeed.fly.dev. What is the first website that we're going to test?

Jerod Santo:

Changelog.com.

Gerhard Lazu:

Of course. That's exactly what I did. That was the first website which I've tested. So what it does behind the scenes - it connects to a couple of data centers, Fly data centers, and it loads the website, the changelog.com website in a Chrome headless browser. So this is a full like DOM interactive sort of situation, and we are looking at Frankfurt, looking at Hong Kong, we're looking at Chicago, Sydney, Tokyo, Sao Paulo, Santiago, and Mumbai. So it's as if you would be running speed tests for a website from all these places. This is really cool. So what do we see? Germany's a bit slow...

Jerod Santo:

Mm-hm. Hong Kong's a little slow...

Gerhard Lazu:

Just to, again, add more detail... Changelog.com, when it loads on this headless browser, that is running in Frankfurt, in a data center, a Fly data center, took 1,400 milliseconds to load. That's a bit slow. So I'm wondering if we reset it...

Adam Stacoviak:

Not for me...

Gerhard Lazu:

So click Go again. Second time, it's a bit quicker. And this shows you the caching.

Adam Stacoviak:

\[01:11:58.26\] Oh, that's why mine was slower then. Because I pulled up my own page and did the same thing, and mine was 494 milliseconds for Frankfurt. And yours was 1,400 and something the first time. Okay, that makes sense.

Gerhard Lazu:

The first time -- so now you can test caching. So this is more realistic. 575 milliseconds once the cache gets primed. Which cache? The Fastly cache, because this serves it from Fastly, so we can see that... So this is pretty cool, but I'm wondering, how does it compare when you go directly to Fly? So this should be really quick, because -- well, kind of quick, because we are going to the Fly origin, which is running in the Fly data centers... But from these remote locations, they have to traverse the Fly network, and they have to end up in Virginia, where the Changelog application is running. So let's try that. Let's do fly.dev. So if we're loading the website directly from the origin, this is what we can see. So it's not that much slower. 648 milliseconds. Somewhat slower, but not that much slower. I thought this was interesting.

Adam Stacoviak:

I like how you -- this is like a subtle jab, this segment here. We won't say why, but let's just say it's a subtle jab again.

Gerhard Lazu:

Yup. So let's keep going. Let's keep going. So how does News Y Combinator compare? It's slower, right? 850 milliseconds.

Jerod Santo:

It's way smaller, too.

Gerhard Lazu:

It is. Is Changelog faster than Y Combinator?

Jerod Santo:

It looks like it.

Gerhard Lazu:

It looks like it is. So that's good.

Jerod Santo:

And they're sending probably a \[unintelligible 01:13:29.22\]

Gerhard Lazu:

\[unintelligible 01:13:30.25\]

Jerod Santo:

I mean, that 52-kilobyte page... It's tiny.

Gerhard Lazu:

I know, right? So what's going on there?

Jerod Santo:

They're still taking a while.

Gerhard Lazu:

Yeah. I'm just going to go back and I'm going to reload it again.

Jerod Santo:

Theirs is all text, so it \[unintelligible 01:13:39.27\]

Gerhard Lazu:

I know, right? So yeah, 850 --

Jerod Santo:

They must have a slow CDN, if there is one at all. Or maybe they just exist in one server somewhere.

Gerhard Lazu:

Yeah, it looks like that.

Jerod Santo:

Fastest in Chicago, Illinois. 315 milliseconds. So they probably have a US-based server. It's probably not at all distributed around the world.

Gerhard Lazu:

How does google.com compare? I have to check myself, right?

Jerod Santo:

I mean, Google's slow.

Gerhard Lazu:

Google.com -- it's like, I don't know what's happening here, but yeah...

Jerod Santo:

Maybe this is Chris's tools having problems.

Gerhard Lazu:

I don't know. But --

Jerod Santo:

Although -- similar to Hyperping. I mean, that's what I thought of when I first saw it, I was like "This looks like Gerhard's Hyperping service, basically."

Gerhard Lazu:

Yeah.

Jerod Santo:

But right there in the browser.

Gerhard Lazu:

Yeah. So Sydney is especially slowing for Google, 2000 milliseconds.

Jerod Santo:

I doubt that that's true. Can they continue to make money with that being true?

Gerhard Lazu:

I don't know.

Jerod Santo:

2.262 -- that's two full seconds To load google.com in Sydney, Australia? I mean, they're surely losing money. So from a Fly node near Sydney. Right? Is that how it works?

Gerhard Lazu:

Yeah. It runs like in a data center around Sydney, yeah.

Adam Stacoviak:

That's non SSL though. Does that matter? HTTPS vs HTTP...

Gerhard Lazu:

It is. I'm saying --

Jerod Santo:

It's probably upgrading.

Gerhard Lazu:

I'm saying HTTP -- I mean, yeah, I can try HTTPS explicitly...

Adam Stacoviak:

Well, I just noticed that it doesn't do that when you just type in changelog.com, the domain. It's not defaulting to secure.

Gerhard Lazu:

Yeah. I know that we have a redirect, and...

Jerod Santo:

Yeah, we do. They probably do as well.

Adam Stacoviak:

It'd be cool to have this tool as traces back into Honeycomb, like you did. It'd be cool to pick a service like this, as opposed to pick the places that matter most to you. Not so much matter most, but ones you want to make sure "Okay, what are ups and downs in Australia? How frequently are we slower than we want to be?" And track that, similar to the way you do with GitHub Actions, and traces, and that whole flow, which - I didn't chime in too much there, but I thought was just magical, how you can like stack-trace, essentially, where in the bottleneck of GitHub Actions a deploy fails. That to me is just like super-magical, to have that kind of visibility. But the same kind of visibility into PageSpeed, especially if we care so much about it, because I think we do.

Gerhard Lazu:

Yeah.

Jerod Santo:

Go back to that Google one, because there's some interesting results on that Google one.

Gerhard Lazu:

Yeah. Going back to the Google one.

Jerod Santo:

So in Frankfurt, Germany, it took 800 milliseconds. That page was 1.4 megabytes. In Chicago, Illinois, it took 775 milliseconds, which is faster. The page is twice the size.

Gerhard Lazu:

\[01:16:13.12\] Yeah, I don't know what's going on here.

Jerod Santo:

What are they serving to us United States folks? Do they know we like it supersized?

Gerhard Lazu:

Maybe... Tokyo, again, it's 900. Tokyo is like the smallest one.

Jerod Santo:

And then Brazil - it's large again, but it's faster. And then in Chile, it's large and fast... They have different payloads based on where you are, to a large degree, in terms of data transfer.

Gerhard Lazu:

I'm wondering if they're like serving the cookie notification... You know that? Like when you load Google for the first time...

Jerod Santo:

Oh, maybe.

Gerhard Lazu:

...you have to accept to some cookies; maybe that has something to do with it. I don't know.

Jerod Santo:

I don't know if that would double the size of the page, but yeah.

Gerhard Lazu:

Yeah. But I'm wondering how does bunny.changelog fair.

Adam Stacoviak:

Is this still a thing? You still have that?

Gerhard Lazu:

It's still a thing. Yeah, it's still running.

Adam Stacoviak:

Okay.

Gerhard Lazu:

You can check it out.

Jerod Santo:

A little faster.

Gerhard Lazu:

Yeah. It's 300 milliseconds, 400 milliseconds... Pretty good, I think. This is has been the fastest one so far. Cool. So Chris, we want to know more...

Jerod Santo:

Why do you have to open up old wounds, Gerhard?

Gerhard Lazu:

Well, you will understand in a second. We're almost there.

Adam Stacoviak:

Oh, my gosh...

Jerod Santo:

Oh, gosh...

Adam Stacoviak:

\[unintelligible 01:17:13.24\] Jerod.

Gerhard Lazu:

It's happening.

Jerod Santo:

What's happening...?

Gerhard Lazu:

So I'm going to quote Jerod. And Jerod was saying, on the last Kaizen, "I like the idea of having this 20-lined varnish config, that we deploy around the world, and it's like "Look at our CDN, guys."

Jerod Santo:

Pipe dream.

Gerhard Lazu:

Yes, pipe dream.

Jerod Santo:

You got there? Did you get there?

Gerhard Lazu:

It's so simple, and it can do exactly what we want it to do, and nothing more. But I understand that it's a pipe dream. I'm still quoting Jerod, "I understand it's a pipe dream... Because that Varnish config will be slightly longer than 20 lines, and we'd run into all sorts of issues that we end up sinking all kinds of time into." So by the way, I think I have a title for this show... "Not a pipe dream.

Jerod Santo:

"Not a pipe dream." I like that title.

Gerhard Lazu:

So if you introduce pipedream.com, it's not that. So if you make this mistake, that's an honest mistake. Pipedream.com is a thing. There you go, pipedream.com is a thing. I didn't even know.

Jerod Santo:

Oh, wow. It's actually like a developer service.

Gerhard Lazu:

Yeah. Connect APIs, AI... There you go. I thought this was going to be the first AI of the show.

Jerod Santo:

Well, they have to put that in the hero. This thing has almost 8,500 stars on GitHub. This is like a legit deal.

Gerhard Lazu:

8,500. So yeah. It has a video, and all...

Jerod Santo:

So this is not what you built.

Gerhard Lazu:

No. This is not it. This is not it.

Jerod Santo:

Alright. Well, then get off this page.

Gerhard Lazu:

Okay.

Adam Stacoviak:

Gosh...

Gerhard Lazu:

So you just need to do pipedream.changelog.com.

Jerod Santo:

Okay.

Gerhard Lazu:

You can load it. You can try it. And I'm going to share something else while you load it, and while you look at it.

Jerod Santo:

Okay.

Gerhard Lazu:

Okay.

Jerod Santo:

Very excited.

Gerhard Lazu:

I'm glad. I kept the best for last. That's what's happening here. This is the grand finale.

Jerod Santo:

That's a good title, too. "The last for best."

Gerhard Lazu:

Cool.

Jerod Santo:

Okay.

Gerhard Lazu:

Can you see my terminal?

Jerod Santo:

Yes.

Gerhard Lazu:

Okay. Can you see that? What did I type?

Jerod Santo:

ls.

Gerhard Lazu:

Perfect. That's exactly what I typed. Cool.

Jerod Santo:

This is a weird quiz. \[laughter\]

Gerhard Lazu:

A weird choice. So here's a run script. It looks fancy. It's an executable, and it can do a bunch of things. So what we're going to do today - we're going to run demo 2024.06.2021. That's what we're doing right now. Cool. And by the way, this is coming in the pull request. So by the time this is out, you should be able to see it. So let's keep going. And you may be wondering "Hey, this is not the first demo." There was another one.

Jerod Santo:

Back in January, it looks like.

Gerhard Lazu:

Back in January. Yes.

Jerod Santo:

When you guys ran into problems.

Gerhard Lazu:

Yes, exactly. With James.

Jerod Santo:

This is you and James Rosen?

Gerhard Lazu:

\[01:19:51.29\] Yes. The video is out. "Let's build a CDN." You can check it out, the exact problem, what we ran into. So that was the first demo. Now we're doing a better one. Cool. We hope. So let's see. So we're looking... We ran httpstat, which - I love it. It's a tool built by Dave Cheney.

Jerod Santo:

Yeah, Dave Cheney from the Go community.

Gerhard Lazu:

That's the one, yeah. Httpstat, the Go version, and it gives you a breakdown of the request to an HTTP endpoint. So we can see all the headers, and we can see how long the DNS lookup took, two milliseconds, the TCP connection took four milliseconds, the TLS handshake took eight milliseconds... Server processing - 573 milliseconds.t That took a while.

Jerod Santo:

Okay.

Gerhard Lazu:

The content transfer, 727. Total time for the pipe dream to load, 1,300. That's a bit slow. So let's keep going. So what was happening is Pipe Dream was only running in Sydney. And now we're going to use the magic of Fly to scale it around the world. It will run in 16 regions.

Jerod Santo:

Yes.

Gerhard Lazu:

Yes. So you approve. I like this. I like where this is going.

Jerod Santo:

I approve. I like this...

Gerhard Lazu:

So let's scale it --

Adam Stacoviak:

To narrate, there was a prompt that asked him Yes/No, and he said Yes.

Jerod Santo:

I said Yes. I'm ready. Bring it on.

Gerhard Lazu:

And that's what's happening. Boom.

Adam Stacoviak:

Executing. Done.

Gerhard Lazu:

The machines are up. They're running. And we have a question to answer. How many dollars do you think this is going to cost per month? There's emojis, there's everything. \[laughter\]

Jerod Santo:

That's gonna be a lot of dollars, I think... Do the big cash bag emoji.

Gerhard Lazu:

Can you see that?

Jerod Santo:

30 bucks.

Gerhard Lazu:

30 bucks.

Jerod Santo:

Not too bad. So this is just for the instances themselves, just to be clear.

Jerod Santo:

Not for the bandwidth.

Gerhard Lazu:

Yeah. Not for the bandwidth. We're only using the memory, 256 megabytes of memory. But one instance per month cost $1.93. 16 around the world. $31 per month. Plus bandwidth, maybe taxes. I don't know.

Jerod Santo:

Right, right. Plus or minus 10 bucks.

Gerhard Lazu:

So let's do that again, and let's see what happened. 369. And what do you see there? \[unintelligible 01:22:09.03\] miss. So this actually was served very close to where I am, but because it was a miss, it had to go -- so this was a cache miss, and it had to go and fill it. We can see the TTL; this will be considered fresh for 60 seconds, and it's going to be considered stale for 24 hours. That's how this is configured. Cool. We keep going to the next one... So what I'm using now is a tool - I love it - a CLI. I used Oha. It's from the Cargo community. And it will run a latency test on endpoints. Right now we have it configured to send 30 requests for one client, spaced one second at a time. To see the latency distribution for this HTTP endpoint, to see how many succeed, how many fail. \[unintelligible 01:22:57.18\] it's beautiful, there's bars, there's green, there's yellow... You have to try it out. So what do we see? We see that 26 out of the 30 requests took 38 milliseconds.

Jerod Santo:

Lovely.

Gerhard Lazu:

Isn't that nice?

Jerod Santo:

The slowest one was 300 milliseconds.

Gerhard Lazu:

300 milliseconds, yes. You will always have some outliers. And the fastest one's also eight milliseconds. I think we can ignore them, because you always get one which is super-fast, and you always get one which is super-slow.

Jerod Santo:

Sure.

Gerhard Lazu:

Maybe there's something with the tool, but the bulk of the requests, the majority of the requests took 40 milliseconds. And this is pipe dream.

Jerod Santo:

This is not a pipe dream.

Gerhard Lazu:

It's not a pipe dream. I don't think it is.

Jerod Santo:

I'm beyond tickled. I want to see the code for this. I wanna see my 20 lines of Varnish.

Gerhard Lazu:

We're getting there. We're getting there.

Jerod Santo:

Okay.

Gerhard Lazu:

\[01:23:48.25\] So the next thing, we have to compare this to Changelog.com. We're doing the same measurement. We measured Pipe Dream, and we went to pipedream.changelog.com, to see the webpage, how it loads. So now we're doing the same speed test. Same Oha, 30 requests, one client, one second apart, to Changelog.com, and in a few seconds we will get the result. How is this going so far?

Jerod Santo:

This is going great.

Adam Stacoviak:

It's visually impressive.

Gerhard Lazu:

Amazing. I was like, this brought me so much joy to put together. This was just like the best. 48 milliseconds. So Changelog.com, same test, same tool, same everything. 10 milliseconds slower than the Pipe Dream.

Jerod Santo:

Than the Pipe Dream.

Gerhard Lazu:

This is what you were asking for, right? It's almost as if I could tell that you're going to ask this. How many lines of Varnish config? That's what we want to know, right?

Jerod Santo:

That's exactly what I want to know.

Gerhard Lazu:

So we're counting, we're seeing 43... We keep going. 91, 117. However, there's a lot of comments. A lot of to do's... It's a well commented config. 117 in total. How many lines of Varnish config without comments - now you get to guess.

Adam Stacoviak:

91.

Gerhard Lazu:

91. Jerod?

Jerod Santo:

47.

Gerhard Lazu:

46.

Jerod Santo:

Wow, good guess.

Gerhard Lazu:

46. If there were any prizes, you would get it. 46 lines of code. 46 for this whole thing. Cool.

Jerod Santo:

That's amazing.

Gerhard Lazu:

So that was it. That was the end of it.

Jerod Santo:

So that's Pipe Dream. It's a Varnish config that's deployed to 16 Fly machines around the world.

Gerhard Lazu:

Yup.

Jerod Santo:

When you go to pipedream.changelog.com, it is a proxy for changelog.com.

Gerhard Lazu:

It's actually Varnish. It's a Varnish configuration...

Jerod Santo:

Yeah. It's just that Varnish configuration, just running around the world.

Gerhard Lazu:

That's it.

Jerod Santo:

When do we ship it?

Gerhard Lazu:

Exactly.

Adam Stacoviak:

And it got down to - what was the milliseconds? In the sub-100s, right?

Gerhard Lazu:

38. You know what? If only there was PageSpeed. let's try it, okay? So I'm going to share another screen. Back to PageSpeed. Let's see how it loads. How does it compare?

Jerod Santo:

World PageSpeed? Yeah.

Gerhard Lazu:

So how does it compare? We see \[unintelligible 01:26:00.18\] here, so we're going to open another one... Let's google pipedream.changelog.com. HTTPS. We're going to HTTPS, and it'll redirect nothing. Let's see. 700, 500, 436... This wasn't cached in some regions, so let's reset and let's try it again. Let's try the pipe dream again. What do we see? 400 milliseconds. 500. 300. 300. 457... If only, if only we could see this over time, right? So let's see. What do we see here? So we see -- again, Hyperping...

Jerod Santo:

Well, you do have it over time.

Gerhard Lazu:

24 hours, last 24 hours, from a bunch of locations... Average response time, 140 milliseconds.

Jerod Santo:

I love it. Let's ship it. Let's ship the pipe dream.

Gerhard Lazu:

\[laughs\] So hang on... We keep going. Changelog.com is 312 milliseconds. Same test, same everything. So twice as fast as changelog.com, worldwide, as measured by a tool that was made for this. So before we ship it, before we do that --

Adam Stacoviak:

Well, there is other challenges, right? Because like there's still bandwidth costs.

Jerod Santo:

Yeah, there's other things you have to do.

Gerhard Lazu:

Oh, yes.

Jerod Santo:

I'm just -- I'm being excited.

Gerhard Lazu:

Yeah. This is amazing.

Jerod Santo:

That's a very basic Varnish config, right? I'm assuming that's not handling any of our specific rewrites and needs and all that kind of stuff that we have in Fastly right now.

Gerhard Lazu:

Correct. Yes. That's correct. Also, we're not serving any static content. So that's one. But maybe we can leave that for later. Maybe we can just serve the website. We're not sending any logs to Honeycomb, and we do want that configuration... There's one thing which we still have to figure out... For example, when an application gets redeployed, for some reason the backend, as it's called in Varnish, remains sick. So this is a Varnish term, which is the backend, which is the application. When there's a redeploy, or when the application shuts down and restarts, the backend remains sick. I'm not sure why it's happening, but obviously, we need to fix that. There is no feed proxying... There's a bunch of things like that. And if you want to purge anything, we still have to have a system that purges across all these Varnish nodes. So how do you -- because you can't just purge one. So the question is, do we want to continue with this pipe dream?

Jerod Santo:

I would like to...

Gerhard Lazu:

\[01:28:12.03\] Adam? What do you think?

Adam Stacoviak:

I'm torn. I don't know. I think it's -- we have to weigh it against the whole "Shall we buy it or should we build it?" That's the question, really, here. We're saying we should build it, because we can customize it to our specific needs, but then we have to maintain it. And then we have to find a way to pay for it, which may or may not be something that we can get given to us, so to speak, as part of a partnership, which is how we've done it before... So there's definitely some -- in dreamland, I think, as a pipe dream officially, it does make sense to pursue, because it's the itch; it's the scratching our own itch. Back to the really simple CDN idea we came up with four Kaizens ago... I don't know, that's kind of what this is, is can there be a version of this that doesn't have to be - not so much that the Cloudflare, CloudFronts, or Fastlies, or Bunnies, or anybody else out there is a bad or a good thing... But like if you can build it yourself -- I think that's the question; like, if you can build it yourself and maintain it yourself, and fine-tune it, and it's worth it to you, is there a way you can go your own route? And I think that's where a really simple CDN comes into play, where that's really what we want. We don't want all those bells and whistles. I think the purging thing is probably the most pertinent thing, because we do purge frequently enough when we have misses or issues or whatever. That's a frequent feature. Otherwise, it's what nodes and regions matter to us. Can we monitor speed and fastness, and can we fine-tune the system we've built enough to eke out every millisecond possible? And I think the answer is yes, but the question for me is how do we pay for it? If we can figure out how we pay for it, and make it not so much profitable, but make it something that bakes into our core story, like we have done in the past, to me I think that's a win/win. And I prefer triple wins. Win/win/win. \[01:30:04.19\] *We win/win/win. We all win.*

Gerhard Lazu:

The answer, when it comes to paying for it, is that this would be within the Fly sponsorship. We wouldn't exceed it.

Adam Stacoviak:

You don't think so?

Gerhard Lazu:

If we don't handle the static content. Because that's where the bulk of the bandwidth is. Even if we handle the static content, even if we serve the static content through this, even in that case, we would maybe exceed it by a little bit, but we would still be like -- we'd be right there at the limit. And this limit has been in place for a couple of years now. So maybe that's a Fly.io conversation. But it's not twice as much. We're not talking twice as much. We're talking maybe 50% more, maybe.

Adam Stacoviak:

Yeah... I think you said before we do like 20 terabytes of bandwidth a year. Is that right?

Gerhard Lazu:

Maybe. I can't remember. I need to check the numbers. Maybe.

Adam Stacoviak:

Yeah, thinking here on the fly on the podcast, it's like podcasts as negotiation; like, we just -- we need to revisit the Fly conversation, basically, and see if this is interesting to them. Because I think there's two sides of this coin from my perspective. I guess three. One, it's super-cool. BA. This is awesome. That's my abbreviated version of bad-something-else, to keep our show clean. Then two, I think, should we build it, and should we maintain it - what is the maintenance burden we're taking on to build and deploy it? And is it literally just enough in maybe a couple iterations to be exactly what we want? So should we build it, should we maintain it? That's one question. Then the other question is can we have a partner love what we do enough to sponsor it or to bring it into the fold that we've already got? It seems like the first one's a yes, because it is awesome. And the third one's a possibility, because it's within the range... And I think the middle one is probably one to maybe spitball right here, which is - you've built most of it. Is this 80% of it? Is there 20 more percent, and is that the hard part? Like, how close to a version of done is this?

Gerhard Lazu:

\[01:32:07.26\] To be honest, it feels like we're 50% there. It took me some number of hours. We're not even talking days, we're talking hours of effort so far. So it was nothing massive. It didn't take me like days and days of like wrestling with this. There are a couple of challenges. I am curious about this myself, because I think it's really cool to be able to build your own CDN on top of Fly. Kurt blogged about this years ago, and he was using Nginx in that context. I think we would be fulfilling - not the prophecy, but the story, which is we're using Fly the way it was meant to be used. We're using Tigris as well. R2 migration? Maybe. Who knows? I don't know. I'm just dropping it as an idea. But if we use Tigris for the static content, and if you use this for the dynamic content, we may have a winning proposition. The only unknown is the logs. So that's the one that -- but now we have full control over that. We're not like "Maybe we can do it." No, no. We can definitely do it. It's just a matter of spending some hours to figure it out.

Jerod Santo:

Well, I think that, as I said last Kaizen, there is a halfway there, similar to my halfway off Supercast move, which is we leave cdn.changelog.com alone, which has the bulk of the bandwidth, and has the logging issues, and has all of that stuff, and we just focus in on changelog.com and the feeds, and just a straight up content cache for the dynamic pages, with purging, and we see if we can tackle that. If you can tackle that -- because we need purging on feeds, for sure. If you can tackle that, you're halfway there, and you're way further than you are now. And once we get this into a place where it's like it's a Varnish config that I can author and test and try out, then I think that it's a lot more something that even I could take the last mile... Because I can figure out some of that stuff. The purging is definitely an issue. Like, do we have to have some sort of an orchestrator or something that you notify "Hey, it's time to purge", and then that thing trickles down to all these little Varnish caches? ...versus the app knowing all the Varnish caches. There's some sort of middle piece of layer in there that we have to figure out, but... The rules around the feeds and all that kind of stuff, if we'd just focus in on that and leave the cdn.changelog.com alone, then we learn a lot more, we build something that's continually cool, and in the meantime we can figure out your other questions, Adam.

Adam Stacoviak:

Do you guys agree those are the three kind of pillars that are hinging upon? Like, one, it's awesome, two, should we build it, should we maintain it? And three, can it be -- what's that called? Not endorsed, but provided by, courtesy by, kind of thing, whenever a brand gives you something and you do something with it, and it's not really --

Jerod Santo:

Can't afford it... Yeah.

Gerhard Lazu:

The way I think of this is simple. And we come back full circle to where we started. Fastly is a bidet. How do you pronounce it, again? \[laughter\]

Adam Stacoviak:

Bidet.

Gerhard Lazu:

There you go.

Jerod Santo:

A bidet.

Gerhard Lazu:

It squirts, it remembers, it does the water, the whole thing.

Jerod Santo:

Okay, keep going. I'm tracking this analogy.

Gerhard Lazu:

Pipe Dream is a wet wipe. \[laughter\]

Adam Stacoviak:

Oh, gosh...

Gerhard Lazu:

It's simple, it's practical... Take it anywhere, same experience.

Adam Stacoviak:

This has gone too far. \[laughter\]

Gerhard Lazu:

And there's one more step.

Jerod Santo:

There's one more step... He's gone one step further...

Gerhard Lazu:

No CDN - singles-ply.

Jerod Santo:

Oh, my goodness...

Adam Stacoviak:

We're off the rails now.

Gerhard Lazu:

That took me all day.

Jerod Santo:

That took you all day. That took him longer than it took him to make Pipe Dream, was to figure out that analogy.

Gerhard Lazu:

Exactly. It did.

Jerod Santo:

That's hilarious.

Adam Stacoviak:

So I've got a proposition here then. I think this is an endpoint, to some degree. I love your \[unintelligible 01:35:52.20\] by the way. This is a good laugh. But I do have one more question that I think might make more sense potentially for Plus Plus. So is this roughly where we want to end? Because if so, then I have a potential five-minute, eight-minute Plus Plus.

Gerhard Lazu:

I think this was definitely the crescendo.

Adam Stacoviak:

Okay. Well, let's pause a second and say that demo that you did was definitely like the Fourth of July here in the United States whenever it's the finale scenario. The skies were ablazing, and it was thunderous... For those who didn't see it, we'll throw some screenshots. Kara, pull some screenshots if you can, of that, so we can throw them in the shownotes... Because I think it was super-cool. It's worth seeing. And I think that, you know, for the Plus Plus folks who are subscribed - boom, there you go. You're getting a little bonus here in a minute or two. For those who are missing out, you're just missing out. Go to changelog.com/++. Or changelog.com/plusplus. Right? Don't both redirect, Jerod?

Jerod Santo:

They both work. You can spell it out, you can use the characters...

Adam Stacoviak:

Either way. And get there. It is better. It's better! 10 bucks a month, 100 bucks a year, and see what we're gonna do with this CDN, this saga. I mean, it never ends. There's subtle jabs in the middle of shows, there's finales at the end, there's pipe dreams... I mean, what more do you come here to Changelog & Friends Kaizen edition for? I mean, this is what you come for, right? That being said... We done?

Jerod Santo:

Yes. Kaizen!

Gerhard Lazu:

I had so much fun. Three years in the making... It's only getting better.

Adam Stacoviak:

Only getting better.

Jerod Santo:

Yeah, man.

Gerhard Lazu:

I love it. I love it.

Jerod Santo:

Good stuff.

Adam Stacoviak:

Bye, friends.