#5 – Kyle Mathews: Benefits of using a sync engine, personal local-first apps, ElectricSQL

And so you're like, Hey, API, give me the data from my widgets. And then I render it. So it's sort of an imperative step, kind of like manually telling the client, tells the server, give me that thing. Where sync engines, it's a a move from an imperative to declarative.

So instead of saying, give me that data, it's sort of like for this page, I declare, I need this data and then the sync engine makes it so, often run into the problem where like, just like sort of getting data is easy, but subscribing to updates is hard. Welcome to the local-first FM podcast. I'm your host, Johannes Schickling, and I'm a web developer, a startup founder, and love the craft of software engineering.

For the past few years, I've been on a journey to build a modern, high quality music app using web technologies. And in doing so, I've been falling down the rabbit hole of local-first Software. This podcast is your invitation to join me on that journey. In today's episode, I'm speaking to Kyle Matthews, who's in the past has been the founder of Gatsby. js and is now exploring local-first software.

In our conversation, Kyle shares his experience building some small scale local-first apps for his personal use and how using a data syncing engine frees up a lot of his development time. Before getting started, also a big thank you to Expo and Crabnebula for supporting this podcast. And now my interview with Kyle. Hey, Kyle, welcome to the show. How are you doing? Uh, doing great. Uh, glad to be here. Thank you so much for, for coming on. You're another fellow early Local-Firster.

So I'm very curious to hear about your background and then hear about what led you to Local-First. So, uh, I've been doing web development of various sorts for like the last 15 plus years. I've seen a lot along that way, you know, like the whole jQuery era, did Backbone. js for those who read history books and, you know, it was early React, GraphQL. I actually started with Drupal.

Funny fact, Drupal, Drupal was big back in 2000, still widely used, but people are like a little more expansive about their view of Drupal back then. Then switched to kind of JavaScript, single page apps, like really early 2010 ish. And, uh, yeah, I've been kind of writing that ever since. Yeah, then, then of course, uh, Gatsby 2015 introduced that, uh, which kind of like, Took react and it was sort of the first proper production ready, you know, react, uh, meta framework.

And so kind of rode the whole react, uh, wave for a long time. But yeah, in general, my interest is what are the tools we use to build software? Cause like, I think software is very important and. The quality of the tools that we use determine the quality of the software that results. We shape our abilities and our ability to shape us. It's sort of like we, we, we shape our tools and then the tools shape everything else.

Cause I think there's really like high leverage to kind of continually pondering that question. Cause if we can genuinely move to a genuinely better tool, then there's enormous Effects that result from that, not just in our actual day to day life, but like, you know, we're a software creators. Like there's a lot, a lot of people whose lives are affected by software. So yeah, I've seen like genuinely better tools emerged in sort of my, my software career.

So like, I think react was a massive step forward. I think Grackula was a smaller step forward, but also a genuine step forward. And I feel like, you know, local-first, it feels like this is another sort of massive step forward. Leap forward, um, and how we build software. And I think it's something that in the coming years, more and more software will be kind of built in this fashion. I definitely agree.

I mean, uh, I'm also definitely a connoisseur of good tools, but luckily there's a lot of good tools and many different areas of development overall. Like you just. You can also look at other language ecosystems. I'm definitely very interested and inspired by all the good stuff that's happening in the Rust ecosystem, but also super cool ideas that you find in other ecosystems, such as like the Elixir ecosystem with live views, et cetera.

So there's like a lot of great tools and great ideas, uh, in all of those places. But I'm curious, which sort of problems have you experienced in the past that has led you. To, to local-first in particular, and where you now see a different approach, how to deal with those with local-first.

You could sort of think of like client server development, there's sort of three problems. There's sort of the UI, like how do we construct the UI? How do we do interactivity, charting stuff? So there's a lot of like different areas within like the UI and then there's sort of backend problems. There's a lot of like complicated computing bits. There's like, how do you run infrastructure at scale? How do you. Make things efficient.

So you're not like, you know, spending enormous amounts of money, uh, security. There's endless domains of problems in the backend. Um, and then there's kind of like the middle bits, which is like state transfer. Like, how do you move bytes back and forth? Cause generally the client can't hold all the data. So you have to like, say, I need part of the data to be pulled over. And like, generally things are changing to like, how do you get that data from the server to the client and so on?

And so there's a lot of. Problems around moving state back and forth. And so, yeah, so I think I'll go first. It's interesting because I feel like the front end and the backend, there's been just enormous number of innovations, both sides, all the investment and, and, and react it's felt and solid and view, uh, and like VEET and Webpack. And there's just like enormous investment that's gone into making the front end experience dramatically better.

Plus all the, you know, all the libraries, there's like NPM has like Millions of libraries or something like that. I don't know. Like each of those things is just like enormous effort from individuals and groups all across the world. And it's made the UI world dramatically better place. Like it's, it's just remarkable to me how much faster it is to build. Amazing stuff versus like 15 years ago.

Like I, the first like big JavaScript dashboard I built, it took me like six months to ship the first version, partly because I wasn't a very good programmer, but partly because like, there was just like no guidance anywhere. Now there's just like endless starters, endless use cases. Like React is such a compact, simple abstraction. It's dramatically better. And that's because there's been like tons of innovation there.

And the backend too, there's like all the cloud giants are pouring tons of research into stuff and there's like tons of. Individual cloud services where before you're like, Hey, that's three months of programming. And now it's like two hours to implement an API and off you go. So there's been lots of innovations too in the back end, but yeah, the state transfer bit, like the, you know, moving bits back and forth hasn't really, I don't know, got as much investment. Partly.

I think it's like, it's not always a problem. Often. It's pretty simple. You know, if you're just sort of like, Building a dashboard. You're like, get me the data for the dashboard, render the data.

It's not a very complicated problem oftentimes, but I think what we're seeing is that we're like the best teams, the best apps investing disproportionately their time compared to everyone else, because that's kind of where, where the best teams see a kind of an advantage that they can like build something significantly better than other people. And so what I think you're seeing now is like the best.

Apps in the industry, you know, like the Figmas, the Linear, Superhumans, et cetera, they're pointing now towards sort of state transfer, like, how do you load your app extremely fast? Because everything's cached locally. How do you have real time sync between, you know, different people? So you can build like, rich collaborative experiences. How do you, you know, save locally and then sync backwards? So you're not like, Doing little spinners all the time when you save stuff.

So it's like always fluid and fast. And they're all kind of arriving at the same idea, which is like, Hey, this whole like rest, you know, like posting things back and forth, pushing bites around manually, doesn't cut it anymore to build like really complicated, amazing apps, you need a higher abstraction. You need something more sophisticated for.

Handling that state transfer and, and that's sort of like this local-first and, or, you know, like sync engines, it's kind of like the component that sort of handles that, that state transfer back and forth between the client and server.

That's super interesting. So you, you mentioned the term sync engine that, that might be not familiar with everyone in the audience. So could you elaborate a little bit what a sync engine is and how that might relate to. The, the approaches that you've mentioned before. I feel like it's the best term for uniting a bunch of different tools and their approaches. But yeah, the basic idea is that REST API, Grackle API, TRPC, whatever. They're all based around rendering a page for widgets.

It's a widget page. And so you're like, Hey, API, give me the data from my widgets. And then I render it. So it's sort of an imperative step, kind of like manually telling the client, tells the server, give me that thing. Where sync engines, it's a a move from an imperative to declarative.

So instead of saying, give me that data, it's sort of like for this page, I declare, I need this data and then the sync engine makes it so, and often run into the problem where like, just like sort of getting data is easy, but subscribing to updates is hard. Like everyone kind of notices there's a massive gulf and difficulty between those two things.

And so what's cool about sync engines is that because it's sort of more of a declarative step, there's actually no additional complexity to get subscription. This guy is the same thing. You're just saying, I need this data. And then the data shows up. It's a new abstraction for how data. Arise on the client where, yeah, again, you're going from imperative to just being like, you're describing the shape of data that you need to show up on your site for a particular page.

I know another good analogy. It's very similar to shift from like jQuery to react where jQuery, you're like, get this DOM element and tweak the text. Or get this DOM element, remove the child, construct two other children, drop it in. You're like a little kid playing with like blocks or something. And you're like, you're like fiddling all the time, which was actually fun. I actually enjoyed it.

It was a fun little challenge, but it's a huge waste of time where, you know, we're moved to React and React is more like, I don't know, I want it to look like this and just sort of happens somehow, like React does a bunch of weird magic under the hood and it just kind of efficiently somehow shifts everything around. Uh, without tearing and voila, it's there. You're like, cool, it's there. And yeah, that's just a lot less to think about.

When you're just like, I want something to be so, and I don't really care about the complicated details. That's a lot less to think about. Which is the value of abstraction? Because we, we have a finite amount of things we can think about. So, if we drop two, then we can like, replace it with, Two other stuff instead. So if you drop a lot of thinking about state transfer, then you have more time to think about other things.

So you've now explained a little bit, the benefits of a sync engine and that it simplifies a lot of things. But for someone who's now like curious, how does this actually work? Can you explain what a sync engine does and how it does it? I'll, I'll use an example for the one that I've used the most electric SQL. So electric SQL is sort of Postgres based and you have, you know, Your tables and basically the shape is like kind of declared like this is the shape of data. I want polar.

It's just like a sequel query. So you're just like, I want. this table and rows that match this criteria. And then the sync engine kind of like takes that and then it says like, okay, I will send you sort of the initial bunch and then any updates that I see from then on that match that shape off it goes. That makes sense. So, and this would be one particular example, one particular technology for when your starting point is an existing Postgres database.

So is that the way how I should think about it? That like I have my And like traditional way of how I'm dealing with data in a more global state, for example, Postgres, and then I'll just slap a sync engine on top of it, connect my clients to it and off I go. In theory, in practice, all these tools are like, this whole space is pretty new still. So I think that's sort of the platonic ideal that everyone's working towards is like, you just sort of slap a sync engine on and off you go.

And that does work. In some parts, you know, all these tools are like, there's some limitations that you have to kind of live with and that they're working to, to lift, but there's understandably a decent bit of complexity to kind of like take this 30 year old, whatever old Postgres is technology and toss a new thing onto it.

As long as you live within the abstraction, I kind of, or live within sort of the existing limitations, then yes, that is actually the very, very most of the reality I've been building, like a number of sort of toy stuff, both to play with this local-first approach, Sinkage approach. But also, you know, exploring a number of other things and it is like very simple. You just say like new table and then it creates a client side library. Kind of like very Prisma esque.

Uh, actually they're still using, I think, Prisma under the hood to generate the client. And then you just immediately just start reading and writing from like the local SQLite. So because they use Postgres in the back end and then SQLite. It's very straightforward. Just like write a query and data's there, do an insert off it goes. So I can't promise it'll work for like, yeah, anything will just work.

But if you're living within sort of the current restriction, it is very much a magical experience.

That sounds great. And I think there is a lot of different approaches, how a sync system could work. We've also learned a bit about auto merge in previous episodes, but I'm curious to hear more about your experiences building apps with the stack that you've mentioned, can you tell a bit more about the apps that you've tried those new approaches? Uh, yeah. I mean, none of these apps have been, we've all been for like personal use, so they're not like anything earth shattering. The most kind of.

Complicated one is a one I'm just actually finishing up and my wife had been working on it and it's like a kitchen AI. So basically you, you can like take a photo of different ingredients in your kitchen and then the AI will then sort of like convert those into ingredients in the store and then you can set like, what is the fill level? When's the expiration date?

And then when you want to go do, do a recipe, you can upload, like you kind of copy and paste the recipe into it and it kind of like parses out all of the ingredients. And then what's interesting is like, it's actually kind of like ingredient or recipes all kind of describe ingredients slightly differently. So like just doing like a naive matching or something.

You won't actually work, but then, so I'm using like embeddings to, to kind of do a more fuzzy semantic match between the ingredients in the kitchen and like the recipe ingredients. And so then I'll say like, Hey, here's all the ingredients you have. Here's the ones you need to buy, but also sort of like tell you, it's like, Oh, Hey, you're running low. You should double check if you have enough of this one, or, Oh, this one is like expired, so you should probably just get some new spice.

So you're, you're, cause the old one doesn't taste very good anymore. Anyways. And then it's sort of a one click. You know, create a shopping list with, uh, we're using Trello. So just sort of like sends it off to Trello. Another cool thing that we did is like, we just give it an enum of sections in the grocery store. So it's like, there's meat and seafood and dairy and produce and canned goods and stuff. And so then each ingredient is coded to one of those sections.

And so then when you go to the store, you have this like neat list of. Uh, the different things, so it's actually pretty fun to use. Uh, it'd be pretty fun to build, but, but yeah, so, so the actual process of building it is, you know, the local-first stuff, the electric SQL stuff is defining my schema. Well, another cool thing about all this too, is it's very data centric.

I've noticed a big shift in how I build stuff now versus before is it used to be steady at the back end and like the database, everything was like annoyingly complicated that I just sort of delayed it until I was sort of more or less finished prototyping. So what I would do is like, just keep all the data in memory in the browser and then build the UI, try a bunch of stuff and just kind of like. Have a little JavaScript rays and objects, you know, whatever.

And then finally I was like, okay, now I kind of like know what my data structure is and everything. And now I'll just actually set up the database and start persisting it and whatever. But now it's like so easy with electric SQL to kind of.

Set up the database and have a client and have everything syncing that like, like the very first thing I do is write my crate, you know, table stuff and, and, and I have the tables and which is a huge improvement because it's very nice having a fully typed database, you know, I, because you have all the Postgres stuff you can like, you know, pull up PSQL and start writing queries and anyways, there's a lot of power you get.

For free and you have like type checking, you have, you know, foreign key constraints. So you have kind of all these nice things that database give you from day one while you're just prototyping. Normally what I do when I'm prototyping is I just blowing away the database and restarting. But that again is like 30 seconds or something, um, to do. It's also quite fun to like chat GPT is like great at like, Creating SQL.

So I use that a ton when I prototype and I just sort of like, just kind of freehand describe all the data I need, like, and then it spits out, you know, like five create table statements and I just like pop it in and off I go. It's very, very productive. Uh, and it's quite fun just to have a full blown typed database backed, you know, SQL first reactive system while you're prototyping.

Yeah. It's both like a very productive prototyping system, but what's genius about, of course, about the whole thing is like your prototype. That is immediately your production system too. Cause I can just like, as soon as I'm done, like, I don't have to like, fix up a bunch of data problems cause it's, it's been good from the get go. I just ship it and, and it works perfectly. And I'm not like, Oh no, I have all this like crap data now that's accumulated production system.

I took all these shortcuts, you know, to like work on my prototype. It's just like, it's just, you have all the database guarantees from, from the get go. I, I love that story. I think this is so powerful and the analogy you've provided before in terms of going from jQuery where you have to do everything manually to React freeing you up to no longer having to do all of this manually.

Now that really makes sense to me because now you could like you've, you've mentioned a few of like the, the new, like AI aspects of this, or that the app is an app just for you and your wife. And that's totally fine. Like you're basically now freed up by so much data wrangling of before you would need to like maybe set up a GraphQL endpoint, a REST endpoint, and you'd spend so much time just to have all the, the data moving stuff taken care of.

And if that now just works in the same way as React. Automatically working and keeping your views up to date. And now if we elevate to the next level of Muslow's hierarchy of, of needs for, for app development, and we have the data automatically taken care of now, you can focus actually on what makes the, the app fun and unique. So that makes a lot of sense. And I love your anecdotes there of that. You don't no longer need to think about.

There's this like, it's kind of like stage one, everything is crap in prototyping mode. And then at some point you need to make it real and production mode. But with react, you also don't really distinguish too much about like, Oh, am I now writing this for production or for, for prototyping? You're just doing it right. And it's easy from the get go. And if you get the same thing for data, that's super powerful. So I love that.

Yeah, that's actually, that's probably a good analogy is like, How powerful, how good a quality is, is some part of your stack as if, if the prototyping is the same as like production, like there's no difference. Just another random tool I've been using lately, like SST. I don't know if you've played with that, but they have sort of their live. Development, uh, Lambda function.

And it's like killer because you develop against an actual Lambda and they kind of like intercept the call to the Lambda function and send it to your like laptop. So you kind of have like debugging logs and everything, but your, everything is actually, Running in, in the cloud and it's like going to S3 buckets and everything. So it's like the development environment is the same as your production environment, um, which is like pretty hard to do.

Otherwise going from dev to production with SST is like, there is no difference. Like you, it's the same stack, but the react, it's like, if your components are good for development, they're going to be efficient production. And yeah, with electric SQL, like it works in dev is going to work in product. So yeah, which is, which is awesome.

That's great. I would love to, to hear a bit more about the details of how you've built this app. And, um, particularly you've mentioned that you design your tables and your schemas, and you've mentioned that you're using Postgres, uh, more for like a server aspect and SQLite For, for the client state, do you design one schema and that applies to both Postgres and SQLite equally? Do you distinguish between those? Yeah. Yeah. You create a table in Postgres and then they call it electrifying it.

And then there's sort of sync engine then automatically sets up that table for you in SQLite. And also like does migrations and all the other like complicated bits. You can do local only tables, but I've never needed to. There could definitely be stuff that's sort of like very specific to a client. Um, but generally speaking, like all the data, I just want synced all around.

Cause you know, again, one of the cool things about all this is that, you know, your desktop, if you have a phone running the app and you have a desktop running the app, they're like always in sync. So it's, it's kind of like you pull out a phone and it's like, got the data. In my mind, all data should be synced all the time.

So that kitchen app that you've been describing, that sounds super cool, but it sounded you've been building a few different local-first apps to really give this, give this a try and get started with, with that local-first paradigm. So which other apps have you been building? Yeah. And another app that I was pretty pleased with and I think is, uh, another good example of kind of the benefit of this like sync engine approach.

It's a very simple, it's a one, one page, one view app, but basically it's like I use Garmin. I have a Garmin watch for like tracking exercise, like running, biking, et cetera. And they have a bunch of like charts and stuff that they kind of like pre designed for you, but I was like, Hey, I want to like, look at it from a different way. But basically I wanted a pure volume. Based view of my exercise. So, cause they're, they're like emphasize like intensity or mileage.

And I'm just like, I just care how long I'm running or how long I'm biking, et cetera. And so I was like, Hey, they have an API. I'll just pull in the data, analyze it, create a chart off, you know, off we go. So the cool thing about it is that, so basically how the app works is like. You know, it's using electric SQL again. So it has all of your activities cached locally. So when you load it up, like it's instantaneous, it doesn't have to load a whole bunch of data off the server.

It's just like runs, click SQL query presents the charts and stuff. But then also when you load it though, it like it hits a Lambda function, which then syncs in any new activities from Garmin. Into the system. And what's fun about that is like, just again, just how simple it was. Cause all it does is it like hits the API and then writes out new data into Postgres, which is like, okay, that's fine.

But then the next step, you're like, well, how do you like tell the front end that like the new data is done or whatever, there's like a bunch of other stuff, but what's, what's great about it. It's like, there is no other step because the sync engines just was like, Oh, there is like a new Garmin activity pushed into the database. I'm going to push it to the client because it's subscribed to it and it's active. So it's just kind of like, it just shows up.

That's again, it's kind of an illustration of like, you have this sync engine. It's not just like reactive in the client. Like there's a lot of like, we talk a lot about like client reactivity and like, there's a lot of libraries that do that, but this is more like a global reactivity. Like the whole system is reactive. Like any change anywhere, it gets pushed to every other node that cares about that. And that's just like a really powerful leap forward.

Not just for the, like, Client server, but like, even like server server, like I thought a lot about like microservices, everyone's like, you know, like one big problem, microservices, they get very chatty cause they're always like, cause it's the same problem. They're always like pulling like, Hey, any new updates, any new updates, new updates, you know? And then they're like, anytime they want to do something, they're like, Oh wait, I got to go get some Jeff there. I get some Jeff there.

But with the sync engine, it's like, well, no, like each microservice can just say like, this is the data I care about. And it has like a local SQLite database running that it can query against whatever it wants and like every bit of data is like up to date within like a, you know, fraction of a second that just like would dramatically speed up a lot of sort of microservices set up and dramatically reduce the chattiness going on in between different services.

So anyways, I think there's a lot of global reactivity powered by SyncEngine.

Yeah, I love that analogy. I think this was really like what blew my mind about React in the, like, when I saw it the first, the first time, not just that it made my life simpler by not having to take care of like all of the dumb children, like appending, creating, etc. So it made everything simpler, but also having all of that just work automatically without me having to even like tell it like, Hey, now update.

That reactivity for the views, but also for like the local state, but applying that now for global data, uh, I think that is really a killer feature of local-first that I think is not well understood by, by a lot of app developers. It's just so funny because like it's the water we live in, you know, it's like, Oh yeah, when you're building stuff, you're just, you spend a lot of time moving data around and checking it and updating it. And it's just like endless little problems.

Uh, I remember like Gatsby cloud, we would like, it was such a common problem where it would like ship an update, which like, you know, use a slightly less efficient query or up the number of calls Internal API calls, and then all of a sudden, like the post, our Postgres database with like red line, like, and we like, ah, no, like, or whatever. And like, it was just like a constant headache because like, when everyone's constantly like asking for stuff, there's just enormous inefficiencies.

Cause like, I remember that was a big thing where people are like early on with React, people are like, oh, like. My jQuery calls are so efficient, you know, and like react is doing all this extra work and whatever.

But what pretty soon everyone kind of realized it's like, well, yes, like react does do extra work to accomplish the same goal, but it's only actually more work if every single jQuery operation was optimal, like you'd actually thought through the algorithm, you know, in detail for every single one, because in practice. Everyone was just doing lazy, weird shortcuts. You're like, you're like, I just got to ship this feature.

So I'm just going to blow it away and rewrite the whole thing or whatever. He said, we were doing all sorts of like weird, crappy things to, to, to kind of like update the Dom, which, yeah, it was just like very janky, very slow all the time and whatever. So, so, so react was like, yeah, it's like, it was an overhead to doing automatically.

But it was sort of like a consistent overhead and everything was consistently a little bit slower than optimal instead of like sometimes optimal, sometimes widely not optimal. And so the same thing with like moving data around.

It's like, you know, people are like, Oh, yeah, like, My data fetching is so clean and so good But you know, you have one engineer who just accidentally grabs five megabytes of data from their API and like, whoops, and then all of a sudden, like the whole thing falls to pieces where, again, with like sort of more of an higher level of abstraction, it's like, you just, it just can't happen because you can even put in checks, you're like, You can only ask for max of 300 kilobytes

in a batch or something like that. Whatever. I mean, there's all sorts of like. Smarter things that the system can do for you. All kinds of new abstractions of kind of like both face these same challenges of, of, or complaints. It's like, Hey, like I do it really well and you're doing it poorly in all these cases and whatever. But if the abstraction is like good enough, it eventually pretty quickly becomes. Better in most cases and often better in all cases.

And even if it's not better in all cases, like taken as a whole, it just eliminates a lot of really suboptimal problems. It's like memory management, you know, that's sort of another great example. Like people are like, I manage memory so well, like these GCs, there's lots of overhead, blah, blah, blah. And then like. 30 years of, uh, CVEs of, you know, security issues. It is very, very hard to build fast, efficient data loading and keeping it up to date and whatever.

After like, you know, spending all this time building local-first stuff, even like very good apps built with like great teams, like, you know, name a name, like Vercel's dashboard. I mean, they're pushing like react server components and all that stuff, but like you click around their dashboard and there's like endless little, like.

Loading, you know, like if they had a sync engine, they would know that the next few has all the data and it's up to date and they wouldn't have to like go back to the server, like, Hey, did anything change? Did anything change? Cause I'm sure they have a local cache, but like what they're doing is always like. Asking again on every click, they're like, Hey, has anything changed since the last time this person was here?

And that, like that request, like, or sort of that, that fundamental uncertainty with an imperative approach, like, just because you just can't, you can't, you can't know if, if, if the data is up to date, um, there's no way of knowing for sure. Cause there's no system that's like reliably telling you that like, yes, the data is up to date for this next view. You have to check. And that check. Take some amount of time, which adds glitchiness into your UI.

And there's just no way of, no way of getting away for that without, without a sync engine that can like provide the guarantee. I think it's really interesting. You've used the term, uh, thinking about your app data first, maybe as opposed to view first or, or react first. Yeah. I think that's kind of a big mental shift that I've also observed in my own app development evolution of my perspectives.

And I think this has been a really powerful step because it intuitively makes sense that if you don't have the data, then if you want to render it. And you don't have it, then you also can't render it. And so thus you render a loader or a skeleton, et cetera. Whereas if you think about your app. Navigation routing user experience overall more in a data first way to think about, okay, I'm currently here.

This is all the data that I need, but also think a step ahead of like, Oh, what if I would go to, to this other route here? Maybe I should already have the bare essentials in terms of the data for that, that if I'm going there, it's immediately available. And then you think about your data more, maybe you think about it as a graph or like a collection of documents, however, you want to think about the data, how it fits your app's use case.

But once you start thinking about your app data first, I think it's so much easier to build a really. high quality, fast user experience then, um, and it's also so much easier to make that available offline. I'm not sure whether you need Vercel's dashboard, which is gorgeous, but has a bunch of loaders, loading spinners. I don't think you need that necessarily to work offline. But another way to, if it works on offline, then it's crazy fast. And I think it would be great if it was a lot faster.

So I would, I would completely agree with kind of thinking about your, about your app development workflows as data first, as opposed to React or Vue first. So we've been talking a lot about the, all the good things now that you've experienced with local-first, but I'm sure you've also experienced like some pain points, some challenges, um, in regards to that new approach of building software or in regards to the technologies you've been using.

Tell me a bit about that, which sort of challenges have you been experiencing here?

Yeah, there's definitely a few. I mean, there is some learning curve. It's not the same as what was before there. There's a lot of things that cross over. So it's not like a huge step. You're going to have to like spend some time learning stuff regardless of whatever tool you choose. You're gonna have to like learn stuff for that that you don't know now. But I think there's also at this current point of immaturity in the market, there's not like a clear market leader.

Yeah. Yeah. Who's very good and mature that lots of people use. So, you know, like in in, in sort of we're like, Hey, next Js. Like that's sort of the obvious choice. 'cause the billion people use it and it, it's kind of covers everything and whatever. So you both have the, I have to evaluate a bunch of tools, a step to get, get into this world. And then you're like, and then I have to learn a specific tool in depth.

'cause there's, there's, it's just different than, you know, like no one's done it before. Almost nobody's done it before. So you're going to have to like learn stuff to do that. So that's definitely a bit of an overhead, uh, in this approach. And another result of the immaturity is like, there's just things that don't work. So there's sort of the magical, like I'd be kind of describing like sinking, just like, Oh yeah, they're just sort of magic, but they're not actually magic. They actually.

are an engineered product and there's stuff that they just don't do yet, um, that you'd rather like them to do. And so it's, uh, you know, learning those limitations and like working within them or even, you know, even decide if you can or not is part of sort of the evaluation process and building process. So far for my toy apps, it hasn't been an issue, but like, there's definitely things that I'm like, I can't build those things with this stack right now.

And like, yeah, the engineering, you know, the teams behind these, like they know about it. And like, they're working hard to, to, to fix them. A lot of stuff is kind of like just the same. It's just building a map. You're just like, I need data and data's there. Before I'd write a REST API call or GraphQL call or whatever. And on the backend, I'd write a query. And now you just write the same query, but it runs locally.

And the day to day, just like building views and stuff, it's extremely similar. You know, it's like before you'd probably like to sort of hash out like the, the data locally and the client. But now I just started like, Oh, I need to like change my data model. So I'll just tweak the, you know, the database structure a bit and then, you know, change my query a bit in the front end. So yeah, I mean, day to day, it's like, it's very similar, but just like less steps.

Cause you don't have to like change your GraphQL or you don't have to change like the client call. It's only like two steps. It's like you change the database table. And then you change your query in, in, in your, in your component. And then that's about it. So it's pretty, it's pretty simple. That makes a lot of sense.

And I mean, just overall trade as for, for most early technologies, if you're early to those technologies, you bet on some superpowers and you probably acquire those new superpowers earlier than most of us, but you also got to pay a higher price than most of us.

Yeah. It's just like, yeah. I think the biggest thing is like. I just randomly have gone into like, Oh, do, do, do, do, do, I'm doing something. And then like, bam, I hit something and then I'm like stalled for like a day while I either figure it out or, or figure out a workaround. So I think it's, it's, it's kind of like if you're using boring technology. It's like, it may be tedious and hard or whatever, but it's sort of like a known tedium.

You're like, this is going to take me like one week, where with a new technology, it's either going to take you an hour or two weeks, you know? So, uh, it was a lot more uncertainty on what's going to happen. And you might run into stuff that just like absolute blockers where again, with the old stuff, it's kind of like every single possible work around. Has been figured out for every possible thing.

And so it's just a matter of finding that obscure forum post from like five years ago, where they're like some weirdos, like, Oh, I spent five days on this, but here's the fix. And you're like, freak. Like we are, we have a kinship. You and I know we two people are the only ones that have run into this problem before, but thank you. You, you wrote it down for me, so I don't have to figure it out myself.

So, so yeah, the new technology, it's like most of the problems you run into, you are the very first person to run into it. Which is fun. It's sort of like, yay, you're like, I don't know. But yeah, it's, it's, you're gonna have to be like a pioneer, sort of a more robust, I don't know if Europeans talk about pioneers, but anyways, like American pioneer, you're kind of like, ah, we're going off into the wilderness and got an axe on our shoulder and, you know, whatever.

We're just gonna like make it happen. You kind of have to adopt that, that mindset, um, when you're kind of into new technology. Yeah, that's very relatable for me. I've always considered myself to be kind of like a technical pioneer. So what you've just mentioned in terms of, yeah, sometimes like it's a gamble and sometimes like it's the, the sort of promise of everything will be simpler and easier in the future that plays out.

And you can't believe like how the thing that just took so much effort and so much work in the past is now so easy, or you run into a wall Or in theory, it would have worked, but you run into some edge case that you're the first one running into. And it might be fixed with the next release of whatever you're using, but you still have run into it. But luckily you've run into it. And then the next person is no longer running into it, depending on how quickly someone releases.

Yeah, there's there's sort of a sense of community service you, you, you. Get, or at least it's helpful to get with new technologies. You're like, yeah, I'm going to pay down the cost for everyone else a bit. So another observation here, I think it's really interesting that the use cases you've mentioned, your ambitions for those apps. We're not to like take it to market and like roll it out to millions of users.

It was, I think like part of the fun that you built a software that's just for you and you can make it as fancy or ugly as you want, like no, one's going to complain. and you know like the little quirks, uh, of it and that it took you less effort to make that happen, to build the thing you wanted to build and like spend less time with like all of the boilerplatey things. That's great. And I think that, that is totally fine. Like no one's going to take you, take that away from you.

Uh, I think what's still a little bit less proven is like, how do you really If you were to take that app now and say, I do want to scale it to millions of users, et cetera. I think that's another aspect of where we are still pretty early. And on our pioneering journey, we have not yet figured out like hit all the traps. But, uh, I think from first principles, I think, uh, this approach should actually work out.

I don't, I don't see. Any like showstoppers for sort of scale to millions and a lot of ways again too. It's like, it's actually no apps do scale to a million. What happens is people build it and it falls apart and then they like rebuild it several times and get to millions. So what actually I think is actually pretty interesting about this and like promising is like there is a story here prototype to millions with like no changes to your basic architecture.

Which I don't think anything else can make that claim, but like the base system of like what everyone plans to do is like you can horizontally scale sync engines without any issues because you can just have sticky, sticky connections from clients to, to particular sync engine and it's managing the state for each user. Postgres is easy to vertically scale. What's interesting else, but like Postgres actually has like a lot less demand in this world because far fewer queries are reaching it.

So it's actually like you don't have to scale Postgres nearly as fast as you kind of typically expect to, you know, with this approach, people either do sort of normal API stuff because you can still just call Lambda and do whatever and write to Postgres. Can I was talking about the Garmin example, you can write locally, like, I want to do this sort of our sort of event sourcing style where you're like, the client says, I want this to happen. And then the back end.

Serverless function picks up the job and then kind of writes it back and then it gets synced back to the, the client anyway. So that, that, again, that approach is also extremely scalable because it's all serverless and stateless and you can have as many workers, you know, spinning up to, to handle stuff for people. But yeah, we're not there. This is all very, that's all very potential. But I think what's extremely promising is like, you know, what are the different stages?

What are the three major stages? It's like how efficient is a tool of prototyping? How easy is to go from prototype to production and then how easy it's to scale. So those are like three distinct problems. So problem line prototyping, I think both of our experiences, it's a lot easier to prototype with a stack. Like it's data centric, you have like a full type safe client, you know, from the get go, a lot of boilerplate is gone. Like you just like, bam, you go.

And then again, like prototype production, extremely easy. Like I, like I've done lots of side projects in the past that I always kind of came down to it. Now I'm going to. Deploy it. And I was like, uh, I have to rewrite so many things and I have to like, do all this crap that I didn't have to do before. Like, Oh no, where's the steak going to go? Whatever. Like I I've gotten, I've gotten my sort of go live checklist. It's like five things and like, it takes me less than an hour.

So it's extremely fast to go from like prototype to production. And then, you know, production scaling. Yeah. That's, that's, that's the part that doesn't really exist, but in theory, like, I think the architecture is like. A very scale free, like, it's mainly just a sync engine that, um, needs. Proven at scale. I mean, I do think that I fully agree with the way how you delineated that into those like 3 steps.

Um, but I would even say that the scaling part I think has been proven by quite a couple of companies by now, whether, uh, Whether it's like thinking about Replicash, uh, Rosicorp has built with Replicash and Reflect, et cetera. I think they have a couple of customers that are really going at scale and the entire system I think is quite inherently scalable and, uh, the, it doesn't change the, the simplicity of the system, which is, which is the beauty.

Um, and a lot of, a lot of the, a lot of the parts of the system can still be considered stateless, which makes it a lot easier to reason about. linear, linear had a really great talk about, uh, linear CTO, uh, had a really great talk about, uh, how, how they scaled. And it's, it's pretty fascinating, like all the things that they did, but yeah, to your point, people have gone down the path. So, you know, innovation kind of goes through, like, is this thing even possible?

Or somebody has to, like, discover something. Like, there's this like, oh, there's this new thing that we can do. Wow. Who knew? And then it's sort of like. It goes from that to then sort of eventually to sort of a product where it's kind of like that technology is then encapsulated to something that anybody can use. So the fact that multiple independent teams have successfully gone down this path shows that then sort of now. What we're seeing is sort of a productization of sync engines.

If you want to do a sync engine, you have to like buy it. You don't have to build it. You can just buy it off the shelf. Um, I think we're pretty close to that being a reality.

So you've been going through this journey, I guess, over the last year or so of like learning about local-first and step by step building your own apps with local-first. What would be your advice to an app developer who's curious about local-first, but has not yet taken their own first steps? What would be your advice to them? I don't know. My, my, my general advice on any technology is just to dive right in. I mean, there's lots of stuff you can read about it.

So, so by all means, like, you know, listen to cool, cool podcast. There's lots of blog posts. Yeah. So it was definitely lots you can read and listen to and watch and stuff. But, um, there's no better way to feel it, to like, to understand it and to really feel it than just to like start building some stuff. All the different tools have starters and they're very approachable. They're not like particularly hard to use. And, you know, Take a day or two and build something.

Cause yeah, it's really just like, when you just see it sinking between tabs and you're just like, you're just kind of like, you, you really kind of have to feel whenever you're just like used to doing something. You just, you just forget about it. We get numb to all the stuff we do. So you really just have to like.

To really feel the difference of local-first you just have to build something and then you're just like where's all this stuff that you just sort of feel voids in your in your soul you're like you're like where's all the missing stuff you know like where's all those things I normally do and you're just kind of like oh they're just gone and then like after you've done a few you're kind of like You just start to forget about it.

It's like this bad, weird dream of like spending all that time on all those things that are now gone. But yeah, you just, you can't really appreciate that until you've actually built something. Uh, let's see whether in five years from now, we look back into the days where we haven't built, been building with local-first and we see it just crystal clear as that analogy you've given before.

With going from jQuery to React and React freed up, uh, freed us up from having to think about how to render the views in the correct way. And now we get the same thing for data. So this is beautiful. Now, yeah, talking, talking to younger developers, like sometimes I mentioned jQuery and I'm like, Oh wait, they've probably don't jQuery is.

And it's just a very weird feeling to be like, Oh, yeah, just jQuery thing it was like this and they're just like, yeah, okay, whatever like yeah It'd be it'd be really amazing if like apis was the same thing.

Like yeah, we used to like You'd write like an HTTP method and then you sort of like make up a name that sort of meant something and then you like how to like remember to like write it all the same thing and your client you know you're like it's not a post it's it's a put and you have to like no typos and they're like that's it's like a lot of tedious, fragile, annoying code to write. And you're like, yeah, it was, it sucked.

And they're like, I just used like the library and I just sort of typed stuff and like, you know, Copilot v10 just sort of like spits out whole, whole, whole stuff, you know, cause it just introspects on my whole data system and like writes out super efficient stuff for me. And yeah, I, I, I'm very big on like, you know, getting rid of work we don't have to do. Um, and I think there's a lot left in programming that you really shouldn't have to do.

Let's leave it here. Anything you want to share with the audience? Have fun, build cool stuff. That sounds great. Kyle, thank you so much for coming on the podcast. This was a lot of fun. Yeah, it was a lot of fun. Thank you for listening to the localfirst.fm podcast. If you've enjoyed this episode and haven't done so already, please subscribe and leave a review wherever you're listening. Please also tell your friends about it.

If you think they could be interested in local-first, if you have feedback, questions or ideas for the podcast, please get in touch via hello at localfirst.fm or use the feedback form on our website, special thanks to Expo and Crab Nebula for supporting this podcast. See you next time.