#17 – Kyle Simpson: Local-first identity

if we build a world where it is very compelling and straightforward for people to start building in peer to peer communications into their apps, we really start challenging the question of Why did you need the cloud in the first place? For most things, you didn't need the cloud except that you did not have a way for two clients to speak to each other. But if we get rid of that, then you stop needing to pay for most of your cloud bill.

And that is one of the largest overheads that companies face these days, and they're increasing in cost.

Welcome to the local-first FM podcast. I'm your host, Johannes Schickling, and I'm a web developer, a startup founder, and love the craft of software engineering. For the past few years, I've been on a journey to build a modern, high quality music app using web technologies. And in doing so, I've been falling down the rabbit hole of local-first software. This podcast is your invitation to join me on that journey. In this episode, I'm speaking to Kyle Simpson.

A prolific JavaScript engineer and author of the book, You Don't Know JS. Over the past years, Kyle has been researching user identity and encryption in a local-first context, which we explore in depth in this episode. We start the conversation with a story that led Kyle to local-first, including what he calls web 2.5 and zero servers. Before getting started, also a big thank you to Rosicorp and PowerSync for supporting this podcast. And now my interview with Kyle.

Hey Kyle, so nice to have you on the show. How are you doing? I'm great. Thanks for having me. I'm thrilled to be here. You've been digging into some aspects and some corners of the local-first ecosystem that I think not as many have yet really dug in so deeply, namely around local-first identity, encryption, et cetera. So I'm really excited to get into that, but, not to get ahead of myself.

Maybe you want to briefly introduce yourself, give a bit of background and, maybe share how we, how we get to talk about this today. Yeah. my name's Kyle Simpson. Most people online, and especially in the web and JavaScript community, will know me by Getify. I've been around for a very long time now, and, probably some of the biggest things that I'm known for is having written the You Don't Know JS books. the first edition came out in 2014, 2015. That's a series of six books about JavaScript.

And then I did a second edition, some of those books that we wrote in a second edition, around 2020. so that's how most people know me. I've done a lot of teaching, a lot of conference speaking in the web and JavaScript space, and kind of tried to promote the web as a platform. And JavaScript is a key piece of that platform. Specifically with local-first, my interest, I arrive at local-first kind of almost accidentally because I was working on things that I thought we needed to create.

And I didn't know that there was anybody else that had come up with terms or pioneering this. So I sort of separately was trying to invent something not that dissimilar from local-first. And I had a series of like specs that I was going to try to write around it, but that comes from a passion for why we need to own our identity, as opposed to offloading that to GitHub, or Google, or Microsoft, or whatever. We need to own that.

We need to own the control of the data that we create, and the data that is created about us, to whatever extent is possible. And so that kind of got me interested in this space, this particular usage of the web platform, and of JavaScript as a technology. I found the local-first space through some employment, and then started getting involved in that. community, the meetup that runs on the discord and all of that. Met Jans who runs that.

And so that's what brings us to today is that my passion kind of started over the last three to five years, really trying to leverage what I've been able to do and what we can do with this platform for the good of moving the web forward, I believe, back to us owning our identity. Awesome. Yeah. You've mentioned that I think you had now two opportunities to, see a couple of like different products in the local-first space.

I think one of them would be Socket Supply, which is, exploring peer to peer software, which is really interesting. And then also the AI startup, Vella. I'd be interested in hearing a little bit more what motivated your move to work at Socket Supply, like given the focus on peer to peer, maybe there's an interesting backstory here.

Yeah, there is a little bit of, it's a winding road, right? There's not a straight line path. so going back several years when I was starting to get interested in this, I was looking for a job at the time because I had gone through some layoffs and I'd been unemployed for a little while. And the company that ended up, interviewing me, they reached out to me. interviewed me and then heavily recruited me, was actually a Web3 company, a crypto company.

Now I'd known about the Web3 crypto space for a while, had felt like most of it was not something that I would super get behind and endorse, especially the currency side and all the, you know, the speculative financial instruments and all of that, not something I'm into. But I was interested because this particular company builds smart contracts and I do think that smart contracts are one of the core gems that I think Web3 could bring, some real improvement to the world of software.

So they recruited me and they specifically asked, can you come and join us? And because you're so big in Web2, can you help us attract Web2 developers? That was my role. I wasn't really an engineer.

I was really more almost a dev rel, but I needed to learn the space and so I spent that four to six months I was there trying to immerse myself in the web3 world and understand that and in no way do I want to say anything bad about web3, but I'm not here to support it in any way because I ended up leaving that world because I didn't fit.

I didn't fit because I don't believe that the world that's being spun in Web3, which by the way comes from some pretty deep roots in like the genre of cyberpunk fiction, and I read some of those books to like get myself into the mindset of this world. I don't believe what they believe, I think, which is that they believe that the world's current track is inevitably going to lead to ruin, and they've created the only oasis that anybody can jump to.

And they believe, I think, that it's not only inevitable that people will jump, but that it's self evident. That people will make the jump. If just more people hear about it, they will obviously say, let's do that. I didn't fit in that world. I'm not criticizing the people that believe that, but they just kind of papered over that gap that exists between web2 and web3. So over that four to six months, what codified in my mind was there's definitely some things that we need to work on.

They don't look like Web 3 at least yet. And I left and I told them I'm leaving cause I'm going to go build what I'm going to call Web 2.5. If we ever are going to get to Web 3, which I don't know if we are or not, not my say, but we have to take an intermediary step. We're not going to make that jump. I feel confident in predicting that we will not simply make that giant leap over the chasm. So we can talk about what I define as Web 2.5.

But those ideas came from kind of swinging the pendulum too far, I think maybe, towards that decentralized world, which is what Web3 is all about. And then saying, let's swing it back a little bit and figure out some way to achieve that with what the web currently is and building on top of and evolving what the web currently is. That is what led me to Socket. Socket supply. They, build a mobile, tooling set and framework.

others in that space are Tauri and Ionic and way back in the day, PhoneGap for building hybrid applications where the UI is built around a web and in a web view with a backend that is native and extending the native capabilities into the web space, that's what they do. I was very attracted to that specifically because of their ability to allow a web application to have full peer-to-peer communications capabilities.

They exposed UDP based, relay based peer-to-peer protocol directly to a web application. I felt like that was a big, it still is a big missing piece for the web platform, and it felt like the way to move forward. Not that I love native apps and I just want us to all build only native apps. I really do love the web platform, but this felt like a good compromise.

If we can build an app in the web platform and then wrap A rather thin wrapper around it to give it these extra capabilities that for some reason, the web just won't give apps right now, then I think we can actually move forward if we can have true peer to peer communication. So, join Socket Supply to work on that and in particular, to work on putting encryption into that peer to peer relay protocol. That's what I spent a lot of my time there doing. I also did a bunch of DevRel stuff.

I'm no longer with Socket. I was there for about 10 or so months. I'm not with Socket anymore, but that was part of my journey. And then several months went by where I was unemployed after leaving Socket before, the founder of this Vella. ai. he's the one who runs the meetup in the local-first space. He came to me and said, I've seen you active in the local-first community. would you consider helping me co found this? And the story that he spun was incredibly attractive to me.

It seemed like a really obvious place to attach my passion for identity. What he said was I've been working on smart assistant. It's going to use AI, but I really believe that AI should work as much on the device as possible. Given the constraints that we have in devices, put as much of that on the device as we can, because this is really sensitive personal data that doesn't need to be just slurped up into the cloud. And that really resonated with me believing in local-first.

And more importantly, if you're going to build apps like that, where you have your data on your device and you have it on multiple devices and you really are trying to break down some of the silos between apps like mixing your Google data and your Apple data and your Spotify data all into this one thing.

What was really obvious to me was we're going to need a way to, if we're eschewing all of those services, then we're going to need a way to define our identity so that we can keep things straight. And that identity needs to be really local-first. So that's what attracted me to Vella, was go work on the identity piece to try to help their story, progress. I spent several months with Vella, they're a great, group of folks.

Fortunately, we just I didn't have enough funding to keep me working in a paid capacity there. I'm still advising them, still wishing them the best. And they're, they're continuing on in their sort of smart email inbox app to rule the world. And I hope they achieve it because I want to use that app. I look forward to that. So that brings us kind of to today. Vella is the reason I started putting together some of the puzzle pieces for Identity.

I was going to build it for Vella, but I'm just building it right now in the open source so that hopefully these Lego pieces are something that apps like Vella and others, hopefully in the local-first space will pick up and build upon. Perfect. Well, that's quite the arc of like different chapters from peer to peer to locally running AI, et cetera. I'm eager to get into each of those, but maybe taking a few steps back, you said something that captured my attention.

I'm also not a person who's a big believer in Web3. I'm rather skeptical. If it happens and if those ideals prevail, I won't stand in the way of it. But I'm a little bit, just skeptical of like what it's the bad stuff it attracted so far. And so I'm very interested in the good ideas behind that. A lot of like interesting research that came out of that. But I agree.

It is too far of a step, too of a radical step from how things, work today and like how practical everything is today to a much more idealistic, but in many regards still, to me, it seems like impractical, A to B. So if there's like a middle ground there that maybe has a bunch of like the more attractive things from the future, from that Web3, Utopia, who knows? and you framed that as Web 2.5 . I'm very curious how you define that and, how you then also, then connected the dots to local-first.

Yeah, so speaking specifically about local-first, just because I know most of the people that'll be listening to this do understand local-first, but I want to re emphasize of the seven points of the original Manifesto from Ink & Switch I think the kind of, narrative arc that comes out of that is that we have built software for the last 20 or almost 30 years with an increasing mindset towards server and cloud first. We build apps where we, architect and design what that's going to be.

And then we create extensions of that experience into the client. And I think what local-first says is let's flip that paradigm. Let's start with what can the client experience be and what should it be? And where should that data reside in the client? And then optionally, you might choose to enhance that experience with a cloud or server experience. Flipping that default assumption where you start what you prioritize when you design.

I think that's one of the most important aspects of the local-first manifesto and movement. And I would say that that requires you to ask, what does my app do in a zero server environment? How does my app behave in a zero server environment? Zero server is, that's my term, not a generally accepted term, but here's how I define it. It is either that there is no server in existence, Or, that server effectively doesn't exist because there's no connection to it.

And there's a thousand different reasons why that connection might not exist. Might be down, might be the company went out of business, might be flaky internet on the client or the server or both. But, Zero server means this app is on its own. There is no connection to a mothership and how's it going to behave? And it really should behave like a full fidelity experience. Here's my, go to example for this. Banking apps.

We all know that banking apps are online apps, and they're all built with the assumption that when you log into your banking app, you want to see the most instantaneous, up to the minute information about your current bank account balance. And they certainly own the source of truth about what your bank account balance is. No argument there. However, there's a ton of information and functionality that is not requiring live updates of data.

There's all of my previous banking history, years and years worth, tens of thousands of transactions, some of which might be useful if I'm preparing taxes or I'm trying to analyze my budget or figure out what I spent things on or whatever, and I don't need a live internet connection and a live up to date bank balance to do that. But as everybody knows, you can't log into your bank app without an internet and get in access to that data.

It's very much my data and I very much still have the app on my device and I can't pair those two together because there was an assumption that the server was a required piece of the puzzle. And I think that's just a failing. It's a failing of business models. It's a failing of product design. It's a failing all the way across the board. To have not prioritized, assuming I start with this and then we add on later.

That bank app could very easily let me see all of my data and have a very prominent icon there that says, this is not your live bank balance or just hide the bank balance if they were really worried about that, but they could very easily give me all that functionality on my existing data that was kept on my device. So that's one of the reasons I resonate with local first.

And in a way, it's also the simplest use case where basically all your transactional history is typically like append only and immutable. It's not that your bank's going to say like, Oh, actually like this 10, 000 transaction back there, like it's never happened but all of that is like basically set in stone or should really be set in stone. That's like the nature of the domain. So this should make it the simplest use case actually to support this. Yeah. The CRDT there is pretty simple.

In local-first terms, there's not a lot of merging. You're right. That's a, that's a good point. Like the technical barrier is not really there. It's, it's a failure of business model and a product design.

So, I very much believe that we need to do those things, and, I call that version of the web, or at least that's one pillar of what I call the Web 2.5, a web that stops being drunk on the idea that the only way to build an app is to start with cloud infrastructure and then work your way back to the client, but rather start with client infrastructure and maybe or maybe not work your way toward a server. I think there's a ton of apps that never need any servers whatsoever.

The probably most motivating example that got me interested in this space was going back several years. There's a landmark, you know, world changing legal ruling in the U. S. that I'm, where I'm at, from the Supreme Court that eliminated federal protections for women to have the right to choose abortions, and other types of, healthcare. And I'm not a uterus having person. I can't. Get pregnant or have an abortion, but I absolutely have people in my life that I care about that can.

I have a daughter, I have a wife, like, this matters to me, and there was actually a case. It wasn't just sort of made up. There was actually a case where data that a woman had in a period of menstruation tracking app was Captured and used against her by government and law enforcement authorities. And that is just like that is so quintessentially the bad future that we never should have built. We never should have enabled. And it just snapped in my mind.

Like we have the technical capability to let people create really useful and engaging experiences around data that they own, that is about them, that is very personal, and there's no reason that data ever has to go out to somebody else's device because that's the trap. I make the assertion that if you do not have custody of your data, then you do not have ownership of your data. Custody is the whole game.

So when we give it out to those servers, then of course you're going to have governments that are going to find either ways to backdoor the encryption or legally force them to get access to that data and they're going to use it against you. And no matter where you are on the political spectrum, no matter where you are on the techno spectrum, you must agree with me that that's not a future that we want to build. Where they can do whatever they want with our data and manipulate us.

That can't be what with a rational level head says is, is what we ought to be building. But we have built that and we've gotten dangerously close to that. So Web 2.5 is saying we need to back away from that, completely reverse that paradigm, put data first. On the device, the way local-first suggests, and don't even build a server if it doesn't make sense. But if it does make sense, be restrained in doing so. That's what I mean by zero server.

And I think that's also a really interesting point about like health apps more, more broadly. I mean, a health app is typically I'm inserting some data about myself, or maybe with like a smart device that tracks my heart rate, et cetera. And there's probably not a good reason to have that data be synced to a remote server. Maybe I want to have it, maybe I have like a Apple watch and I have like an iPhone or on some other platforms, maybe those things should sync between each other.

But that a third party is the kind of de facto source of truth where all of that data is stored. I think the only explanation for why that is the case is because it's been so much easier to build those sort of server centric applications. I think I might take issue with you in saying that's not the only reason. That might be the most convenient excuse, but I think the most salient reason is because there's much more money to be made if my data sits in their server versus on my device.

And I think that's what's driven a lot of it, but I think you're right that We have absolutely created a technology landscape where the easy paved cow path is to do that. I agree with that. I think those are definitely two major points.

but I think even if someone has more of like the, aspiration to build something in the interest of a user and is maybe not after turning this into a billion dollar monopoly, then it's still a lot harder to build a non server centric application, at least in the past. And I think slowly but surely that is starting to change with all of those amazing local-first technologies. But yeah, the example you've mentioned is certainly a very stark reminder that we have to do something there.

So that's one of the pillars of Web 2.5 is We need to build a Web 2.5 where the paradigm is that we own our data and we own our identity so that we can control that data and say where it goes. Sometimes it's going to go to my devices, like I call it my local cloud or my local ring to borrow the term web ring from. 30 years ago that I'm old enough to remember. Your local ring is all your devices. It's your watches and tablets and laptops and phones and all that.

You definitely want to share data there. And that explains why I was interested in peer to peer technology, because that is where that is how that data should get. between my devices. By the way, I'll just, I'll make a plug. Aside from true networking based peer to peer, which is what Socket was trying to do, and what their protocol I think enables, whether you ever use Socket, I think they just have a great protocol that we could build upon that's relay based and self balancing.

I think there's a lot of, really good stuff there that somebody should, should pick up and run with. But whether you use that or not, there is actually a working group in the web standards community that's trying to work on what's called local peer to peer. I'm very excited about this. they're trying to build an API into the web platform that will tunnel itself over a couple of different transports. One being the local Wi Fi that Apple and, Apple iOS and Google Android have.

They're competing standards, but they're going to try to create a web API that papers over the difference between these standards. It uses your Wi Fi connections to connect between devices. Failing that, it can use Bluetooth or NFC or any other of these, like, local, localized transport layers to be able to communicate between devices. This would be, I think, one of the most important things the web could ever land if this happens. There is a working group on it.

There's a kind of a spec already working its way processed through. I don't think we're going to have it in six months. There's going to take a little while, but I'm very excited that at least finally, there's some movement towards let's not lock down, the web. And by the way, one little side note, there's a whole bunch of stuff that the web platform should enable, but does not enable.

Locks down or hobbles because we are, for some reason, and we, I mean, as a community, and especially the spec bodies, we are unwilling to admit that there is a fundamental paradigm difference between, I've visited a web page one or more times, There's that experience, which definitely we should be distrustful of. We should keep at arm's length. There's a difference between that experience and I've gone to a website a few times and I really like it. And I installed that onto my device.

Once I've said, I am installing that onto my device. I am saying, just like with any other app that I install, please let it have all the capacity that I'm willing to give it. Pop it up and have it ask me, do I want to give it this, and this, and this, and let me say, yes, I want to give it those permissions. I don't understand why the web thinks of itself as a unified medium when it is used in these completely different paradigms.

We need to be forking what we allow in the web platform, based upon that flag, has it been installed? Once it's been installed, I think that is very much implicitly the user saying, I trust this app. Let this app do what I want it to do. So things like local peer to peer, they don't make sense for a drive by website, and you should rightly be scared about a website being able to like talk to your watch.

Like I get that, but if it's an app that's built with web technology that a user trusted enough to install, it should have this capability. So just a little, side rant there, but anyway, peer to peer communication, the local ring. We absolutely need to do that, but none of that needs servers. We can build this without servers. I love those points and I was not aware of this, standards work body, in working in favor of bringing more local peer to peer supports for the web.

I think this is very, very much needed. Right now. I think the closest we got that also, makes that a little more focused on my devices is like I'm using tail scale to create sort of like a secure bridge between my devices, but that is still on an operating system level and not something that the web platform can easily, make use of to my knowledge.

But the other point you've been making about the distinction between a website, which you might just open once or a few times, maybe for a webshop or for like some, news outlet that like just has everything littered with ads. And so surely you want to be much more careful about providing too many privileges to that. But if it's, let's say your mail app or other sort of apps you use on a daily basis, sure.

You want to give that more access and possibly also in a way where, there is more of like a local AI context on your device, maybe even take a step further and provide a way how a native web app, like a native feeling app that lives in your web browser, maybe can even get access to that. I feel like we're making some steps towards that.

so for example, for Overtone, the music app I'm working on, I've actually just over the course of the past couple of weeks, I've added support for file system mounting, which is not available across all browsers but, Chromium based browser supports this. I believe Oprah also supports this. I might be wrong on this one, but I've been primarily adding support for, because I'm using Chromium browsers myself. Are you talking about the OPFS? So there's two things.

There is OPFS, which I'm already using as a persistence mechanism for pretty much everything for also for persisting my SQLite database and images, media assets. But there's also. a feature that has mostly the same API as OPFS, but instead of just creating a new file system, like a folder or files within the virtual file system, you can actually ask the user. to bring in an existing file or existing folder.

So, in my case, I would allow the user to select their music directory from their local computer, or possibly even from a NAS device. And so this works both for sort of like the show file picker model, as well as for drag and dropping, an existing file or existing folder. So this already is a big step towards like a more native feeling experience in the web, but there's so many more aspects to this that we haven't covered yet.

networking certainly being a big one, whether it's just TCP connections, UDP connections. One thing that I'm also somewhat hopeful of that this might bridge the boundaries a little bit is to rely on browser extensions to be sort of like a thing that you could trust more and that could facilitate that additional privileges for a given website or a given web app. That's something I'm thinking about whether I should maybe explore that for Overtone to give you more native like affordances.

but yeah, that's maybe a topic for another episode. I've done some work in that. I actually built a web app and built a browser extension that I could use with that web app and extended the capabilities in like audio and visual sense. So I've done some work with that. There's a lot of challenges there, but I do think there's a lot of potential.

and in particular, one of the things we'll talk about, I think a bit later in this episode, I very much intend to go down the route of building, Whether they be extensions or just side companion apps that do give extensions of capabilities to normal apps, because there's too many barriers right now.

I do just want to say, I only, have described so far one pillar of how I define Web 2.5. There are two more and I don't, they don't need quite as much time, but I do just want to state kind of for the record, what, how I define it, it's just a term I made up, people may not like it, but it's, it's how I define it. So the first pillar was we do need to build a web where data is ours, identity is ours, and we can choose where we share that data with our local ring of devices.

We can choose if we want to share it out to cloud for other, you know, back, there are reasons why you want to be able to back things up or whatever. Point number two, I think this speaks actually to, a lot of the skepticism around local-first, which is there's no business justification for doing local-first, if it's harder, if it's more risky, if it's newer and weirder businesses won't do it, and if businesses won't do it, it doesn't matter.

You can have all the great ideas in the world, but it won't matter. one of the things that I liked the most about my time at Socket Supply is they really confronted this head on. And they said, if we build a world where it is very compelling and straightforward for people to start building in peer to peer communications into their apps, we really start challenging the question of Why did you need the cloud in the first place?

For most things, you didn't need the cloud except that you did not have a way for two clients to speak to each other. But if we get rid of that, then you stop needing to pay for most of your cloud bill. And that is one of the largest overheads that companies face these days, and they're increasing in cost.

No matter what your provider is, whether they're more specific providers like Vercel or whether they're more broad providers like the AWS's and Azure's and Google's of the world, your bills are going up because you're doing more and more of your stuff there. And one of the reasons you're doing it is because it was technologically easy to do so and less risky. But the other reason you were doing so is because somebody hadn't given you a business reason to do the reverse.

one of the great things about peer to peer designed systems is that the bigger the peer to peer network, the lower the costs go. That's the opposite of, systems that are centralized, the bigger the centralized system gets, the more expensive it gets. You don't get economies of scale and you know, you're driving down your AWS bill, it goes up the more you scale. But in peer to peer systems, you have this inverse relationship. So I think there's actually a really compelling business case.

For why companies who are trying to figure out how do I cut down on my costs might be asking themselves, is there any way that I could offload any of my cloud bill to peer to peer communication? And even if I did just a little bit to start with and then realized, oh, that was great, and then I expand that. The more you expand that capability of your apps, you will create. More compelling user experiences than you will create in local-first. And that's important.

But you'll also significantly reduce, and maybe in some cases, completely eliminate the footprint of your cloud overhead bill.

So that's number two, that I think there is a really compelling business narrative to building a web where businesses Can more easily compete without almost the rent seeking, landlord seeking, Oh, we gave it to you for free when you were small, but now you're big and we're going to just charge you an arm and a leg, and you can't get out from underneath us, kind of thing like that. That's how drug dealers work.

And that's just not a business model we ought to be building the web on, in my opinion. So we have to fix that. That's pillar number two. And then pillar number three. I maybe personally feel the strongest about, even more so than the ownership of data. I have been to many places in the world that it is a reality that you do not have Unlimited internet and even unlimited electricity to run your devices.

I've been to, you know, rural towns in Africa where people hang their cell phones off the light pole in the middle of town to charge their phone all day. And it costs them a week's wages to do so. Like we take so many things for granted. The web should be. The largest human umbrella, the largest umbrella that mankind has ever made, that humankind has ever made. It should be that, and it should encompass, or be able to encompass, all 8 plus billion people on the planet.

Right now it's serving, like, a third of the world. Two thirds of the world is not privileged enough to experience the same web that you and I get to experience. They don't have an always on internet connection like the way that we're recording this right now, right? They have spotty internet, or no internet at all, and it's metered, it costs a lot. and we are not building a web where they can participate, and, and where, and even if they can't participate, they can't participate as equals.

They don't have the same footing. We should be, I think morally, if not for any other reason, building a web that they can participate. And I don't see any way I don't see any way that we extend the web to the last two thirds of the world's population. simply because some billionaire hangs a balloon up in the air to give them internet or whatever. Like, that's not the solution. That's how billionaires want to solve it.

The way I want to solve it is from the ground up, by rebuilding the web in a way that says, once you get the information that you need, the data that you need, or create it, and once you have the application, you don't need the internet to keep doing what you're doing. You're a local fish farmer, you're working, you can upload the data into the thing, you can drive to market, synchronize with the market, and sell your goods, and you don't need a cloud server to do any of that.

We should be building that world. So that's the third pillar of what I call Web 2.5 is building a web that actually works for all of humankind instead of only the privileged part of the world. I love those points. And I think this is where we can also now like close the loop nicely, since I think all of those points are really, really welcome and like really core of what the local-first movement is all about. And I think this is also what brought you under this local-first umbrella.

I love the last two points that you've laid out. There is not just about the second point where you made about, the bigger a centralized system gets, the more expensive it gets. but I think it's also, if you think about the developers who are the creators who are building. Smaller apps that have an ambition, maybe similar to how I'm building Overtone. I actually, I want to charge for the, the value that the app is creating for the quality of the app, for like the differentiation of the app.

I don't want to charge for, like someone's internet usage and the data is actually in my way there to make that happen. like data shouldn't be part of my equation for how I'm charging for something and to make matters even worse, maybe less so for a music app or, but for many other apps, for example, let's say someone wants to build another, like highly specialized and highly personalized health app.

This is where it's not just maybe at some point expensive to move the data around, but it's actually data can be seen as a liability, particularly in a more highly regulated environment. And I think this is where encryption, which we'll get into shortly as well, I think is a very nice antidote to that.

And then the last point you've made about, like that the web shouldn't just serve the highly privileged, fraction of the world population, I think that's also very core to local-first and maybe another meaning to local-first as well. Not just like that local-first, the app works in your local context, but I think it's also that an app should maybe be more. Locally created in the context that you're working. We've explored that in the conversation I had with Maggie Appleton a while ago.

And I think this is really what's driving her motivation around local-first. That de facto right now, most apps that are out there are being built by like a very small group of people living in Silicon Valley, living in New York, a few hub, like hubs within the world. but that's actually not where most of the people are that are using the apps.

If we address many of the underlying challenges that we want to solve with local-first, I think this can also empower and enable so many more local creators to build much more specialized and custom apps for specific local parts of, different parts in the world. So I just want to offer that as an extension to the last point you made. Yeah, 100%. I love that.

You know, a good way of saying that is that it's built by the privileged for the privileged, but why not let everybody build what, you know, I'm never going to be able to understand what that fish farmer needs, but if we can empower him to build the thing that he needs, he'll understand it better. And he would be in a much better position to build an experience that works for him.

We need to create the pieces of that puzzle for other people to assemble rather than assuming I'm not going to build all the world's apps. I know that I'm hopefully going to build a couple of Lego pieces that will be useful in many of those apps.

Awesome. So let's shift gears a little bit and get a little bit more technical since one ingredient we'll need to make all of this happen is not just moving data around, which we've explored in many other conversations on this podcast so far.

But one particular aspect of that is also that like, not just two random devices are exchanging data in a completely trustless way, but a device might also be owned by a person who has a particular identity or might even on that given device have maybe there might be multiple identities at play. And so how do you retain an identity?

How do you, kind of exchange identity information, that your, I love the way how you put it, like your device ring, how all of those devices can trust each other, how all of that is made work, how, and I think there's this typical tension in security where if you want to make it secure, that often comes at the cost of convenience and vice versa. And that's sort of. Sweet middle ground where it's both convenient and secure. I think that's also very tricky.

So this is an area that you've really deeply explored and you've built a whole bunch of different projects. I think one of them is being the Lofi.ID, project. You also have another project on GitHub called Local Vault. So maybe you want to give us an introduction to what you've been working on there, and then we can take it one step at a time. Yeah, so big picture overview.

is that if you are going to own data on your device, and that's going to be the source of authority for that data, which is the premise for local-first, then I believe that you both need to be able to secure that data, meaning securing it at rest, which involves encryption, and you're going to be able to need to Securely ensure that data goes only where you want it to go. Plain text data being sent around and backed up in other places, that's not going to cut it.

So we are going to need encryption to be part of this equation. And To boil a lot of real complicated technology down to hopefully something that just about anybody can understand, we have this who's protecting the master key problem.

No matter what encryption scheme you design, whether it's password based, where you derive an encryption key from something that I type in, or whether it's I've generated a key and I'm going to Randomly, and that is your encryption key, no matter which mechanism, or there's a bunch of other variations thereof, but no matter which mechanism you have, you have this kind of encryption upon encryption upon encryption, but at the very top of that chain, you have the

master key, the master password, whatever. And how do you protect that? That's the common problem. And in fact, going all the way back to the work that I did at Hero, the Web3 company that I worked at, they had a very similar problem.

They were trying to create smart contracts around the Bitcoin chain with the separate blockchain, and they needed the ability to, you know, to round trip transactions in a completely trustless way between one blockchain and the Bitcoin blockchain, between Stacks and Bitcoin. And they have this exact same problem, which is who's going to hold the master keys? And how are you going to do that in the open, but not let everybody have it, right?

So we always run into this problem of It all sounds well and great until we get to the final piece of the puzzle, which is how do you handle the master password? I've been banging my head on this for years, trying to figure out there's got to be some way. And the advent of biometric passkeys was when the lightbulb went off for me.

Because What Passkeys do and what the biometric subsystems on these devices do is they offer a, I think, compelling answer to that question, which is there is no perfect storage for the most secure root, master, passkey, whatever you call it. There's no perfect solution, but the best that we can get is dedicated hardware on the device that is not Just free and open to any app, right?

If we can have a dedicated hardware for this purpose, the secure enclave or whatever it's called, and then we can have operating systems that are designed to very strong, strictly control access to that, and it's designed in like a write only way, you cannot read from it. You can write to it once, and then you can never read from it. You can send data in and get results back out, but you can never ask that thing to give you its private key.

That really offers, I think, the most compelling answer we've had so far to where do we store the master key? But one of the big problems with that, it's not in and of itself a solution to this, to this question is because. I don't have access to that key. That means I can't use that key for encryption purposes myself. I can create a passkey with my thumbprint or my face or whatever, but I can't get access to that key material to use it for my own encryption purposes.

And I couldn't figure out how we were going Make passkeys work in a zero server assumption where there's no servers and in a way that I could piggyback upon it to create encryption. And then it occurred to me that we can actually bury the key material in the passkey by way of the user ID field, or there's actually another mechanism that is coming along that I think will be even better than the user ID. But the main point is, we can actually piggyback on top of these passkeys.

And in this secure part of the device, store something in a way that I think is maybe 99 percent guaranteeable, that's a lot better than almost 0 percent guaranteeable with anything that's in userland apps. Is it perfect? No. But it's absolutely a, you know, a paradigm step up in terms of security. And so the whole rest of everything that I'm building is based on that one assumption, which is where we're going to solve the master passkey problem is by basing this on passkeys.

That does have some questions that it raises. How are we going to onboard apps? on devices that do not already have biometric capabilities. They don't have these secure hardware. I think the good news is that more and more devices are getting that. And I think that will continue. I don't think we will see a, you know, a pullback in that. I think we will see more and more devices getting those capabilities.

So over time that becomes less and less of a problem, but we do need an exit ramp for not just telling somebody, sorry, you're out of luck. So I've been working on some ideas around creating kind of a secondary way of doing things that's not biometrics based, pattern drawing that's more complicated and can create more entropy, things like that.

So I think we do need some exit ramps there, but I'm going to make the assumption that for the primary class of devices that we want to target here, we have the biometrics. Once we have that, once we have that capability. We needed a way to be able to manage pass keys without relying upon servers.

So the first library that I wrote was WebAuthn Local Client, which was designed to wrap around the web API that exposes the passkey subsystem, but doesn't make any assumptions about communicating with the server. the use cases in local-first don't need what the server would do anyway, so that's not a problem, but it's just no other libraries out there that I had found where it could work if you didn't have a server.

So I wrote a library that exposed that API in a way that made the assumption you weren't going to communicate with the server. You were going to store things securely. You were going to generate the challenges on device. You were going to keep track of the key and verify that was coming out was not man in the middle. You were going to do all of that. on device, not on server. So that's what WebAuthn local client was. That was the first little piece of the puzzle.

Then I said, okay, well now we need a way to, piggyback on top of the passkey system and create something that we can use for encryption purposes so that we can protect data both at rest and in transit. we need some keys. And I did a lot of research in this. I'm not an expert. I'm not a mathematician.

But I settled on, this was back when I was working at Socket, I settled on, Elliptic Curve keys, specifically the Edwards key, the ED25519 key pair as the best master key pair because you can generate, that can be used for signing, and you can generate a sub key pair from that. That's the Curve 25519 keys for encryption and decryption. I settled on that as the best, but that's not the only way of doing it.

And certainly anybody else could substitute their own encryption, or maybe we're going to need to upgrade that encryption in a post quantum world in some few years. But, that's where I landed was, let's use that, because I think that's good enough and strong enough. And I found the Libsodium library, which is big and complex, but it's, I been ported to tons of platforms and that was important to me. And so I based the ideas around securing our data on that type of key pair.

So we generate a 25519 key pair. We take the seed of that, or in my library, I call it an IV, but IV or seed, we take the seed of that. And that's what we store in the passkey. And with that information coming back out, each time you put your thumbprint or your face ID in, you can reconstitute that key pair and then decrypt whatever data was previously encrypted. That is how we fix that master password problem.

So I built a library to help you do that and that's called local datalock is the library that wraps around the WebAuthn local client, but then it does the generated encryption key, stick the seed into the passkey, it does all that stuff. it does not do anything about storage of it. Right? So you might use local data lock when all you care about was transmitting secure data, but it does provide the pieces for the next one.

The next one was we need to figure out how we're going to store that data in an encrypted fashion on the device. And then, so it's encrypted on write and decrypted on read. How are we going to do that? Well, I actually realized I need two pieces, not only one to manage the encryption piece, but actually I needed a library to manage the storage piece, because there's a lot of different variations of storage, as we were just talking about with OPFS and things like that.

So I built the Two libraries to help with this piece. One library is, I've spun out actually, it's not even local-first really, it's its own thing, which is just abstracting local client storage in a key value way across all the five major storage mechanisms. So cookies, local storage, session storage, indexDB, and OPFS. And technically under OPFS, there's two different ways of managing it. One is more device limited, but it's the in thread communication that you can do, asynchronously.

But the more broader device support, basically all devices at this point, or all modern devices at this point, is OPFS in a worker. and if you do OPFs in a worker, which you're nodding because you've seen the same problem, but managing workers and all of that stuff is difficult. So that's what this library does, is it abstracts across all of those mechanisms the exact same asynchronous key value API. And that library is called Storage. It's part of my BYOJS Bring Your Own JavaScript initiative.

so we'll have a link to that. But that storage library, even if you didn't do anything local-first, you just wanted to store data, I think that would be useful to help. Think of it as kind of a more modern Local Forage, maybe. Local forage has been around for a decade, but it's not really maintained anymore, and it didn't support all the mechanisms.

I think storage is a compelling option to look at if you're sticking anything on a local device, even in local storage, maybe have a better API and a more secure API for that. So first of all, storage there, that has nothing to do with encryption. It's just The raw reading and writing from a disk. And then the last piece of the puzzle, the one you mentioned before, is the

local data vault . So that library is what takes storage, WebAuthn, local data lock, and all those pieces together, and gives you that local vault, which is a secure, encryption secured, based on passkeys, encrypt on write, Decrypt on read key value storage mechanism. And you can, of course, choose which of those places you want to store, like store it in IndexedDB or store it in OPFS. But it does all the encryption and decryption stuff for you based on the passkey subsystem.

So that's what I've built so far. Where all of that is going is towards, there's two ways that I see this happening. First of all, I think apps can start to use, web apps can start to use those libraries, whatever pieces of that makes sense to you, use all of it, use part of it, but start to use those pieces to do some of this stuff yourself in your own apps, build your own solution if you want. I also think that we need to lower the barrier for apps to do a lot of this stuff.

And I also think that the more consolidated of approach we get, the better chance we have of something like this catching on. And so instead of just writing like a standards doc and saying, if you do it, you have, you must do it this way. the approach I'm going to take is to build a companion Wallet app that allows you to manage as a user, any number of your own identities, Protected, of course, through this passkey subsystem.

And apps will have an SDK that they can interact with the Wallet app, whether that Wallet app is a browser extension, like we mentioned a little while ago, or whether it's a literal actual side, you know, companion app. They'll be able to interact with that app to ask that app, Hey, tell me who this user is. Tell me that it's the same user as they were before. That's one question.

So instead of them needing to build their own authentication, they'll simply make a call to this Wallet and say, verify for me that this user is who they say they are and give me back their identifier, which might be their key or some other UUID that you specify. That's one question. But also I think those apps shouldn't need to roll their own encrypt all of my data and decrypt it all and all of that stuff.

So they'll also be able to provide the data to the Wallet app and ask it to encrypt it for them through that SDK and decrypt it for them. And again, based all of that around passkey. So they don't need to invent any of that stuff. The Wallet app will just take care of it for them. And then the last piece of the puzzle is.

if you build an app, let's say like it's a, you know, it's a note taking app, or it's a social media app, or whatever, and it does need to communicate with other devices, instead of you doing your own synchronization logic, I think we can actually have the Wallet app built with peer to peer capabilities so that my Wallet app on my phone and my Wallet app on my laptop and on my tablet can synchronize my identities, but can also synchronize userland app data through the Wallet.

Use the Wallet as a tunnel to do synchronization. when I say synchronization, I do not mean that I'm solving anything that what, what the larger local-first community says when they say synchronization with all the CRDTs and all the merging and stuff that I want to leave open to this community to solve. You can plug in whatever. CRDT systems you think make sense and whatever strategies you make sense.

I'm simply going to provide the transport layer through peer to peer various peer to peer technologies in this Wallet app so that those bits can get from my phone to my watch, to my laptop, to my tablet. And then you'll, your app will decide what do we do with that? In your own app logic, you'll decide how do we merge these competing sets of bits that are coming in. I just don't want for people to have to go invent all of their own wheels there.

And I think a single unified Wallet app will allow people to do that. there's other things that I want that Wallet app to do, but that's kind of the main, starting point, the 1. 0 that I want this Wallet app to do is to give that to the local-first community and say, please consider building upon that Lego. Perfect. So this was a lot and, kudos to you for doing such comprehensive research, deliberating the different options.

There's always like so many different paths that you could go in regards to like all of the different decryption encryption mechanisms, like choosing, should you use Libsodium? Should you use WebCrypto? Should you use other things, for what it's worth for Overtone? I've also landed on using Libsodium, which, I needed to even compile my own WASM version to trim it down a little bit more.

but, yeah, so you've covered a lot of ground there and I have a few follow up questions where I would love to learn more. also one observation that I just, as someone who appreciates like good terminologies and clear concepts, I just love the term Vault. As something that, both signals like, Hey, this is something where it can put stuff in and get stuff out and it is secure.

So a vault being sort of that combination of your storage mechanism and that search mechanism has nothing to do with encryption at that point, but if you then combine it with encryption and decryption, that makes a vault. I think that's a very intuitive concept that is, that works both as a concept for developers, as well as even for end users. like I think this is what also 1Password, for example, has started to use.

And I think, 1Password users are familiar with that also, I'm sure for other password managers as well. And so that as a concept, I hope that it's not just a. a concept that is used within the local vault and like the stack that you're exploring here but hopefully that's something that as a term, can be useful for others as well. Yeah, I agree. I think it does communicate what it needs to. I workshopped that with some of my social media community, by the way.

I had lots of different suggestions and I, you know, kind of pieced together various things, but I workshopped some of the naming of this, you know, crowdsourced it because I wanted to make sure I got a name that really communicated properly, what I was up to.

So to me, it sounds like you've landed on a great option here and I might actually steal that for myself and, the ways how I'm like in my own data stack, where I have a database right now, it's not encrypted address, but I want to encrypt it more. And so maybe I can also use for the SQLite database there. Maybe I can also start calling that a vault if it's encrypted. I like that a lot. and I also like how you've already thought a step further.

it's not just as that, like as a tech stack that can be implemented by a given apps or high level libraries, but also from a user perspective, if you're not just going to use one app, the kind of part of the entire promise here is that, like here you have data in one app and data in another app. That, and maybe those apps want to work together and that is actually something that is like really, really difficult in today's web2 world. Where like, just think about like how much of an effort it is.

That's maybe the best we got. Maybe it's like a Slack, integration that like Slack is like a little bit more aware of like what that Figma link means. And, I can open it all in my browser. So from my perspective, it kind of feels like, Oh my gosh, why don't you have that context? So if we, embrace a little bit more of the user. The identity concepts, and then also let, that dictate a little bit of the share, the context sharing and access control.

I think that can lead to much better user experiences. there've been many, many years of attempts to create in the mobile space. there was like web intents and then web share and share intents and like all these other variations and it went through various different names and standards processes stumbled with it.

I don't even know where that currently stands, but that's exactly where I'm headed is basically, let's just get around any of those limitations and allow, for example, a use case where, you know, I might be in a local-first note taking app, and I might have written a note and I want to, you know, copy a piece of that note, and I want to send that out to one of my text messaging recipients or one of my chat recipients or whatever, so I can take that note and I can synchronize, I can

share that information to this other app in a fully secure, end to end secured way. But it ends up in my other app, and that other app pops up, and there it is, in the exact same way that, you know, you can currently do sharing intents. Native apps have that, but the web has always been, you know, a third class citizen at best in that sort of cross app collaboration and in sharing story. And I think we can make it a fully first class story this way.

And I think that also takes us in possibly even a step further that goes beyond today's conversation. so far we talked about identity and also a part of that identity is like authenticating as assuming that identity or being denied that identity.

that's often abbreviated as Auth, but Auth can also mean another non abbreviated word, which is authorization, which I think this is not yet covering that, but there, I want to plug another very interesting project that is more around authorization, which is called Beehive. that's also been published on the Ink and Switch website. and there's currently, I think, not yet a full. Inc & Switch style essay about that, but there are some notes being taken on this.

And so this is a project that, Brooklyn Zelenka and Alex Good is currently working on that is, also ongoing research in combination with AutoMerge.

And Brooklyn has been exploring a lot of that, as her previous work On vision and related projects, so, and this is where, the authorization concepts to my understanding is sort of based around the ideas of capabilities and, that different users can basically, share capabilities, and privileges basically up to a level that they have themselves, whether it's read or write and so on.

So I'm sure this is like an equally deep and challenging topic and, but they feel a little bit, complementary to me. So hopefully there's like some convergence here as all of those are like very deep topics. I did see that Beehive announcement just recently. I think that's great. I think we need a lot of different flowers blooming in this field to figure out and find the places where there's going to be overlap and collaboration.

By the way, just as a little bit of a side note, I literally just yesterday. learned something that maybe everybody else listening has already known and I didn't know, but just for the benefit of the audience, the word auth is you, A U T H as you correctly point out, is a little too ambiguous. It's a little too shortened because we don't know, do you mean authentication or authorization but I saw somebody do auth Z for authorization versus auth n for authentication.

So if we just add on that one extra letter to that shortened word auth, n or auth z then we know maybe better what we're talking about. So I learned that yesterday and I'm going to use that going forward. Amazing. I was not aware of that, what that little letter N in that context meant. there's also what you've mentioned, web auth N. Maybe that is what it's already using. Perfect. Well, today I learned, thank you for sharing that little learning.

and kudos to everyone who was already aware of that.

So I have not personally used passkeys yet as part of the applications I'm working on, but I am using applications that use passkeys. I'm like more and more like. One web service after another is like rolling out support for it, and I currently manage pass keys as part of 1Password. I'm giving 1Password the benefit of a doubt that they do manage all of that securely for me. maybe there's like at some point 1Password armageddon but hopefully not.

knock on wood but, Using web keys for your own web applications or applications more generally, can you help me through, understanding that what that entails? Like, what are the boundaries of that? For example, if my, either one password manages my pass keys, or if my operating managers pass keys, what does that mean in terms of re authenticating on another device. So let's say I'm like, you've built this amazing note taking app. I'm starting to use it on my desktop computer.

I have like gone through like some sort of initial setup process. I needed to like scan my finger. So the passkey was created. And now let's say on my tablet, I want to also work on the same notes. How does the setup process there look like? Do I use, for example, as I'm in the Apple ecosystem, do I trust Apple to synchronize pass keys? Is there like a more manual pairing step that I need to do? Similar to how, like in, WhatsApp, for example, there's the QR code scanning.

What are the, are there different ways, how to do that? Is that just taken care of? Can you help me understand that? Yeah, absolutely. So, we need to be very clear that there's the standard way that almost all web apps, basically, practically all of them, are currently doing it, which involves a server, and then we need to distinguish that from what I'm proposing as is the future for local-first, which is servers become optional, and in many cases, don't exsist at all, right?

That's the zero server world that I was pitching. So the way that things currently work and the way almost all web apps, you know, my bank uses biometrics, so let's just use them as an example. Right. So what my bank obviously has is a source of authority record about who I am, and they want to make sure that I'm the only one, no matter what device I come in from, I'm the only one who can access their source of authority for my banking info.

the way that they do that is they have multiple authentication mechanisms attached to my account in their server. They have a username and password, of course, because you're going to have to access this from devices where that's the only option. But they also have Other records in, associated with my account that are the pass keys that I have registered when I was already authenticated with the bank account through some other means, including potentially another pass key.

Anytime they have an inbound, here's a new pass key. They just associate that with my account as well. So I have like two or three different devices where I have pass key authenticated with my bank and their, I don't know their database structure, but a, Ostensibly, they have like a one to many relationship from my user account to all the various different ways that they know that it's me, right? So any inbound passkey that looks like this, it belongs to Kyle, show him his bank account.

So the way that process works is that on the server, in a place that they fully control, as opposed to in the web or a client that they may not control, on the server, they generate a random challenge, which is just a random set of numbers. They sent that up to the device where the passkey is going to be presented. And that challenge is sent into. The device, and it is encryptically, it is digitally signed, doing a digital signature, which is different from encryption.

They create a digital signature using the internal private key of your passkey that nobody has access to. It's stuck in the secure hardware. Nobody can get at it, right? They send that signature back out, and they send that signature along with the public key that you've already got. That's what you've stored, is you've stored the public key already.

But they send that signature back to the server, and the server checks to make sure that it was able to create a signature that matches the public key that they already know is your passkey, right? So that's how they know that it was the same you on whichever device is if you were able to successfully sign the challenge and send it back to the server. What many of these services do is have that one to many relationship where you could have as many of these biometric keys as you want.

Some of them are a little bit more naive or ignorant where they just literally have one, but the, Important thing from their perspective is that they have an identifier that they know is me, that my bank has an identifier that they know is me. I don't know what that is, but they have an identifier they know is me.

And they either are going to stick that in the passkey, So that when that passkey comes back in, that they know it was me, or they're going to manually store that public key or both, but they're going to be able to match those things up and say, this challenge came from a device that Kyle owns and has previously told us is him. So we're going to let him have access to his bank. That's basically how passkeys work. Currently work.

I've glossed over some details, but it's basically how passkeys currently work. And so you notice that the server is playing an important role here because the server ostensibly can't be compromised to generate a fake or replayed challenge. The server keeps track of every challenge that it's ever sent out, and it never sends out the same challenge twice, and all of these others, like, secure things.

And also, the server, because it's going to be what's called FIDO2 compliant, this really complex specification, that basically does a, traversal of the public key certificates for the relying party to verify that it is in fact, I mean, for the authenticator to verify that it is in fact a valid authenticator and not some made up, you know, faked one or whatever. So FIDO2 servers do all of that complex stuff.

some apps, I would imagine probably banks They really, really care about this stuff and they probably implement all of that complexity in the backend. I would wager to say that most apps do not verify the authenticity. They simply rely on, if you sent me a matching signature, I'm good with it. Even if it was a man in the middle, I don't care, whatever.

A bank definitely cares if I've had a device that's been compromised and there's a man in the middle, that's, you know, taken over my authenticator. So they are probably going to keep running up the flagpole of the public key certificates, you know, all the way up to the root certificate or something. They're probably going to do that, but most apps are not.

But that's why you have a FIDO2 servers, because you do not want to do that complexity yourself, and you definitely don't want to try to do that in the browser. Now, we move to this other way of thinking, which is that there is no central, decision of who I am. There's no like central source of authority for who I am. Who I am is just simply that I have chosen to collect a bunch of me's together into one identity. One me on my phone, one me on my laptop, one me on my tablet.

That's me in an identity sense, right? It's not me in a humanness sense, but it is me in a digital identity sense. And, that's what the bank has done. You know, they've verified my humanness by make, you know, I couldn't open an account without a driver's license and other things.

We're not dealing with that part of authentic, you know, of identity right now, we're just simply talking about digital identity is a collection of essentially these key pairs that I have said, these three key pairs constitute me across my different devices. When we start talking about how do I synchronize my passkeys, in that scenario I was describing before, you created a different passkey on each device, and that bank kept track of each of those different passkeys.

But you notice that I said some, don't let you do multiple pass keys. They only let you do one. And that is why Google and Apple allow you now to both of them now support this Google just recently, but they now allow you to synchronize your pass keys between your devices, using your Google or Apple accounts. So I could literally have the same, you know, passkey on my phone and on my laptop.

But something really important to note is, it's not actually the same passkey, because the device is still generating the secure key pair. What they synchronized was not the actual secure enclave key pair, because that's not even possible. The hardware is holding on to the key pair. What they're synchronizing is just the metadata in that user ID field and the user handle and all that other stuff. That's what's being synchronized so that that matches up.

So my same ID shows up in Two different passkeys, but it looks to the bank as if it's one passkey. That's basically what that synchronization is happening. So. Rightly or wrongly, but I think rightly, people have complained that passkeys are definitely more user complex. People have to think about like, you know, all of this synchronization stuff, which is why nobody does that anymore with their usernames, passwords. Almost everybody's using a password manager.

And it's why I'm arguing that since the world is using password managers, we can just have essentially a password manager that I'm going to call a Wallet that does the same stuff for you. You don't want to worry about, Synchronizing between all your different devices. That's crazy, right? So back to the local-first world. Since I'm simply saying that I'm going to create these identities on my different devices, and I'm going to kind of group them together, I'm going to decide when they sync.

the Wallet can actually synchronize those key pairs between devices. So you could literally say, I want the exact same key pair on all of my devices. And then the way that you're authorizing, the way that a device is knowing that it's okay, we basically just flatten it out to, if you're able to decrypt the data, then you have the identity. And if you're not able to decrypt the data, then you don't have the identity. We've just basically simplified the whole thing.

Now, I recognize that there are trade offs here. We don't have quite as strong of an assertion as we do in the bank account centralized server world, but I'm going to actually argue, not even on this podcast, but just as my continuing work, I'm going to argue that it's better for us to move in this direction. It's better for us to not have banks getting to have the say so about who we are. It's better for us to get to say that. So I'm going to argue that those trade offs.

are actually an improvement, not a downside, but there are trade offs where we don't have quite the same level of central authority guaranteeing as somebody's humanness. So I appreciate you giving me a very deep introduction and beyond an introduction, quite the lecture in a positive sense on refreshing my own knowledge and understanding on like some cryptography aspects and how those things work together.

And I think there is an interesting evolution that will probably, is already happening and will just continue to happen, where security is really, really difficult. Like, like you said, like a bank needs to go all the way, and there's like many pieces of our technology there that we're using, like a browser itself probably being one of them, where it needs to go really great extents to make everything as secure as possible.

And while still trying to make us things as simple and easy for both users and developers as possible. And there's like this really interesting middle ground and where there's still this tension of like, okay, sure you can make it a tad easier for users. And that might come at the expense of security. But that boundary is also like shifting slightly. And I would say passkeys have actually been, both a step forward in terms of security for users.

And also, things are moving in a direction where they also actually become easier for users. I think, yeah, I think we've got some growing pains where right now, at the immediate as we're having this conversation, it does feel harder to most people. But I think we, We have made a paradigm shift where we will absolutely achieve what you're talking about.

Because If you just take a step back and you think about all the complexity that is currently around the username and password standard for authentication, which requires people to like pick new passwords for all their device, all their accounts, but nobody, most people don't. And then all the different requirements, like this site requires capital letters and this one requires numbers. And most people don't even manage that. Now they just let a password manager do it.

And then all the complexity around what happens when my password gets known. And I did share it. And now that can be used, like all of that stuff completely goes away. And in replacement is my thumbprint or my face. Now I understand that getting to that point is definitely a bit of an uphill climb that we're getting closer to the top of.

But once we are there, I cannot imagine that any user, even a completely non technical user is going to be like, wait a minute, I don't understand this thumbprint thing, but I was totally cool with all of these emails and password requirements and capital letters and all that stuff. Like nobody is going to say that eventually, eventually they are going to say pass keys are so much faster.

So much more streamlined, so much more user experience optimized, but we are still in the process of getting there. And I know why people bristle at that claim right now today, there's rough edges now, it's getting better. And I think there is a second Category of those sort of uncanny valley symptoms, similar to how, like when web 2. 0 just started to become a thing and people started to figure out how do I do authentication, in the like web 2. 0 world.

where, like, I remember the first PHP thing that I've built where I needed to authenticate users. I used good old, like, MD5 as, like, the password hash. Well, that very first system, I did not hash at all, and then I used a much more secure thing, MD5, and then I learned about Rainbow Tables, etc. And eventually, there's, like, things like Auth0, etc., and, like, single sign on, which took away quite a lot of that responsibility and that pain.

But now as we're shifting to local-first, we're reopening then all of those cans of worms again with new security technologies, such as. Pass keys, et cetera. Things have moved on. now there are quantum computers possibly. So we need to rethink how we even encrypt things. so it's a moving target.

And I think right now there's probably also the complexity an average app developer needs to think about in regards to, making their app secure and private for users is much higher than what I would say in five years from now, the local-first data frameworks that will be available in five years from now, most of them are already having this on the roadmap, but in five years, it's going to be, or even like in one year, three years in the future is going to

be, Much less complexity that a app developer needs to think about today, we all appreciate and need to hear this sort of lecture on like how all of that works, because the technologies are not quite there yet, that I could just say. I pick my favorite local-first data stack, and they all implement the best practices. And all I need to look for is sort of like, Oh, the work with past keys is done.

And I don't need to, as an app developer and as an app user, I don't need to do anything more right now. As an app developer, you need to know quite a bit about the moving pieces. And then each month and each year by year, We can forget about the underlying implementation details as all of that matures. So I just wanted to highlight that, and that is a bit of an uncanny valley, but things are moving in the right direction.

Every one of them currently is inventing their own solution for identity in some ad hoc and informally specified way, right? They are all doing that right now. Of course as, as a human with an ego. I hope that they just see how amazing this Wallet is that I build. And they're like, let's stop doing that. Let's just use Wallet, right? Like I hope that's the case in reality. I think they'll pick something much lower level. And what I strongly assert they should do is base it on passkeys.

And I hope that even if nothing else that I've done makes sense for your stack, the WebAuthn local client making it easy for you to deal with the passkey system without worrying about the servers and FIDO and all of that is, I think the way that local-first should go at a minimum. I hope that the other stuff I'm building is useful, but at a minimum, I really think building that on top of a passkey system, and I think a library like mine will make that much more approachable.

I wanted to use passkeys, and I didn't want to build a library for it. But when I surveyed the landscape of those libraries, they were all server centric. And that's why I built one that can at least work for the cases where a server might be optional. So for whoever has made it through all the way to now and has like absorbed all of those details and hasn't had enough yet. I highly recommend you just check out those amazing projects that Kyle has been working on.

Both try to build a little app with it. And if you're extra curious, also like look at the implementations. Actually not that much code in most places. And you get to see how the web APIs under the hood work in case you need to use something that's a little bit more low level than you've already seen a mechanism, a implementation, how this works. That's typically how I go about those sort of things. I try to see, can I use an existing thing?

And if not, then I try to learn from the existing thing to build my own thing. And then this is sort of this dance back and forth. Later on, the existing thing is good enough and I throw away my own thing again. So try to learn from Kyle's implementations and try to use them in your own apps.

Before wrapping up this already pretty long conversation, there was another project that I think you worked on before, and I'm not sure how mature it is or whether it was more of an experiment, but it did catch my attention because I thought it was, very interesting and had a bunch of like putting things together in a novel way to me. and so the project I'm referencing here is like the QR data sync project. Would you mind giving a overview what that was about and also how it came to be?

So, first of all, status. QR Data Sync is absolutely a first class citizen in this same ecosystem and is absolutely going to be part of the Wallet app that I'm building because we need multiple transport mechanisms for data that go between devices. Obviously, a preference would be, make it just happen super seamlessly under the covers with something like the local peer to peer or another peer to peer protocol or Bluetooth or anything.

It's like Any of those ways would be better, but there is always going to be a need for fallbacks. And one of the fallbacks is the QRDataSync. There's another one, which is not a library that I wrote, but I absolutely intend to use. It's called SilentJS. SilentJS actually transmits data between devices using supersonic sound waves. So it uses your microphone and your speaker between two devices and it transmits data that way. Fascinating stuff.

I didn't write that library, but I think it's super awesome and that's going to be in the Wallet as an option if you are trying to synchronize. Very important to point out. Some of these fallbacks are going to be bandwidth limited and size limited. You're not going to synchronize a gigabyte of data through sound waves or through QR codes.

Okay, so the bare minimum that you might do would be to synchronize your identity between those devices, which is very small, you know, A couple of hundred bits of data. And then once you have that, you now have established the ability to create a more secure tunnel through other mechanisms through the internet or something like that. So that's where I see things like QR data sync and silent JS being is kind of that lowest level.

Let's synchronize my identities across devices, but then all that heavier data synchronization stuff that's going to happen, it should go through a more high throughput channel, like. You know, something that's or UDP based. Just briefly to touch on QR data sync, the way that library works is it creates what are called animated QR codes, not my original invention, but I saw it years and years ago, and then I didn't really see it ever go anywhere. And it just stuck in the back of my head.

An animated QR code is nothing more complicated than a whole bunch of QR codes shown in rapid succession, like an animated frame. Each QR code can have a chunk of data in it. And so you take a big chunk of data, say like, you know, 10k of data. It's a string. You just break that up into a series of chunks, and then you do some padding so that you make sure that the QR codes are all uniformly sized. And then you just show those chunks in succession.

My protocol in this library just has a very tiny little header on it that has the total number of frames that are going to be shown and what the frame index is that's being shown. So then you have a camera on a different device that is reading QR codes. And by the way, it's reading them very, very quickly. It reads them in like less than 15 milliseconds or something, but you just hold up your camera to a device that's displaying a set of animated codes and the camera is reading those.

And it's just doing an infinite cycle, by the way, the sending is just sending them all in order because your camera might miss a few of them. And so it just holds in its memory, all the ones that it knows that it needs until it just keeps. keeps going until it has read all of the frames, and then it reconstitutes the data on the other side. So that's what QR Data Sync is. It is hopefully a fallback mechanism at best, but it's certainly going to be an important key piece of what Wallet is doing.

I love this so much. This stimulates so many parts of like, what I like as a geek. particularly when some, Digitally tricky concepts become visually a little bit more tangible. And given that here with a QR data sync, the visible and visual component, but given that you've also mentioned the Salient project, which move that to another dimension, to the dimension of like hearing and sound.

this sparks all sorts of like ideas in my head in the same way, where you can, somewhat style a QR code to look a little bit more like, brand related. I'm wondering, could I actually make the audio pairing, whether I can, manipulate that in a way to be related to what you're doing. In my head, there's sort of like those, those like old school modem sounds. So it's like back to the future in a way. so I'm, I'm very glad I asked about this project.

and it's not just a very, curious and cool aspect of it, but I think it's also very practical and that's something that, people in the Web 2. 0 world are already quite familiar with. What I've mentioned before, whether you're pairing a, WhatsApp app from one device to another that is not animated. This is where a single static frame already is enough.

But I think the patterns are already there, end users are familiar with them, and putting this all together in a pluggable way that allows for different options, all under this umbrella of your packages that you're working on. Super fascinating stuff. So Kyle, I think we're already running well on time here.

but I really, really appreciate you coming on, sharing your background, sharing what led you to local-first and then sharing everything about your explorations and deep work and local-first identity, AuthN, local vault, all of those projects. Thank you so much for sharing all of that. Can I give one last little tidbit?

The world, if we went back 10 years, and if you tried to convince people 10 years ago, that the whole world needed to move away from usernames and passwords as the single factor for authentication and that the whole world was going to move to owning multiple devices and we were going to be able to like generate these numeric codes that were randomly changing in time and things like that.

And if you told people 10 years ago that billions of people in the world would do that, most people would say that'll never happen. But if you fast forward 5 or 10 years, the vast majority of the world has upgraded to multi factor for authentication. We've still got a long way to go. I'm not saying that it's a fixed problem, but we have absolutely upgraded the world to understanding the concepts of multi factor.

So what I'm suggesting in the move forward with going to a Wallet and going to this local-first identity is akin to that. Will it take a while and will it take users changing their mindset and will it take big players pushing it and requiring the adoption? Yeah, it absolutely will. It's a, it is a hill to climb, but I just wanted to point out that, concept of multi factor auth is not going to go away. It's just going to evolve in this app. So for example.

This app is going to have a TOTP number generator, where you can use it as a second factor of auth, but instead of you having to look at your phone and then type in the numbers, if you have the Wallet app on your phone and on your laptop, and your laptop is challenging you for a second factor of auth, All you need to do is open up the Wallet app on your phone and say, give me a code and instantaneously that code can synchronize to the Wallet on your desktop and then paste from the

Wallet on your desktop into the form. You don't need to sit here and type all of these numbers so we can have an upgraded view of what second and multi factor auth is if we go in this direction. And that's one of the things I'm most excited about because we should not be compromising on security. We should be upgrading security, but also, as you've said several times, putting the user experience first and foremost here.

I think arguably a lot of the multi factor auth was pretty rough for users at first. SMS codes and all these other things, you know, waiting 10 minutes for a text message to come in. We still have a lot of that. There's still a long way to go.

But because we've made so much progress there, I am very bullish on the fact that we can make the same kind of progress towards this web 2.5 where we own our data and we own our identities as we take them through different apps and through different devices. That's a wonderful summary. I'm very excited about that future. Kyle, thank you so much for sharing all of that with us today. Thank you for having me. It's been a thrill to dig into it. Thank you for listening to the Local First FM podcast.

If you've enjoyed this episode and haven't done so already, please subscribe and leave a review. Please also share this episode with your friends and colleagues. Spreading the word about this podcast is a great way to support it and help me keep it going. A special thanks again to Rosicorp and PowerSync for supporting this podcast. I'll see you next time