We're All Building a Single Digital Assistant

I think the main thing that we're missing right now is that our AI system doesn't have a single entity, it doesn't have a single identity, it doesn't have a single personality. And I think that is the interface that we will move to. I think a bunch of people have sort of figured this out. I mean, I believe OpenAI is heading in this direction with some sort of device, right? They hired Jony Ive to work on some sort of wearable. And the idea for them is they want to bypass

the mobile infrastructure, right? They don't want to deal with Apple and the iPhone anymore. They want to have their own OS, essentially like an AI OS that basically everything goes through and then all their infrastructure and stuff on the back end. But I think when we talk about harnesses, context engineering, prompt engineering agents, especially like we get stuck in the weeds, right? We're talking about, okay, you know, what's the best agent framework? What are the best agents,

what are the best skills? And I think the best way to think about this is to imagine all that stuff abstracted away. And the way I like to think about this is to and I learned this when I was at Apple and we got this DNA, I believe we stole it. A lot of it from Amazon, actually. It's the concept of reversing backwards to go to like into the future where you believe you see some sort of outcome that you want, a product that you want or a future that you believe is going to happen.

And you basically articulate that and you say, okay, this is the thing that I think people want. This is the thing that I think people will, you know, resonate with and ultimately really enjoy. And then you say, okay, what is a, they call it a PR, a public release. What does a release look like? Right. And then they work backwards. So I've been thinking this way since 2014

or something. So in 2016, I wrote this shitty book, which you don't have to read because I turned it into a blog post, but I basically said that everything is heading in this direction of you're going to have a single day, which is an AI, a single AI digital assistant, which is going to be your conduit. It's going to be your buddy, your friend, and most importantly, your digital assistant knows everything about you. Okay. That is like the primary concept from this book in 2016. This

thing knows everything about you. It knows your work, your life, your relationships. It knows what you struggle with. It knows what your strengths are. It knows what your weaknesses are. It knows what you're trying to accomplish, right? So if you look at all my various projects, well, first being that that thing in 2016, but also you look at substrate, most importantly, you look at telos, if you're familiar with

that at all. So telos is essentially a system for defining yourself, just defining what your goals are, what your problems are that you're working on, and essentially having all that in one place. Like what are my challenges? What are my projects? You know, what are my budget? You know, if this is a company versus a person, what is the team that I'm dealing with? What are the different dynamics there? What is the active work that's going on? So the idea is to have all this clearly articulated

inside of a system, right? And then with that context, your digital assistant can then basically monitor this. The other crazy concept here that's very much related that I've talked about a bunch in the last year, is this concept

of ideal state and the concept of current state. So the cool thing about all this AI stuff and all these agents is that they can constantly gather, they can constantly go and collect context, you know, research, knowledge, facts, activities happening in the world, news signal from different sources. It can always be gathering, but here is the central concept. Okay, your Da, I'm just going to use mine. Mine's name is Chi. Chi for me is constantly going and collecting things.

He is constantly collecting and organizing knowledge for me inside of my system, which my harness is called Pi. It's actually a public repo. It's an open source project releasing 5.0 about to come out, so you should get that very soon. Might actually be out by the time you watch this, but this infrastructure is not designed to be agents. It's not designed to be AI tools. It's not designed to be workflows. It's not designed to be any of that.

It's designed to be the back end infrastructure for context to collection and management. For my specific individual unitary digital assistant whose name is Chi, I intend to interact with Chi. Chi then understands everything about me, all my preferences, everything I'm trying to do in my world, in my life, in my work, with my friends, with my relationships. And this is the backplane. This is the foundation of everything that is happening in my AI life. Okay, this is

the direction I think everyone is going to go. I don't know if it's going to happen, really, if it's going to kick off in 2026, because we still seem pretty obsessed with like agents, but it's starting to happen. I feel like it's starting to happen because there's more and more talk of harnesses. And I think the next thing we figure out after harness is, is that we actually just want a single person, a single entity to interact with. And because we are humans, we want that

to be someone with a personality and a memory. Now, Open Claw kind of helped push this along a decent amount because it was somebody who was proactive. This is a major, major feature that's required that previous agents didn't have proactivity. Right. The fact that you can give it some things that you care about to some degree, right? And it puts it in a text file or whatever. And it could just like check on them regularly. You could like check a scheduled tasks, right? This kind of

moved it forward a little bit. Yeah. So I put together this personal AI maturity model And this, ironically, was a couple of weeks right before Opencore came out, which I can't even remember. The first name went through like four names, but this was right before that happened, which I was very happy to see it come out after this. And I'm like, okay, cool. This is definitely catching on. So,

so check this out. The concept here is you move through these stages and I've got three different stages for this. I've got chat bots, which is like you're talking to ChatGPT. That's level one. Level two is agents. And then everyone knows what agents are. I mean, that's 2025 is when that kicked off like full steam. And who knows where we are right now. This is a little bit out of date. I would say we're still around ag2, although

it's not clean lines, right? So you have like in some sense we're already ag3 and moving into As1 again. Opencore really helped us try to move into As1, but the key point is that we're moving from agents to the next thing, which is assistance. And these are the different levels. So you have three different levels of the chat like phase that we went through. We got three different levels of agents that we went through. At first it's just basic CLI, basic web interfaces, transactional, and there's

kind of no memory. You just ask a question, it comes back, it goes and does stuff. It's cool. Then you've got basic voice interaction ag3 you've got extensive voice interaction, right? So the features ramp up and, you know, as they get ready to move into the next phase. And then what we're moving into, which is the whole point of this video is we're moving towards assistance, right? Personality can see and hear around you goal monitoring and personal persistent personality, right?

We don't really have that yet. We've got a bunch of startups who have made like, you know, virtual boyfriends and virtual girlfriends and, you know, virtual best friends. And that's a whole space that's like taking off massively. But guess what? Those tend to not have the functionality of the whole professional system that we work in, which is,

you know, the agent system up here, right? So my whole point to you is that this is the direction it's heading towards assistance, but it's combining all of this agent stuff into the assistant stuff. So I think that's the direction that we're going. And I think it's just going to be extremely powerful. Right. And I want to talk just about like a bunch of these different things

that are happening around this. So the main concept is that the human life, the life of the principal, which is like the, the center, the human that is running all this, I call it the principal kind of like executive protection or whatever. My concept here is that the principal is at the center of the, the whole system, right? None of this tech matters. None of this AI matters. None of these agents matter if they're not doing something for a human right. The whole purpose of all of

this is to have human things happen, right? And human things be optimized. And, you know, it's improving the life of the principal of the human at the center of all of this. Okay, so back in late 23, I think I finally posted this thing in early 24. I've been talking about this concept of augmented, right? So I had this augmented course, I believe it was one of

the first AI courses out there. And essentially what I wanted to talk about was how AI to me has never been about the tools and the websites and all the different, you know, chatbots or whatever. It's about this, it's about a human life. It's about what you could do in your life and how you can improve your life using all these tools as like the back end, right? So what I encourage everyone to do is basically come up with, what are your actual challenges? What are the components?

What are the different workflows and little pieces that you can turn into a granular problem that AI can help you solve, right? So over here, there's security things, there's personal things, there's knowledge things, education things, right? And then you basically turn that into workflows that you can then execute AI on, right? So I've got like ideas here. You have like an idea, a random idea for like an essay or something, right? And you can turn it

into an essay. You could then turn that into a LinkedIn post, right? And this is all using those individual components, but it all comes back to something that you care about as a human. Here's another example. Since my background is in security, like you can do analyze incident, and this just becomes a workflow that you can do. In this case, this is inside of a tool called fabric, which is now part of my Pi project. But you

can essentially go and analyze an incident. You just paste in content and boom, it puts out an incident with great formatting like this. And this is kind of well known now, but it wasn't really known back in 2023. This was one of my favorite workflows. This one's called Extract Wisdom. I still use it to this day. You can basically take any video and pull out like the most interesting content from it, and you could then do something with it. Again, you can go and write something,

you could create a video about it, do whatever. So see something you like, capture it for later in a format that you can share, right? And all of those can be little individual workflows. So all of this to say that this concept of the human at the center doing their life things to me has always been the

point of any technology and especially AI. So this is why I believe and why I have not stopped believing that the direction for all of this is this single interaction point with a personality that knows us better than anyone, probably better than our significant others, and can constantly help us advocate for that stuff. Okay, so let me show you what that looked like originally when I wrote this in 2016. Again, this is the book which you don't have to read because guess what? It's a blog post.

It's free. But if we click up here and we go to digital assistants, most visible and significant role for synthetic intelligence, I don't know why I called it synthetic. Basically, I made the argument that it's not artificial, okay? It's just different from us. So we should call it synthetic instead of artificial. Kind of stupid. A computer system that can monitor human contacts, intentions and commands, interpret them, take action as well or better than a human professional personal assistant.

So that was the idea for a day. And if we just look down here, not just that there will be intelligent, but that they'll know our preferences, they'll be able to adjust the world to our liking. So the other concept here that goes with this is kind of everything getting APIs. That was like the second idea from this set of concepts from this book. So it's like you talk to your AI, your Da who is named. So I talked to Kai. The world is full of APIs.

All the companies are APIs, all the different services and products and the restaurants and the companies. They're all different APIs. And when I want to do something like I want to, hey, I want to buy this product, you know, I want to schedule a trip or whatever. I tell Kai. Kai then goes and scours, looks at a bunch of lists of like the best restaurant or the best Thai food or the best like place to buy a shirt or the best place to do a vacation for the cheapest

amount of, you know, money or whatever. And he can go and research all of that and come back to me and present it to me somehow, which the third piece was, okay, custom interfaces. So we're not going to be using the interfaces provided by the service provider. Our Da is going to custom make us interfaces. So that was another of the big ideas here. And what I said here was this will change how we interact with everything.

So instead of physically manipulating technology, much of which has widely varied interfaces, you have to find it, learn the interface, whatever, start using it in some way. We switched to guess what voice gestures, text part of our natural human communication paradigm. So this is why I think this stuff is predictable. And I talked about this in the beginning of the book. You can't predict tech, you can't predict implementations, you can't

predict who's going to win. You can't predict what company is going to like beat out the other company, what order the tech is going to come out. But what I think you can predict, and this was kind of the basis of the whole thing and the basis of this whole video as well, is you can predict to some degree what humans want because it doesn't really change. What we want is we want to feel seen, we want to feel understood, we want to have a close relationship.

We like to have a trusted relationship. And one of the cool things is that billionaires or, you know, people who make a lot of money, one of the things they say helps them more than anything is having a really, really high quality assistant. So I think this has always been a thing that we would want when we could have it. So this is why I'm so locked onto this model of a single D a that you do all interaction through, right? So instead of interacting with technology directly,

we will interact with our Da. Our Da will work out the details with the necessary daemon API. We speak, things happen, we gestures, things happen, we text, things happen. No need to fine understand or master new tech. That's for the service and your Da. My key to work out amongst themselves, right? Digital assistants will become the preferred interface between humans and the world in a disruptive and foundational way, right? So that's, that's what I'm essentially talking

about still to this day. All right. I want to give some more examples of just like real world stuff, right? So if you're watching this, you're probably somebody who likes to optimize. You're probably already deep into the agent ecosystem

or thinking about going in that direction. And I was just sitting, I was having a friend with my buddy Will recently, and we were just talking, and I was trying to explain this whole concept to him of like, why I think this is the direction everything's going to go and why I think all this agent stuff and the harness stuff just fades into the background because it's just infrastructure that's going to be used by your Da. Okay.

So as we were talking, I'm like, okay, so what cool ideas have we had in the course of this conversation? And what is going to be done with those ideas? And the answer is, well, did we write them down? Do we have a recording device? Is that recording device somehow capturing things? Okay, that's fine. If it captured it now, what can it do with it? If I care about conversations where we have really cool interactions and he gives me ideas, I give him ideas. I want that to

be always captured. Okay. If I'm reading a book and I'm like, oh, this is really cool. I want my Da to be watching over my shoulder, looking at the book and when I say, oh, wow, this is really cool. My Da is looking at the page and can extract what I meant and make a note of it. If I say, oh man, that's really cool, well, that should

be extracted out to make a note of it. That should go into some sort of infrastructure which is waiting to be followed up on, or I'm going to write a piece about it, or I'm going to make a video about it, or I'm just going to think about it. Maybe I'm not going to do anything. I just want to be able to remember that we had the conversation and this useful thing came out of it, right? So there are just a million examples of this. Okay, let's just break down a person's life. Let's say they do

lots of research. Let's say they do security research. Again, because my background security. So it's like I'm thinking about vulnerability research. I'm thinking about threat intelligence. I'm thinking about what are the latest attackers doing all that kind of stuff, that stuff. Kai should be collecting. Kai actually is collecting all of that stuff, collecting, organizing, trying to see if it applies to any of the tech stacks that we work with. Does it apply to any of the customers

that I advise for? Right. Oh, there's a million different things that can be done at any given moment that if that was the only thing that I had time to do, I would do it. If I had the time to do these things, I would physically be doing them myself. I don't have time for them because there are thousands of them. Guess what can do it? A whole army of agents, right? But I can't talk to an army of agents. How am I going to talk to an army of agents? I just talk to Kai.

Kai has all the context. Kai knows what my ideal state is. Kai knows what I'm trying to do in the world, what all my goals are. And check this out. This is like one of the primary ideas here, okay? This is like the centerpiece of the entire tech stack in my opinion. This is where this is going. Your single Da will have basically one prime directive. Know what your current state is from reading all these APIs, from pulling all this context from it can see your heartbeat, right?

It can see your depression level. It could hear the tone of your voice. It could see if you've worked out recently, you could see if you're fighting with your significant other. You could see if you haven't talked to your friends, what is your current state? What is your desired state? What is your ideal state that is captured in your telos that is captured as part of. In my case, my Pi personal AI infrastructure. Right. That is very clear. Kai is watching this constantly when my Diet

Coke runs out. If the restaurant has a slash menu and a slash order API, I could request a Diet Coke. It beeps in the back and my friend the waiter comes over and brings it over. Or a robot rolls out and brings it over, or it drops from the ceiling. Who fucking knows how it's going to happen, right? Who knows who's going to get there? This type of stuff is not predictable. What is predictable is when my Diet Coke runs out. Humans would like their Diet Coke not

to be run out. They would like to have more Diet Coke. That's the way it works, right? So some of these things are so predictable. What's predictable? The conversation I'm having with Will, we want to record that and extract cool stuff out of it? That is extremely predictable. Having the best meal experience, having the best food at the best restaurant. Okay. The other example that I have, I don't have kids myself, but I was painting a picture for him of like, okay, my daughter, she is

in college in New York City. It's late there. And you know, she's walking home right now. So Kai is telling me in my ear while I'm talking to Will, that she's walking down the street. There's actually someone following her. How do I know that? How does Kai know that? Because her Da reports this kind of stuff when she goes for a walk by herself, or she's walking back to her apartment from her dorm or whatever? Again, I

don't have kids. I'm just making this up. But she's wearing some sort of necklace or EarPods or whatever it is that allows her Da to see in front of her and see behind her, maybe to the sides or whatever, and that kind of stuff. Her Da knows to report that to Kai right now. Kai is watching that all the time, right? Because her Da is sending it all the time to Kai, right? Because that's an agreement I

have with my daughter. She's okay with that, right? Doesn't normally interrupt me, but right now, in the middle of this conversation with Wil, it's like, hey, Bing, what's going on? Yeah. So Julie, she's currently being followed. I just called, you know, neighborhood watch, and someone just came out and she's being escorted now. It's all okay. Think about what is possible here. Okay. This is where it starts to get a little bit crazy,

but it's extremely not crazy. It's extremely tangible and possible. And this is like literally what I'm moving towards with Pi. I'm literally building this now instead of building like more and more optimizations because I'm super guilty of like deep diving on all the agents and how they talk to each other, blah, blah, blah. No, this is where things are going. This is where everyone will be in two years. So kind of the whole point of this video is

to like let you know this. So you could just be like, oh crap, and like jump way ahead in this timeline, right? But think about this. Okay, guess what else is is spinning up right now. It's nowhere near as advanced as like the agent stuff, but robotics. Okay, what if she had a dog with her little robotic dog? And by the way, I'm still afraid of those because of the Black Mirror episode, but she's got a little

robotic dog. It has all the cameras. What if the moment she left because the sun was down, her little drone. It's like this big just flies over her head. So Kai is watching a top down view of her going from, you know, the dorm or whatever class she was in at the college, walking to the apartment. It could just watch her the whole way, could see people coming and going. It's watching the police radio. Her Da is watching the police radio. Her Da is also watching the drone. This

is awareness, okay? There's nothing humans care more about than like protecting their loved ones. Like being attractive, being interesting, essentially survival and reproduction. If you reverse engineer survival and reproduction, you will very quickly get to a trusted Da that can do everything for you and understands everything about your life, and can monitor and listen to and see everything in

your life. Okay, so eventually, you know, way in the future, ten years in the future, 15 years in the future, when she's walking, she'll have like four robots for like optimists or whoever's making robots at the time walking with her. Right. I would have to make a lot of money to be able to afford four robots, presumably. But she could have an escort. Okay, if she's going through a bad

part of town or whatever. Plus the drones, right? I mean, this is the type of situation where these Das are going to be able to deploy extra sensors, extra Intel gathering, use extra tokens to do whatever matters to that principle. In this case, I'm the principal. I have things I care about, which includes my daughter. And I am, you know, implementing Kai on my behalf. And her Da is implementing the resources that they have to protect this resource, which

is my daughter. Right? So that extends to everything that extends to a company protecting its resources and its employees. And, you know, everything you're trying to do in life. Just think about this. 24 seven using a giant army of agents, which is all this harness stuff that we're doing, all the Intel gathering, all the different skills and capabilities that tech has, except for it's yours. It is your day watching that millions of things at once. Thousands of things

at once, right? It has to scale up, right? With time and, you know, having enough compute and all this stuff. This is ultimately what we're building. And I am sprinting to this. This is what Pi is all about, right? Is essentially building this harness with a named personal assistant that is monitoring all this stuff for you. And it starts with having extraordinarily good context about yourself and what you want. All right. So let me just show you a couple things from Pi. So this is my Pi system.

Let me just open a new session here.

I've been building all this stuff since 2023. All these skills are all the different capabilities that Kai has to move me from current state to ideal state, right? That is what we're building here. Okay? This is what an agent harness ultimately is becoming an advocate for you, to move you towards your ideal state. And for most people, they don't know what their ideal state is. It's a silly question. It's a silly concept. What are you even talking about? It comes down to who are you actually,

what are you actually about? What are you actually trying to do? Yeah. So we could do telos. So we have telos skill. We can actually interview you and figure out exactly what you are doing with your telos. So if I go k d and I go to skills or actually I go to user, this is like the, the structure here, right? So I've got a whole bunch of different stuff in here. I've got personal, I've got life, I've got finance, I've got business, I've got health stuff,

I've got everything in here. I've got my writing style, I've got Kai's writing style in here. I've got a whole bunch of stuff about Kai's identity, which ebbs and flows right as we have this relationship, he and I. Pronunciations for different things. I've got contacts in here for like my particular contacts that are important to me in my life. I've got contact details so I could say email them. I could say, hey, go do this research on this and send it over to Jason or Sasha

or whoever. Right? The point is, I don't need to describe. Oh, there's a person, they have this email they and re-explain myself every single time I can literally, because I have this system.

even that I'm still inside of a terminal, right? And if I'm inside of an app, that's no better. I should be talking to my agent who lives in my ear, who can see what I see, right? Right now, I'm sitting in front of four screens here. Kai should be able to see everything on my screen. Should be able to control everything on my screen. What we're moving to is, Jarvis. What we're moving to is Minority report gestures. Hey, do this, hey,

do that, whatever. And we're getting pretty close. I mean, I can already do that pretty well with Pi, just with voice activation of like going to do something like, hey, run the pi upgrade skill and see what's new out there for us. So this one will actually go into research, everything new that's happened in AI. And we'll look at our entire harness and see, hey, what's going on with the latest in AI, what new skills came out, what new blog posts from anthropic, what new videos came out

from my favorite YouTube? People who talk about AI. It goes and collects all that. It watches all the videos. It pulls all the transcripts together. And then guess what? It does? It looks at our entire harness. It looks at the entire Pi system and all the context that he knows about me. And it comes back and says, hey, I recommend we implement this. OpenAI just did this cool thing.

I recommend we do this. This one woman, she made a new skill that does research much better and it has better research agents, and it has a fact checker at the end, which would be a good upgrade to our fact checker, right? And I'm like, yeah, cool. Sounds good. Boom. It goes. And does it. Do you know how much research this thing is actually doing? I mean, this is thousands of tokens this thing is using. I mean, this

thing is just getting started. We're currently inside of the Pi upgrade skill running a workflow, which is upgrade, but there's actually multiple workflows in there. It's looking at so many different sources which are customized for me. The point is all I have to do. I didn't even have to say run the Pi upgrade skill. I could just say, what's the latest out there that we should be thinking about, and Kai would know to go and run the pie

upgrade skill. So look at this. YouTube channels. GitHub trending cloud code freshness check. So this is looking at the latest releases, the engineering blog, the red team blog, all these different sources brings all that stuff up and maybe I'll show you what that looks like when it comes back. All right, so let me show you what it looks like inside of pie as well. Okay, so this is another interface to pie that is now in version five.

So because I'm going so heavy into this D a centered single interface point with all this scaffolding and agents and everything hidden behind your Da, your named da with a personality, your single point of contact for for AI, because that is the case. I've basically brought pie into kind of like a web interface world, which makes it clear that this is not just an agent harness, but it is a life optimization system. This is a life OS, right?

Based on AI, which is context focused with a single Da, with personality basically in charge of all of it, right? So I've got life stuff here. Work. Telos. Right? Look at this. I've got my different Telos stuff and I've got the observer thing turned off, so anything sensitive gets hidden out. But yeah, I've got my health stuff here. I've got finances, I've got business stuff. Work life. Right. This is the view that I care about. This is what I care about from AI. So this pulse system

is new in 5.0. And again this is not some product pitch. This is all completely open source. I want everyone to have this. That's literally the purpose of pi is to enable people activate people is what I call it with all this stuff to have these capabilities and more importantly, like kind of the point of this video is to think about AI in this way. AI is not here to be an agent for you. You're not here to manage AI harnesses like, fuck all that. That's ridiculous.

We are here to live human lives, enhanced human lives. And AI is a capability that we've never had to be able to have it go and research and do all this stuff for us, collect knowledge, constantly optimize. It's literally stressing constantly about how to make your life better, right? What better use of technology is there than that? So that's what this looks like. Click on agents. You actually

see look it's currently running the Pi upgrade skill. And yeah, if it goes into the algorithm you see a lot more content there. Oh look at this. This is a knowledge base. Look at this. We've got companies. Look, this is like every idea I've ever had. So Carpathia came out with this concept of an LLM wiki. So I basically took because Pi has been doing this already, I basically have every idea I've ever had kind of brought into this concept. It's actually listed, right? You have full

data on the knowledge. Look at this. This is every document for how Pi works. It's all in here. And guess what? Kai can read it and everyone can read it. Look at all the different skills and what they do. All of our different hooks that we have implemented are bold. That's a little bit sensitive. Can't show that one security system won't show that one performance. Yeah. Just absolutely insane. Oh, this is like, how much it's costing me. That's frightening.

And here's the assistant. Here's Kai. He's online. He's doing tasks. We've got different, you know, scheduled tasks and stuff like that. This is just like absolutely insane. I can't tell you how powerful it is to have this all in one place, organized in this way, oriented around your life, your human life, instead of around the tech itself. The tech is not the point. The human is the point. All right, so that's what I wanted to talk about in this video.

Essentially combination of multiple ideas. But the central theme is that I think it's very clear where all of this personal AI tech is going. And the reason I like to offer this is because it can be very stressful to constantly be tracking harnesses, constantly be tracking different platforms, different chatbots, different agents. Oh, which platform should I use? Blah blah blah. Think further out. Think who is my da? What capabilities am I giving my da? Have I defined

everything about myself. And given that context to my day, and have I defined what my ideal life looks like? Have I defined what ideal state looks like for me, for my businesses, for my finance, for my health, for my relationships? Define what ideal state looks like. Make it very clear to your Da that their goal is to use all these capabilities, all these skills I've got what? What is it? Over 100 skills. Got all these skills proactively,

not reactively. Proactively monitoring for the current state. AM I surrounded by someone that's like looking over my shoulder and trying to steal something? AM I surrounded by someone who's about to steal my phone? Is someone about to attack me? Is someone following me on the street in the middle of the night? There's safety issues. There's enhancement issues. All of this encapsulated into a structure that is visible and

understandable by your Da. I think it's incredibly powerful. I think it's a really powerful way to think about where things are going. And I really look forward to seeing what you do with this idea. Respond with feedback. Let me know what you think and we'll see you in the next one.