Waste Tokens, Save Time

Welcome. You're listening to the Naval Podcast, your authoritative source for new knowledge. We're trying something new today. Uh I have three Frontier founders with us. Um three good looking guys actually and a fourth good looking guy, Naval. And let me just introduce everybody. Guillermo the G Ralish. He's building Vercell into an AI cloud for the world of agents and whatever comes after that.

Blake Shaw, he's building supersonic aircraft in his own factory and jet engines as well. Blake's company, Boom Supersonic. And then Max Hodak from Science, he's building a biohybrid brain interface. that grows living neurons on silicon to restore sensory functions like sight, but then eventually to explore new parts of the brain and new senses. All three of these guys are not composing their products with off the shelf parts. They're building their own factories.

And you know, we don't care as much about what they're building exactly as we do about what they're learning about how they're building. What's the new knowledge they're generating, what's their alpha, what principles are they discovering that other founders can learn from.

What are they trying to figure out right now? And also what are the cutting edge or crazy ideas that they haven't even talked about yet and they're still forming in their brains. Naval, do you have any reactions to any of that before I jump into Guillermo? yeah let's just have fun Yeah, you guys should just jump in.

Yeah, so I can't remember my exact quote by the way, but I've been really pilled uh with this idea of software factories and the job of the engineer being something that you just show up to work. used to used to ship the output directly and everything that inside the company was, you know, how good is person A at shipping output B? And now what's happening is the way that I'm judging you as an engineer is like

are you producing the factory that will produce multiplicative outputs B to through Z, right? Um and that's a that's a pretty significant change because basically like we used to believe and it used to be somewhat controversial that there's 10x engineers. Like now clearly there's a hundred X or a thousand X engineers and the world hasn't fully adjusted to them.

I I used to get flamed on Twitter for saying they're ten X engineers.'Cause you know, it flies in the face of so much like equality philosophy that everyone's equal. But the reality is when you're operating in idea domains, when you're operating in intellectual domains and virtual digital domains, it's not even ten X, it's hundred X or a thousand X. I mean it always has been. Satoshi, notch, you know, the guy who invented JavaScript, the Brendan Ice of the world.

uh John Carmack. I mean these are thousand X programmers. Not to even mention if you choose the right thing to work on versus the wrong thing to work on, that's an infinity difference. And it could just be not necessarily a better programmer, just one who had a better judgment on what to work on in the first place. And now obviously it's less controversial because of uh AI leverage.

What's controversial is the the token leaderboards, right? Like people are still getting a little confused because now they think, Well, I have a bunch of hundred X engineers, look at all these tokens that I'm paying for. I'm curious, uh if you guys have seen the same, like how do you measure ROI? It's like the old measuring lines of code. You know, token consumption lines of code feel like similarly not direct paradigms.

I mean my observation has been that Claude or ChatGPT um or GPT is about is basically as good as you are in a domain. And so uh if you're s if you're a really capable developer, then these things are really powerful. And if you're a junior developer, then you'll kind of find it to be like more of a junior developer. Like on the one hand, these models are incredibly capable. On the other hand, the feedback that you give them

Sporadically seems to be incredibly important. And these little updates seem to totally determine the types of uh performance you get out of them. There's a new kind of support that I give, which is you come to me and like you didn't get good output out of the model and I tell you what to prompt the model with. So like the idea of like the quality of the reprompting, which I think you're alluding to, is is extremely important.

But I mean, and to be clear, I think that this will become less important over time. Like as the models get much, much smarter, then you'll be able to put in less and get more out. Um, but at least at this stage, it really seems to kind of reflect back the judgment that the user brings in in my experience.

I've kind of resisted learning all the ticks and tricks and tips. Like, you know, there was uh, oh, use Ralph Wiggum, use OpenClaw, use Hermes, use this prompt engine, use this scaffolding, plug in this piece, you know, uh Always use plan mode. I just ignored all of that. I just assumed the model is just gonna get better faster than I would figure out how to use it. It would figure out how to use me faster than I would figure out how to use it.

And so I've just been completely ham fisted with them and I get frustrated at them and just sort of I I found myself typing less and less information and doing less and less work as time goes on with the models because I just assume I can brute force my way through it. And I'll throw Codex Claude and Gemini at the same problem over and over and just waste tokens to save time. And I think no matter how expensive these models might seem, they're still way cheaper than a human.

So I would say just waste tokens, save time. Don't look at the tokens either as inputs or outputs. Just look at your time and look at the final output. And even if they're writing low quality code, which I know in many cases they are, it's not necessarily production quality or scalable code. When the time comes and I want to ship it to production, I'll just throw more tokens at it. I'll say, okay, now go through, look at it, rewrite it, and they're just gonna get better every generation.

Uh so yeah, I don't I don't see where this necessarily stops. As long as we have verifiable domains and solve problems, they're gonna resolve those problems. Not in the unsolved problems domain where maybe you're Terence Tau, you're the cutting edge of creativity.

that you need to be, you know, working very collaboratively and carefully and closely with the model. But I'm not in that I'm not at that level in software engineering. Uh Girva, you're probably the most extreme software engineer in the team, right? Like out of this set.

You're probably the one who most hardcore came up from a software background. Like how are you finding these models at the edge of their capability? Well there's one thing that's happened recently that uh what you're saying resonates strongly with, which is It used to be that you would give a prompt to the model and it kind of does the like classic like next token prediction thing and it like runs away with your idea.

And models now have been doing this like intuitive planning mode without to your point not even having to plan, where it it comes back to you and says, Look, what you're asking me for, there's these three routes we can take. there's this set of trade offs that we're gonna go down. That's a moment where like, you know, people do the whole thing on X, it's like, oh, now we have a PhD level engineer model. Like that's very clear that

The models at some point graduated. They used to be junior engineers. Now they're principal engineers. Because they come back to you with a set of trade-offs and obviously sometimes they bullshit, which is hilarious. It tells you this one is gonna take three weeks and this many token it just makes it really bad predictions. But clearly it's now.

this like I respect the models a lot more as a as a peer like that I'm going back and forth intellectually with. But there there are a lot of gaps still. So like if you're a really, really proficient engineer or architect, you I think you're still extracting more juice. So the question sort of that Max was positing of like, if you're junior, do you get junior back? Well clearly not because

A junior gets more advanced knowledge in code that they would have never been able to write by themselves. But doesn't an experienced architect get ten X, whereas a junior engineer gets two X? That's what I'm kinda trying to figure out still.

Yeah, but I mean I think there's there's architectural decisions. So when you think about the development, I'm seeing this now with some of our the junior software and chairs of the team of like what is the next step in their career progression? It's going from like writing implementation for a feature

to picking technologies, like choosing between Postgres versus some other database, or picking between ZMQ versus some other message queue or like some other queuing system. And those, I mean the models can suggest them. But that's the thing where you'll see it and you'll be like, No, no, no, I want to use this other thing. That's the type of little feedback that I'm saying really matters and the types of output that you seem to get at this point.

Taste taste and judgment, right? Taste and judgment. That said, you can ask them which one should I use and why? And they know everything. They'll give you really good trade-offs. That's the change that I was saying has happened recently where you would say, Hey, go and um

put this super high cardinality telemetry data into Postgres and it's like, oh bro, like we don't put that kind of data into Postgres. Like you should consider ClickHouse or Athena or whatever. Like that's happened to me a lot, which is really impressive. Um but I I the the thing I'm still like kinda struggling with is Clearly the human is still completing the model. Like at one point is it the other way about. Like the the human is the one sort of getting the instructions back on like

Go get me this API key, because it's something that only you can do. Uh or get me this amount of capital for my next set of investments that I need to make. Yeah, you just watch. Like that clearly we're still not there. That's a temporary aberration. Pretty soon every good SaaS company or hosting provider will have a CLI and API interface that the models can be directly. They don't even necessarily need an API. Like as long as it's like text-based, Unix-based.

the agent can hack its own API. And then the money part, you insert crypto tokens, you know, put in Bitcoin, put in whatever, and the model goes and just pays for whatever it needs. And I think like, you know, the people working on this But the thing I am now thinking through is is pure software dead?

Like is pure software engineering like an obsolete thing? It's like saying speaking English, right? The models now speak English. We had to learn code to communicate with the models. Now the models speak English. And they speak fuzzy, sloppy English, like a human, and they understand things.

So where's the moat? Like for a founder? Hardware, it's a boon. You know, like now if yeah you had to build hardware. It was hard to build a software company alongside, like Patrick Collison says software is art. And it's hard to hire artists. So now as a hardware founder, great. You can have really good software developed fairly quickly. Um, if you're creating models, maybe that's the new software engineering, training models and tweaking models and post-training and fine-tuning models.

But classic software engineering is that dead is pure software investable, is pure software something you can organize a company, a team around and try to get some leverage. Did you guys see that? B uh there is an article on X by Mitchell Hashimoto called the block economy or the building block economy, something like that. Like his argument is that the most useful thing for agents to have now is really powerful reusable building blocks.

'Cause to Max's example, you wouldn't expect your clanker to reinvent a Q infrastructure system every time it needs to send an email. You need to bring in the right building block that's right size for the task that you're asking for. And you say, well, okay, for this one it's bull MQ.

I challenge the notion that I would want the agent to reinvent the entire universe from first principles in a way that's incompatible with the rest of society and civilization. Like it's almost like reinventing highways, laws, policies, et cetera, just for you. Even if there is a uh potential for extra optimization and extra juice that you can get out of it There's a still a um sort of like cooperation at large scale value of saying we're both depending on Postgres thirteen point two.

And so that's is still really, really, really valuable. I would say like The category of infrastructure software and building blocks. that this agents are gonna use obviously in bias is this what we're building seems extremely valuable and i i don't see the agent anytime soon and by the way you could even uh another metaphor that i've been using is like anything's already been created that the models can reuse is like a token cash

Because you don't you don't want to churn through a trillion tokens to reproduce what's already existing. Uh and so there's always a starting point that the model can fork off from. But it's gonna change things quite profoundly. So these are like Like libraries and dependencies, but for models. For agents specifically.

To Naval's question though, I mean, I learned a program when I was really little and I like that was the thing that through all of like being a teenager and in my twenties, like I get like sucked into it and just like

code for like twenty hours and it was super fun and I knew all this stuff about programming languages. I haven't written a single line of code in quite a while now. And I mean partly that's because my job is different. But also since December I've built a huge amount of software that I now use every day. There's all these projects that I've kind of fantasized about for years that now I'm like using.

Um, that I've actually built. And I didn't write any of that. And I just can't imagine going back to like actually writing code by hand any time. Like I mean I I'm unlikely to do that anyway, but just like in general. I see that I have a hard time seeing that as part of the future. There's something really cool is that you understand how the pieces click together. Like I feel like anyone that understands what an API is and how data flows, inputs and outputs.

performance'cause you kind of you have to orient the model around like this is a certain level of expectation that I have out of this operation. Like that's that's always been infinitely more useful than um than writing code. Like I feel like a really good A proficient engineering leader has been quote unquote like vibe coding through people on Slack or one on ones because you're transmitting your will, your intent, your experience, and you're letting others run with it.

Uh it's just that now we do the same but with agents. Uh and so I think that's why you've been successful with it, but I don't know that everyone sees the same level of success. I mean, I went from not having written code in twenty years to I'm coding all the time now, but through agents and I'm building tons of software and

It turns out that just understanding the basic principles of software engineering and algorithms actually gets you a long ways. Because the reason I stopped coding was because I didn't have time to figure out the latest language, latest architecture, infrastructure, pieces to plug into and I know Versell makes it a lot easier, but even then just getting started was a bare like just plugging pieces together, assembling infrastructure was just so annoying.

The thing that really changed is, I mean, it used to be that you could build a lot like you like there's a lot that was straightforward, but then you would hit some random thing. And then you could spend kind of some indefinite period of time debugging some narrow thing. And now with the agents, what happens is you just don't get stuck anywhere. Which is pretty amazing. Or they get stuck.

Well no, I mean like relatively quickly they can find like the right way to do things. And it used to be that like I remember when other friends learned a program be like, nope, it's just like intrinsically frustrating. Like if like that's part of the deal, that's how you learn. And that just isn't true anymore.