Timing My Vibe Coding

Hi, I'm Paul Ford. And I'm Rich Ciotti. And this is the Aboard Podcast, the podcast about how AI is changing the world of software. And Rich... You're away. You've been in Lebanon for a couple weeks. You're coming home soon. But while you were away, I got up to something. Oh, boy. I vibe. Mm-hmm. I got to tell you. I got to confess. I vibe-coded. That evil mistress vibe coding. Go on. Okay, let's talk about it.

Okay. So what is, okay. In two sentences, we did not build a board. Go to a board.com. You put a prompt in, it builds you some software. How is that different from vibe coding? Why are we different? Vibe coding. takes your prompt and just sort of starts coding. And we don't do that. We take your prompt and we do a lot of thinking, a lot of agents talk to each other, a lot of prep work, a lot of sort of...

setting up, and then an app gets generated. Very different. But at the same time, I feel as a technology company leader that I need to, especially when I'm telling somebody that we... don't do something that we're better and special in some way that I need to really keep on top of what

the other thing is namely vibe coding like what's going on over there and i realized after you left because nobody was talking to me that i hadn't done anything on clawed code or with any of these tools in a little while

And I was kind of, I might be talking from things that were three or four months old, which is really an eternity in this world. So I sat down and I decided to make, and there's a link, you can go check it out. open source software that I made with my with my buddy Claude Code, I made a time timeline explorer tool that lets you with one slider go all the way from the Big Bang. And then on the left

all the way to the heat death of the universe on the right with a bunch of more contemporary stuff in the middle. So some complicated math. And then as you scroll around, it shows you little events and YouTube videos about... that moment so like i hit world war ii i can watch a world war ii video i want to learn about the heat death of the universe i can learn about so a cool little resource for watching interesting nerdy dad style educational content on

bus. And I wrote about that in the wrote about it for the newsletter. We're talking about it now. But I thought what I would do is kind of tell you a little bit about where I think we are now. in terms of vibe coding. And not purely so we can talk about how we're better or different. I just think that this is still a really fundamental change that we're processing in the industry. And I thought this would be a good chance for us to kind of get back into...

some of the things we've been talking about over the last year or so about what's changing. So the, like I said, the app is available on, it's, I put it on GitHub pages. It's got lots of videos. It works. Okay, it's got some gaps, does have light and dark mode, but it is an accessible web-based app. that you worked on mobile and desktop. And it is the sort of thing I did this sort of casually over a couple of hours. It is the sort of thing that it would have been hard to complete and get live.

Even though I think I could have built a lot of this by hand back in the day, I think like getting it out into the world on GitHub pages, pretty accessible, good for mobile. I do think that that is something that helped me here. So I thought I would just talk through the whole process.

And you could riff with me. You ready? Go. Okay. So we'll put the link to it in the show notes. And if you look at the newsletter, you'll be able to take a look at it. So let me describe what I did. Ask me any questions as I go. Minute one. I sat down and I said, I want to be chill here. I don't want to use React or a whole bunch of big web libraries. I want you to build native web components. Use basic web technologies. Don't do a lot of weird stuff. Make me...

a web page, and a couple JavaScript and CSS files when we're working. And I got to tell you, that part, it killed it. It never veered. I said, keep it simple, keep it readable, keep it easy. And in this case, when I'm... I'm saying this to a terminal-based application called Claude Code. You just type Claude in your terminal.

and you sort of talk back and forth to it, and it writes code and creates files in a directory, and then you open up your web browser and you look at what it's doing. Okay. Okay. So now...

So that took a minute. Pause for an hour because now I had to tell it. about time and i had to explain that because i wanted one slider that started at negative 4.3 billion years and went all the way to trillions and trillions of years on the right but in the middle was kind of like around the year 2020 That slider has to do a lot of work. It has to do math as you scroll. And that was like, you can't just say that in a sentence to an LLM.

For a while, it had the American Revolution happening four trillion years in the future. Um, it was really, uh, it was a lot. And so we had to do that. I had to do that like five or six times. And I will say like, I don't think people talk enough about how much of the interaction with these things is repetition. Like you go back and you just are like.

okay, that's not it. Let's try it this way. Let's I need you to write this and then you end up looking at the code and so on and so forth. But we're already out of the miracle zone. It's still pretty good. Like it's doing a good job and it's writing the the math and into doing some of the really boring stuff well but I would say definitely an hour and I haven't really gotten started yet I mean

I'm going to guess that this is a tricky problem, that you might have had the same issue if you had an engineer to your right. Yeah, I will say what the engineer wouldn't do is throw something over the wall where the... American Revolution happens trillions of years in the future. That instinct of just something being wrong isn't there. Yes, correct. Good point. As far as an LLM can tell, if the code executes without producing an error...

it did a good job and a human does has yeah exactly a human has more discretion but then okay i got that working it was pretty good and then i was like you know i gotta get some data in here i need some like i needed you know, the Jurassic period needs to be in here.

And more sort of events and sort of more stuff. And so I'd thrown a few in, but I needed more. And here is where it is really good. I was like, go look at Wikipedia. I found a list of like eons and timelines and stuff. Go read this page and turn this into data. that we can use in this timeline. And that used to be, I would say that was...

That could easily be a day of work because parsing is hard. Natural language is hard. Turning natural language into data is really hard. And you're using lots of libraries and lots of principles. That's 10 minutes now. Okay, so that whole domain of I need to take something really abstract and turn it into data so that I can go to the next step, that's gone now. That just works pretty well with this new world. I also noticed that it like...

At this point, it starts to investigate its own work. It's getting better at looping and being like, oh, hey, I don't know if that's the right way. Maybe I'll try it this way. So that whole space is getting better and better. Except I entered into a journey now. I'm going to describe this journey to you, which is now I want to go get the YouTube videos.

And I said, Claude, you did a great job with those Wikipedia pages. Go get me some YouTube videos for each event. Jurassic period, go find me some good dinosaur content. I don't know. All those events, I was like, go get me one YouTube video per. Okay. Completely improv comedy. Just came up with absolute nonsense. None of them were real.

and in fact it really couldn't get me youtube links it kept finding me web pages that had videos embedded like just went on a journey and i couldn't i couldn't figure out what it was doing or why yeah okay And then so I was like, all right, well, it's not Claude. Let's go try. And I had a stroke of genius. I'm like, who owns YouTube? Who owns YouTube, Rich? Google. Alphabet.

Google, yeah, that whole world. So I'm like, I'm going to bet you Gemini is going to kill this. I'm going to go use Google to find me a whole bunch of YouTube videos. And it was like, I am ready to help you. Absolutely. I felt really, really good about myself. and uh another debacle every video it produced seems to have been real once but it was like out of like 50 videos it found it found maybe like one or two

were real. And the rest would be like, this is no longer available. No idea what was going on there. So not Google. So then I did go to chat GPT. I used O3. I had it make a list. And it did a great job at first. And then it was like, man, you know, this is a new thing. It's like, do you want to go further? You know, you've had a taste. Do you want another taste? Yeah.

Okay. And it was like, and it goes, I will make you so many videos. I will make you a thousand videos. It is going to be absolutely just stunning for you. And I was like, that sounds absolutely fantastic. I really want to do this. I did. I was like, go make me a thousand videos. And it told me it worked. It worked for a while. I used deep research. It's like, I did it, man. I did it. I made you a thousand videos.

whole plan, everything. How many videos do you think are produced in that list? So I'm assuming it's referencing the videos. I'm going to guess 100? 20. 20. So 20 out of a thousand. Yikes. And the thing is, it was... It's just so confident. It's so enthusiastic. It's sort of like everybody's like, you know, it's like an intern or a junior employee. No, the 20 were of extremely high quality. It's more like a weird poet where it's just like, I can only write.

that one poem for you i was like yeah i needed you to do like 50 50 poems nah but the one's really good so and then you know into so now i'm in like i'm in three llms claude is wearing out

Claude, I keep asking Claude for things. I keep trying to bring some of the GPT stuff back. I don't really have, and you know what's tricky here is... If I was doing classic programming, I would have a folder. I would have a structure. I would be like, okay, this is where the text is going to go in. I'm going to commit it in GitHub. I'd create a process around this whole thing. But instead, I got all these files flying everywhere.

And I've crossed what I'm going to call the prompt productivity threshold, or you could call it context exhaustion. And I'm sort of in the bad place. it's it's getting very random and i don't know why and i'm starting to kind of like hit the keyboard and then by this point i'm now like four six hours in and the reality is yes you are correct

I could have done some of this by hand and I think I would have had a lot of success. But then a real surprise, Claude ran out of money. Which is fascinating, right? I think that's the fact that we pay. For it not to do the job is something else right now. I don't know how long we'll have that appetite or tolerance for that, but here we are. No, and then I gave it more money.

It was like, look, if you want 7x what I just gave you, you got to give me $7 more a month. Okay. Which, okay, I did it. I got out the corporate credit card. You know, I'm learning. And I will say of all the experiences I had with Clyde, man, getting in the terminal and giving it that $7, it was seamless. One of the best. It was quite smooth. They nailed it. They are really good. getting good yeah and so now um it's just like i'm in a bad place i mean i'm not like and again

Something does exist. I've got like 30, 40 events in here. It's pretty good, but I was excited to get my thousand. Like if it hadn't promised me, chat GPD hadn't promised a thousand. If Claude hadn't said it was all ready to really do this for me. I wouldn't have been so into it. But I hear I have something working. That's pretty cool. And now I'm like four or five years. Four or five years. Four or five.

hours into trying to make it really good. But really good is out of my hands. I can't vibe code through it. And I will say like I after I did this exercise, I went and I looked at you know, all the awesome Claude code tools and all the others like Did I miss something? And I've got to be frank, I don't think so. I don't think there was like, there are lots of processes where I could have had.

a better a better outcome and we'll talk about that in a minute but there was there's like no magic trick here where i could have gotten from the thing in my head gotten it onto the screen simply through prompting um

I will say, though, then I was like, all right, well, let's just deploy what we got. I got to talk about this with Rich on the podcast. I got to get it out in the newsletter. And then there is a set of tasks, which is like, I need this deployed on GitHub. I need it on GitHub pages. I need a build.

process and so on those things could take hours sometimes days and it just does them instinctively so that part of actually getting things over the line so that other people can look at them so it's this very paradoxical thing where like you're aiming for perfection but the actual some of the killers in engineering projects were that last 10 percent of actually finished publishing it, writing the unit tests.

automating processes making sure it gets on the server in a very predictable way that whole class of challenges that was an absolute murderer for small projects i do think is gone now right so so where are we winning we're winning on a little bit we're winning on content parsing we're winning on getting started we are winning on deploying but we are not winning on the part of like let's let's steadily increase quality and turn this into real software and i went back now for

couple more hours i tried to get chat gpt to up its game but just kind of kind of no dice so yeah i'm launched i'm in the world i can talk about it but i am unfinished that's the the way that i would do this and so i would describe this

And I'll close this out and then I want you to kind of tell me what you think about the world we're headed towards. The way to do this would be to stop as I hit the multi-hour, like, try to get it to work in a specific way. And then define a process, define a set of atomic, small, achievable goals, and then prompt those one by one, incorporating them into the code. Because... it keeps trying to you know a way to we've talked about this before it kind of pushes you into a narrative

Whereas that's not what code really is. Code is more like you're putting a building together and you're plumbing systems and electricity and sort of... you know carpenters need to come in and so on and so forth and so like i think like it's i'm trying to tell a story To the prompt about what I want and I think that that is really in opposition to what finished code needs to be

that's where i'm landing even though i'm really glad these new technologies exist and i like the thing that i built and i want to now that i've said all that i really want to get your feedback okay um I have two thoughts, and then they're really kind of set up sort of a final read on vibe coding in general. The first thought is this. The first thought is that, well, let me...

Let me really share a little story to amplify the first thought. And that is, it is my son's birthday and I walk into a bakery and I say, I need a cake by tomorrow. Okay? I come back the next day. The box is there. It's ready for me. All set. You're going to love it. Thank you. I give him. 50 bucks, take the cake home. I open the box. It is, the cake is for whatever reason in the shape of a violin. My son doesn't know anything about violins.

It says, happy birthday, Sebastian. My son's name is Anthony. And it has a candle on it with the number 71. And my son is 12 years old. Okay? I mean, I think I know what happened here. I just, the baker didn't take out a little notepad and say, what's your son's name? What's he into? Is he into baseball? Is he into soccer? Does he like Marvel Comics? And how old is he going to be? And what flavor does he like? By the way, when I cut it open, it was blackberry filling.

which it was okay, a little exotic for a 12-year-old. I think we need to be clear that this is a metaphoric journey here. It actually sounds very believable. What I'm trying to say is this. The baker would never just say, okay, you need a birthday cake by tomorrow, and I will make a collection of assumptions and just run with it. The thing that AI will do...

is that even though you put in probably a pretty good credible prompt, right, is that it will not come back to you and say, you have to clarify this for me. See, the thing about humans, they will say you have to clarify it because they want to do a good job. They will not just say, yep, I got you, and off they go. They will say, we need to have a dialogue. Let's be clear.

The reason it can't do this is because it's not human. It's a vector database that is statistically matching like inputs and outputs. Another way to look at it is it sees no gaps. You essentially said you made a request and it's decided. that I will take this request and I will run like the wind. I will run like Forrest Gump and we will go. The way AI works is that it takes your prompt and runs.

It doesn't pause. It doesn't say, hey, wait, I'm really not sure what you wanted here. Should I do this instead of that? It just keeps going. And it goes and goes and goes and goes. And when it finally gets done, It doesn't pause and say, hey, what do you think? So when you actually ask it to refine, it's not very good at refining. It wants to keep going. And so it's getting better at it. You'll often see Claude sort of run back through its own code.

but it really doesn't understand your end game and you didn't give it a spec you gave it a few sentences or a nice big paragraph but it's just not enough right so that's problem one problem one is it just makes a bunch of assumptions and just keeps going

And very little software is built that way. Usually there are specs written. There's a ton of dialogue. There's a lot of refinement and whatnot. Let's be real. Let's be real. This is on us, not on the technology. Correct. It's also on the people who are making Claude. they're making chat gpt they keep trying to simulate this i gotta say i've been using chat gpt more and more it is hallucinating like all the time i'm really having trouble getting it and and it just it gets and it's more convinced

than it used to be, that it is of falsehood. Correct. Now, the second point is not on us. The second point is really on... uh on ai which is okay let's say i you did give it a spec and you did give it all the details and you did

put in exactly what you wanted with that cake, right? It's got to be shaped like a soccer ball. He's 12 years old. His name is Anthony, blah, blah, blah. There's another problem that arises, which is... and you alluded to this earlier, is that AI does not think structurally with code. It thinks in a very linear way. And so it doesn't organize its thinking first. It just goes, right? And I think that is the tougher hurdle here. And it's tough because most good software...

As planning and thinking and architecture underpinning it. It isn't just a script. There are scripts, by the way. If you wanted to write a script to scrub your folders. Not most. All.

All serious long-term software has serious architecture, no matter whether it was arrived at after initial development or before initial development. And so I would make a modest or maybe not so modest prediction here that... Unless there are some significant breakthroughs, vibe coding is good for prototyping, good for extremely narrow widget style. outputs, but anything with sort of any meaningful depth or surface area.

and you're going to hit a wall pretty hard. And I think when you keep banging away at it, you actually end up further away for all the reasons I just alluded to. It's a tricky, tricky problem. It's one that we sort of foresaw out of board. I'm not saying that we solved it. But we took a very different approach around it. And we took a different bet, which is before code gets written, a ton of conversation and work and thinking happens. So let's simulate that first, right? And that's not us.

Hard selling a board. We just knew, our instincts told us that I don't think this gets you there, right? And we're going to have to take a different approach. I mean, I saw two people online, Simon Wilson and Tom Coates, were having a debate about whether someone should go get a computer science degree right now, among other things. It was a really interesting one because I do think that...

Because the I do think this interface will become a really big part of software building because it's very useful for understanding and introspecting a schema or figuring out like. learning where the bug could be. It's one of the best tools for teaching you about your own code or about code in general. that i've ever seen it can because of the parts that can analyze language or analyze symbols it has real power that way but it is that architecture aspect is

It's almost useless. It's almost counterproductive because the thing that I did where I went, oh, boy, I've been in this for a couple hours and trying to figure out where this goes.

I kind of know the approach I would need to take breaking it down into all of its individual components. I know how to... architect and application basically and so i'm able to bring that knowledge to this without that knowledge i would never make progress on a thing like what i was trying to build like i just like i need that architectural sense in order to achieve anything and so

I don't know if knowing a specific syntax or knowing about optimizations around math is going to be as important because this thing will always be able to help you with that stuff. The architectural components, as opposed to a lot of optimization focused stuff, I think that will become even more important because you'll be able to build a lot more software, but it'll just be a big spaghetti mess unless you have a very, very...

rigid process working with these tools. So I think we're headed there. I think we are one approach. I think there will actually be thousands. I think this is kind of the conversation our industry needs to be.

be having uh but it was interesting to go back in there richard because it's you know when we first one of the reasons we rebooted a board was we started to see this thing write code and we went man that is a change in our industry and i think we're getting used to that now but it is um i don't think there's there is no get out of jail free card here you have got to learn how the computer works and what's going on in order to accomplish your goals

Yeah, I think there will be new majors in school. I think a lot of people ask us, what do I tell my kid who I think should be a comp sci major or a computer programmer or whatever? I think you still have to go learn those concepts, and I think you're touching...

on it I think new skills will materialize that complement what these tools can do we don't exactly know what they are yet but understand how it all works understand architecture I think is really good advice for young people you're a seasoned programmer

Like I am as well. I don't code every day, but we understand very important concepts that underpin everything. A lot of young programmers think they could just pick up these tools and run. And I think you just end up hitting that wall. I'll close on this. You know, I think that like, I think the trillion dollars in change in the industry and the way that consultants will be able to build stuff and all. I think that's all real. But I think the time horizon for me is starting to feel more like.

five ten years metabolizing with a whole lot of other stuff as opposed to like this really rapid disjunction that we were expecting right i think it is definitely it's not gonna this is going to take a while yeah

Well, good. All right, well. Check out our alternative approach. A board does a lot of thinking. It asks a bunch of questions. It writes a few docs, a few agents get deployed, and then it takes a crack at the software tool you want. It is really geared around business software, productivity software, and the like. But we took a really different take on it, and we'd love to hear what you think. Check it out at Abord.com. Paul, I just want to give you credit for showing infinite patience.

as I try to navigate Lebanese infrastructure. Yeah, we've had a few glitches getting these podcasts recorded, but I have nothing but faith in Lemony's infrastructure long term. All right. Hello to board.com. We love you guys and we will talk to you soon. Rich, I'm excited. to see you back in our fancy studio with our view of hudson yards and uh and let's get to work all right have a good week everyone