From "AI mid-life crisis" to the "time of my life" | Steve Yegge (Sourcegraph)

It's funny to watch people melt down over this. I guess I had my AI mid-life crisis just a little before they did. I had mine about a year ago where I just had an existential crisis. It hit me hard. I was stunned. I was like, reeling because I realized that many of the things that I thought that I kind of like my value, my sense of self-worth, my identity was tied up in my ability to do things like

memorize long things or generate long things or whatever, it's all just BS now like computers do way better, right? So like that was tough. I think a lot of other people haven't gone through that yet, so my blog really touched the nerves. Welcome to the Software Misadventures podcast. We are your hosts, Ronak and Guang. As engineers, we are interested in not just the technologies, but the people and the stories behind them.

So on this show, we try to scratch our own edge by sitting down with engineers, founders and investors to chat about their path, lessons they have learned, and of course the Misadventures along the way. Awesome. Steve, you are very well known for your RANs, the Google Plus RAN, I think being the most popular one. And I will categorize it as a very insightful and suitable for the chili emoji for the spiciness of it.

So me and Ronak, we also RAN, but we do now have the near, which we'll call it, skill level. So maybe some pro tips to get us started. Like, who makes a really good RAN in your opinion? A really good RAN. Well, you got to start off by being really frustrated, ideally a co-worker or a team that's not giving you an API. And then you get really, really hammered one night. And the rest usually just comes naturally. Love that. And when did this sort of start, if you can take us back a little bit?

Yeah, so I mean, like I kind of started it before there was like any thing like blogging or anything like that. I was doing it at Amazon. Okay, because when I started Amazon, there was only like 250 people in the whole company, including customer service. And so the engineering team was only like, I don't know, maybe 50 or 80 people.

And so as they grew, like I knew everybody personally, right? And as we grew, like there was a large contingent of people that was using Java, but they were just like having to build really comical, really dumb stuff back in the early days of Java. And I started getting, I started getting kind of mad that we weren't using better languages. We were using like Pearl and PHP and just a bunch of, right?

So I started this kind of like soap boxing internally, right? It's like like 150, 200 people. It eventually grew to like 700 people. Eventually I left, I went to Google. I brought him with me the CTO of Rinner Vogels. He like called me up and said that I needed to remove one of them. Because it gave away a little too much confidence. Information was so fine. I took that one out and I published the rest of them.

And nothing happened. Nothing happened for six months. So my, actually my biggest piece of advice to you is like, if you're going to write something big, like write it, but then don't expect anything to happen for a few months. It takes time for people to like absorb, right? Six months later, like on the news, all said, and people start walking up to me at Google, going, you know, I read your blog.

I'm like, well, which one? And they'd like name one. Oh, okay, weird. And then somebody else the same day, if you like, I read your blog. And I'm like, yeah, which one? And they named it different one. And I'm like, what's happening, right? And it was, it was that somebody had found them and they made hack or news and people were reading all of all the old Amazon. Right? So like that pointer is like, well, I'll keep going.

So that part's pretty cool. And I'll say, typically a lot of things, I think most engineers ran, but most people ran in private on calls, which are either not recorded or people don't write about it. In many of the posts slash grants that you have in sometimes your YouTube TikTok series, too.

You're on a freight to share your opinions and sometimes name names as well. And typically, at least when I was starting out in tech, I was told Silicon Valley is a tech community in general is a very small world. So just make sure not you don't piss anyone off even if you don't like them.

And it's like, OK, that's a fair piece of advice. Now, in your case, when you're sharing these opinions, a lot of people tend to agree with the opinions you have, but the people you're sharing the opinion of may not necessarily feel that way. So I'm curious, was this a concern ever in terms of when you share your thoughts or like, you know what, who cares that you share? I mean, yeah, almost every single post they ever did at Google pissed off one VP or right.

And they did grumble at me about it, but they wouldn't censor it. But you know, I'm going to say something that I've never said before, but I'm becoming more and more convinced of it as I get old, old, old. Which is that companies, they really are people, or they're very person like, and they can have a personality, right? That's kind of a reflection of their leader plus their culture, plus maybe their domain.

And I think that as we move forward in this world where oligarchs are having more and more and more power and sway over governments, the only people that are powerful enough that we can turn to are corporations, right? And most of them, they don't talk. They're secretive, right? They're all like a bunch of cloak and dagger stealing around alleyways because they got, you know, gun shy from lawsuits or whatever, right?

Definition or whatever, right? You know, plus plus your brand is always in free fall. I don't know if you guys know this, but the Google's brand, when it was peaked, it was like early 2000s. And I was talking to the public relations folks and they were like, yeah, you know, brands always decline, basically over time.

I don't know, Amazon might be an exception to that, right? They've done really, really well with their brand, but, you know, yeah, Google, Google, you know, people decided they were evil, like, you know, 10 years in, they weren't trying to be, I don't think. So, so yeah, I mean, like, I think that companies need to talk more about what they do and what happens inside of them and the learnings that we can all get from each other,

that we don't share with each other. I think it's absolutely ridiculous that people are so afraid to talk about the stuff that we all experience every single day in our corporate jobs. We don't share. And you know what? The time is, it is time for sharing because tech is under a tech because they're considered to have an awfully uninformation, you know, like, you know, unbiased true stuff.

And that attack is going to catch us real off guard unless like tech gets together. And right now it's all back, I'm serious. You guys know this is happening, right? So, like, I'm proud of the fact that I speak up when people are doing stupid stuff, you know, and I think more people need to do it. And you just need to just, if you've got a voice, like, get it out there, you know, and if you got something meaningful to say, you know, make it funny, but when people listen.

So, you really got me rantin' that, that's one new brand new thought for me, right? Very well said. And in terms of the making it funny part, so like, I think this is unique to your writings style plus speaking style too.

And I think that's what at least in my opinion makes a lot of difference where if I, or someone else just goes online and says a bunch of angry things, add people or add things that they are disagreeing with, a lot of people wouldn't be happy, even the ones who would agree with it, because they would just see the anger. But then when someone reads your posts, they tend to always nod at least I do. And I'm also laughing while reading it or listening to you. How did you develop that unique style?

Look, you gotta be able to make yourself laugh, right? I mean, like, genuine, you gotta think of something that's so funny that you were burst out laughing out loud. And you're like, I could never say that, and that's the germ of your idea right there. It's like, okay, how can I say that? And get away with it, right? And it's, you know, it's a bit of art, but I mean, like, you've got to practice it all day long.

You've got to, right, just see that humorous side of things and sort of remember it. And then when it comes time, you know, be bold, right? Say the funny thing. It's rough these days, right? But I don't know for some reason there's a lot of low-hanging fruit in the corporate world, right? So when I have those ideas where in my head, right, I think of something and I'm like, oh my gosh, I'm like the funniest person ever.

And then, you know, 30 minutes later, I go and I write it down and then I try to, you know, like position in, I'm like, okay, you know, maybe not the funniest person I know. Is there like a process you use in terms of, so you said like practice that like all day, but like more concretely. Right, you hit a bottle of wine. I noticed that. I actually quit drinking in 2016.

But no, it's a, it's a, I guess there's the Mozart's out there who like already have it well formed in their head and they just write it down and it's nice and clean. And then there's the Beethoven's that just like scratch it out over and over and over and over again until it's right, okay? I think most of us are like the Beethoven's, right? You know, you sit down and it's not right. You got to sketch it.

You know, some jokes in there, but some of them aren't landing or they're in the wrong order or whatever. And so like it's, you know, the writing process is pure generation and you're just trying to enjoy yourself and make a bunch of interesting points. The editing process can take months. I have blogs that I'm sitting on that I wrote started writing last year that I still haven't published yet because they're not right yet.

You know, and you can tell why I'm rushed and when people are rushing me out and they got to get a blog post out, it's not going to be as high quality. When I can take my time and I can think it through and research it and make sure the jokes still make me laugh two, three months later. I'll pick it up and go, oh, this is good. Why didn't I publish this? Oh, yeah, because I said that. And so, you know, you go through this process, you know, and you, right?

Look, look, if you're not saying something that's like controversial, if you're not saying something that divides people actually fairly evenly, then you're not interesting. Right, unless you're, unless you're teaching them something, right, which is okay. But like if you want to be interesting, you got to stay set that's like kind of like you're not even sure if you 100% agree with it.

Right, but you know, you got to be kind of kind of thinking outside the box and and also, you know, I don't know. Yeah, but that's the important thing, right? Is if you can go, if you can go back and reread it later and laugh, then you know you got gold. I like that. Do you have these like archetypes in your head or like say proxies to friends or something such that you know that, oh, when they read this, they would disagree or like when they read this, you would agree.

That's how you form sort of like a 50 50 split sort of. Yeah, yeah, yeah, yeah. And I actually learned that trick from like, you know, writers like Stephen King and Neil Stephenson from their books, Stephen King's on writing is really good. And then Neil Stefansson came to Google and talked to us for like an hour and a half, right? That was a treat. We were there like, I don't know, 40 or 50 of us in the room.

You know, and you know, we got I got tips from all of them. Of course, the number one tip from every writer is right. Right. Get up in the morning and write and write and until you get to get at it. And there's a lot of truth to that.

But there's also a lot of there's a lot of value in picking up person and writing the post for them. One person and maybe having like a peanut gallery of other people that you know disagree and you kind of just like think about how they're going to take it to you, right? I absolutely do that. I mean, like I can even name the people that I wrote the last like five blog posts for. I won't, but I could.

Like, how do you find those people? Like, how do you yeah, like out of all your friends were like, you know, people that you know, like, how do you find the architect? You know, I talked a lot of people through the industry and a lot of a lot of my old friends are like super, super high up there now, you know, like there they've got, you know, 20 or 30,000 people in their org, you know, massive, you know, fortune 10 companies or whatever.

And it's funny because the walls and their their trials and tribulations are pretty much identical to the couple of hundred people companies and the series, and everybody's in the same boat right now, right? So listening to them talk. I think wow, I can tell a story about something myself here. And I know it it applies to there or two, right? So I'll just kind of like poke the bear a little bit like you.

I'm going to do this on stage in Las Vegas in two weeks. I'm going to stand up and poke the bear a little bit. Very fine. Yeah, I can give it actually know it at this conference. So, oh yeah. So, I don't know, where are we? We're talking about the art of writing. So you mentioned some of these books like Stephen King's on writing. Actually have it in my bookshelf. I haven't read it, but I will. Are there any other resources that you've come across that people could refer to?

I mean, you know, I've tried, right? But I mean, no, you know, I mean, like the ultimately it's. It's like that. It's like that Twitter exchange with the guy who wrote the worst week of all time, the room, right? What's his name? And someone asked, hey, I want to get I want to start writing the screenplay. What should I do? And he replied start.

Right? I mean, this is like fundamentally like, you know, everybody says that everybody who's successful in any sort of entertainment or whatever kind of business has been doing it for a long time. Look at you guys. You guys enjoy it. It meant success with his podcast and you been doing it for a long time now as long as high school. Right? Time really flies by. Well, I've been blogging now for, I mean, like, I guess gosh, probably 2025, close to 25 years.

And right? And so like you just, everybody tells you this persistence, patience, just keep it up because and Neil Stephenson said this too. He told us he said, look, your first novel is going to be a failure and your second novel, your third one. But eventually you're going to ride ahead and people are going to want more of it. And if you've got nothing, if it's your first novel, they're going to be, oh, later.

Right? But if you've got like a new, then no, no, they'll make the way through it and they'll find value and stuff that maybe they didn't see before. It's really encouraging. I know it's it's also discouraging because you're like, oh, man, I can't become an overnight success. And nobody on YouTube, nobody out there is. Right? You get a bullet up with it by little.

What kept you going when, you know, in that six month of like publishing all these blog posts, right? Which I assume after you publish it, you're like, this is pretty fucking great. And then just crickets of like no sort of, you know, validation or no feedback from, you know, anywhere. Like what kept you like keep at it? Oh, you when I published all the Amazon ones like when I joined Google or just in general, and like when there is not a lot of feedback on your writings.

You know, I've gone through a lot of dry spells writing one of my blogs is so much work. It really is. I mean, you have no idea. The funny thing is I need to be able to one of my rules is that I have to be able to basically write the entire blog in one city. And if I fail, I will go back and rewrite the whole thing in one city. So it has to. Now during the editing process, they usually expand by, you know, 40 or 50 percent.

But because I'm cutting stuff that then other people who are helping me review their adding stuff, believe it or not, it's not me. Usually. But yeah. I'm such a, I'm just like I feel like everything that I say has to be new, which is stupid. I'm doing it wrong. Like to be really successful, you're supposed to say the same thing over and over again.

That's how, you know, demagogues work, right? But I prefer to like not say the same thing over and over again because you can always go back and look at it. Instead, I try to find new angles or nuances to things and that can take a long time. So I may be able to publish once or twice a year, you know, anything serious. But what keeps me going is I eventually get pissed off and that's right. Time to.

Right. You know, I'm just, I'm a riled. And that's what I know. I'm going to write that it's going to be good, right? Because I'm heated. Right. But I'm also kind of snarky. So then jokes are going to start flying, right? Because I've seen him. And so that's that's the formula, right? Is you think about it. You let it rattle around. You can let it bake. And then at some point you get yourself worked up into your, you're in LLM generation mode and you with the whole thing out at once.

So if you took an example of let's say one of your recent posts, death of a junior developer, for example, which is an amazing post will link that in the show notes, not that we need to, but we'll sell you and we will take it out. If we had to take that one, like, how what did it look like to write that one? How long did you sit on it? How long did it take to write it completely? And then what does editing look like?

Yeah, I started that one, probably and I started, I started realizing what I wanted to write about maybe in February and it didn't come out to May. So it was a good three months on that one. And I am so glad that I researched this one a little more than usual, like in the sense that I like, I went, I went to a bunch of colleagues that I have a lot of respect for. And found actually that some of them were like vehemently in favor and some of them were just like, nope.

Right. So that was cool. That was cool. Right. Because I was like, oh, I think both of these people are really smart and they disagree on this. So we're on to something. But I mean, like it really made people mad. Like there's some YouTuber, right? You know that like, you know, painstakingly read out all the words very slowly and then got up and half with you and started judging it.

The funny thing was his like followers like got into this heated argument about whether we matter or not as human beings. I mean, that's how fundamental the point is that I touched on. I mean like come on. Seriously, and they're like, no, we matter, which is really funny because I am definitely in that we don't matter.

Right. So that could actually be part of what's the resistance. But I'm basically saying, yeah, the old way of doing stuff, it's gone. Just let it go. Let the LLM do the work for you now. You're staring at a higher level. You're not swimming or driving a boat. And people are like, no, that just means we don't matter. And it's funny to watch people melt down over there. I guess I had my AI midlife crisis just a little before they did.

Like I had mine about a year ago where I just had an existential crisis where it hit me hard. I was stunned. I was like, really? Because I realized that many of the things that I thought that I was that I kind of like my my value, my sense of self worth. My identity was tied up in my ability to do things like memorize long things or generate long things or whatever. It's all just BS now like computers do way better. Right. So like that was tough.

I think a lot of other people haven't gone through that yet. So my blog really touched the nerves. But I went through a lot of editing on that one. We even took votes on the title. We weren't sure if the title was going to be too harsh.

Jean Kim, right, who's who's kindly offered to let me talk at the enterprise technology leadership summit in Vegas in two weeks, which is going to be awesome. He he's the one that contributed like it was chatting with him that I realized this is hitting the whole industry. And that's when I realized, oh wow. Okay. So this is big. And you know what?

Like the funny thing is a lot of people came out people were calling me friends like old friends like retired friends were calling me. I figured out the pattern. It was their kids or graduate and they were getting discouraged by my blog post, right. And they're mad. Right. But the funny thing is like a bunch of other people called to all right. We got calls from some very large, very, very large companies where you'd hear their name and go, oh, that's money.

And they have thousands and thousands of developers and a couple of them came to us and said, yeah, they've been seeing signs of this as well. Right. We got we got confirmation. So there's a huge change of foot. And I was able to kind of like get in just at the right time and show people's nails in it. I think that blog definitely had an impact and a lot of people. In fact, I've forwarded that block to a few of my friends who have their kids or someone they know who's about to enter undergrad.

And they were essentially trying to figure out or at least let's just say until last year, their plan was obviously my kids are going to do computer science or the kid thought they would do computer science. And now when they saw something like this or they just see an ellipse in general, they're like, well, does it still make sense? Should I still go on this route that I thought for the past four years that I should be doing.

So when you get some of these calls, at least for the camp of people who have, let's say, their kids trying to go to school. What do you tell them? Well, I'm never going to discourage someone from going into computer science. I mean, it's an amazing discipline. And it'll give you the foundation. You need to be a great, you know, chop engineer, prompt engineer, you know, doing chat programming.

But I mean, look at look at the you saw the graph in there from indeed from indeed.com, right of software engineering jobs. Basically, they happen jobs come when there's money and there's no money right. Yeah, okay. Right. There was that there was an anomaly. And we talked about the syvegas, but the zero interest rate policy and the trillion dollar stimulus package during coba injected the COVID injected so much money into the economy that everybody had money for engineers.

It's all kinds of weird, weird knock on effects in the industry that I'm talking about, but, but, you know, like there were jobs aplenty. Yeah. And now that that's over and more over, right. Like, yeah, I provided a bit of a bubble, a bit of a boost, but except in coding assistance where we're seeing tremendous gains from AI. Where else is it?

I know that's a silly thing to say, but we know it's right around the corner, but like still investors are asking this question. Yeah. Where's the where's the money? Because it's been a year, year and a half. Anyway, on these lines, so when it comes to publishing posts at let's say when you were at Google, companies that size typically have a policy for what you can write on their company's website.

And publish on your own blog and you say opinions are on the last few blogs that you have published are on source graphs website. And there's a more difference. Like if I'm reading your post versus a post on my else road, all of them are good. Yours is way more entertaining. No offense to anyone else. Is is is there a policy at source graph that let's Steve publish the post when it comes out or they are posts go through the same review policy that others might go through as well.

Well, if I if I just said I'm going to post this post it, they would they would post it. But source graph are my friends and I want to make sure my friends aren't caught by inside it. In fact, we've got one right now that was so. It's a really, really interesting blog. I mean, my God, it's probably one of my most interesting of all time, but it's so sensitive that I basically have to write rewrite about three and a half or four pages out of 12 to be one.

Because we want to make sure that you know, we're sensitive. I want to spend a couple more minutes on the writing part and we can move on to a lot of other topics. You mentioned when you want to when you write a post, you want to write that in almost one sitting at least the major ideas.

Oh, man, I was going to make a joke about you know, getting handcuffs like I was like, okay, you know, where are you getting the handcuffs like to cuff yourself to the chair or is like, do lock yourself in like a room and throw away the keys like, oh, man, so my joke, run. I'm not cool. I'm sorry. Continue please. Other handcuffs and wallet. Let's just ask that first. Sorry.

Many times it happens to me where something gets stuck in my head where unless I do that thing, I don't want to do anything else. Makes my family and happy at times. Is that a similar situation here where you have this block post or you're just so right love that you need to write it down. So unless you do this, everything else takes a back seat.

I kind of work myself into a frenzy if that makes any sense. So like I'll set aside time to be like, I think I'm going to try to do the blog post this week. That I've got I've got I've usually got three or four at any given time that need to be written. Like right now I have two drafts that need to be rewritten and finished right and post like a two more ideas.

And so I'll like I'll pick one that seems promising. I'll go back and read it. See if any of the jokes kind of make me laugh. If I can if I'm into it right and eventually you know, maybe maybe I won't and I'll just go program or whatever. But if I can kind of catch that feel of, oh yeah, I remember now those jerks and then at that point, I'm hooked right and so like when I say one sitting, you know, I mean like the reason I do it in one city.

It's the whole what is it the Edgar Allan Poe quote, you know, where he basically said like a good story has to be readable in one sitting. Yeah, you know, I actually and so if I I type pretty fast. And so if I write it in one sitting, then it's usually winds up being readable in one sitting for most people. And and I find that that's a good. I don't know. It's a good forcing function, right? So that I don't write forever.

And also that you actually write and finish. And one last thing on this bit about just writing and runs. Have you ever experienced any backlash? Well, not people getting pissed off, but getting in your way in terms of you want to do something, work somewhere, convinced people, but something's getting in your way because of some of the things you said. Sure. I mean, yeah, I mean, they're there are going to be a lot of people who in this industry who just don't like me.

And that's going to be pretty normal with if your name is known and there's going to be some percentage of people that don't like you. And that's totally fine. And if it's if it's over my blog, then yeah, they can get, you know, who knows, right? But I mean, like by and large. You know, people are pretty respectful in our industry. And also, I would say that my blogs have done more to open doors for me.

I mean, like, I've kind of like I've kind of developed a thick skin over the hate mail and the angry letters and all that. That was something that bothered me a lot more like 15, 20 years ago, right? Now I realize like you could put out the best video game of all time and get a bunch of hate mail. And I'm like, oh, right? I mean, like I saw that happen with things like tears, the king and whatever said, oh, I get it.

There's just a lot of angry people out there, right? So you don't let that part bother you. But yeah, I don't know. It's, yeah. That's right. I had a follow up for the, you know, I know I'm not in the group being able to vote on the titles of the posts, but I love the AI midlife crisis as the title. Would you be interested in like diving a little bit more into that? Like what? I'm the AI midlife crisis. Yeah, I mean, I guess sure. I mean, yeah.

I mean, I don't think people, I don't think people really, most people piece it together yet, but I remember hearing, I don't know, four years ago that like there were a half a billion users of this chat button China. And this is before LLM's right. This is so this is with good models, but definitely not transformers. And they heard it was quite I heard it's quite addictive. It was right.

You have a billion people addicted to a chat, addicted to a chatbot. That's going to happen here like within a year, right? You know, opening eyes, even warning people, you know, and it's like it's so it's a combination of like I worry very, very, very much about the social consequences. People's bank accounts are going to get drained, right?

Because you're somebody's going to get a phone call a banker is going to get a phone call from somebody claims to be you or it sounds just like you blah blah blah, right? Or you're going to get a phone call from a family member or whatever. And this is terrifying because it could start happening like today. Yeah. You know, so I mean like the midlife crisis has more been like a kind of rolling snowball where first I realized I don't matter. Right. Our brains are just neural nets.

And then I mean, absolutely you can have beliefs beyond this, of course, and can can and do, right? But still our brains aren't really particularly special. We'll be able to enter near them, right? So that's a tough hurdle for a lot of people to get over. And then when you look at what's going to happen to our culture to things that we cherish and hang on to. Even even hardens like progressives, I think are not totally prepared for how much that's going to change.

Maybe I'm I mean, like people call me a doomer though, right? So I don't know. Was there any did you come up with any sort of solution, you know, including buying a motorbike to kind of counter react this midlife crisis? I mean, like just to give you one concrete example. I'm I'm consider myself to be very, very good at arranging music for a guitar, whether it's electric guitar, acoustic guitar, whatever I can play almost anything.

I played I've one point I was playing all 24 of Paganese's Paganini's violin caprisis on a steel string guitar. That kind of thing. Hundreds and hundreds of pages of transcriptions, right? And as soon as LMS came out, I was like, Oh, well, that's done, right? No more that like that's just stupid because a model of the able to do it. You just give them a bunch of examples and be like, give me a good transcription, you get a range.

It's just a little bit of prompting. Like that skill that I developed over all those years is just stupid now. Right? And it's like, I so I don't do it at all. I threw all my right through throw it all the I'd love to train a model. I'd like I'd love to keep doing it, you know, like you can use prompts and you can kind of make it my own music or arrangement or whatever, but ultimately all the clever work of, you know, mapping fingerings and voices and doing inversions and stuff like that.

The model's going to do that, right? So just basically a lot of things that are kind of pattern matching or kind of like almost tedious that I got really good at in my life. I wouldn't even do now like why would you do it? You guys look really depressed. So maybe I should talk about. I think so. We are expecting a baby in a couple months and look and go. Thank you.

Some of these on similar lines when I think about AI and the lens and all the pieces you said that we have come to cherish as part of our culture, I do think about that. And then I find myself without any answers. And I put my hands up and there's like, what am I going to do? Let's just roll with it. That's right. That's right. Roll with it. Big wave coming in. Surf it.

So eventually just at least in my head right now the way I am making peace with it is human somehow figure out a way to evolve with circumstances changing. This is one of the circumstances, maybe a bigger one than others. And where some of the skills matter till now, which may not be relevant anymore, like memorizing stuff. For example, like this is I don't know about many people growing up, but at least back in India when we were going growing up.

The kid who remembered names, the kid who remembered numbers, the kid who could do some of these things like, oh, I can look at that cars number plate and know exactly whose car that is. Like these kind of stupid games used to matter in a way, but at this point, like memory, do you care? Yeah, make a general. I can try to still still recite pie to 50 decimal places because I wouldn't get me on sixth grade.

Exactly. So like these kind of things may not matter anymore, but I guess there will be different things that will matter similar to not having to worry about swimming like you said, but writing the board and what that looks like. That's right. We're leveling up. We're leveling up. That's what's happening. Like I said before, we were swimming and now we're driving.

So you're not just driving the boat here, building one to talking about building air products sometime back. So you joined source graph, I think towards the end of 2022, if I remember correctly, you joined as head of engineering and then you transitioned to being an individual contributor working on Cody, which is source graphs AI code assistant.

Can you tell us a little more about that transition? Sure. It's not the first time I've done transitions like that in my career. I worked my way up at Amazon at one point to a senior manager and then just handed it off to one of my directs because she was enjoying it more than I was, right? There's a lot of people side of management that a lot of people thrive on and they love it. They want to mentor people and they want to guide their careers. For me, I've done that for a long time.

And for me, it's these days, it's more about what can I build and how fast can I build it? Right. And I would have been fine. I would have been fine to stay in that head of engineering role, except that AI was so new and so different.

And I was doing so much non engineering stuff in my role that it was, I was starting to feel like I was never going to catch up. I started getting worried, right? Because like, for example, I never could have written death of a junior developer that post had I not made that transition and been coding like the whole first half of this year is an actual engineer like helping us get, you know, get the product out.

And I'm so happy I did, right? And now I can see I can see where it's going in a way that I never would have been able to see that in a management leadership role. Now, will I go back into leadership someday? That baby possibly, right?

But I mean, with the AI stuff, because like you said, we're building the boat, it is the time of my life. And I'm not, I'm not exaggerating the slightest. I thought about this. I've had some pretty darn good times in my life that this is it. This is the best in terms of just to create my professional career.

I have it a lot of fun, right? Because like, there's a, I blogged about this a long, long, long time ago. I think it was called dreaming and browser swampers. But it was basically like I was talking about this feedback loop to get into when you're building IDE or anything that where as you build it, you're making yourself faster.

And it's so addictive. Yeah. And that's what we're doing it. We're doing it with Cody now. And and everything that we do is wiring up this deeper and deeper integration with the LLM. And there's nothing like it, right? You're just like you're running. It's like you're, you're, you're in a personal hovercraft or something.

And and you wonder where, where's it going to, where's it going to lead? Where's it going to go? Because not only are the engineering, the tools and the protocols and stuff getting better, the models themselves are also getting way, way better, really fast because the competition. And so, you know, probably this time, yet next year, it'll be commonplace to have models and workflows that span big graphs.

Where you've got to, you know, you've got to get, you know, an application built and the tests and the CI CD and you got to get it deployed and maybe even look at the logs. And we're finally at a point where you can expect a lot of that stuff to get done for you in a couple of passes with, right, with the tools that we'll have, you know, with a couple years after that. I mean, then this is what I'm saying. What happens to all the, I mean, anybody who's doing old style programming.

And we're going to chop engineer. So, you gotta do a D chart. There's a lot of things I want to dive a little deeper into before we go there. You said, this building code is like you having the time of your life. What's different about building a products than it is to build something else and you've built a lot of things in your career. I mean, like it's, it's the look, it's like driving like the number of red lights to hit has a big impact on your sort of perception. The right.

And like when I was programming in the 90s, if you got stuck on something, you could expect that it might take you days to weeks to get it figured out because you'd have to go look and book or try things or work with colleagues or whatever. Right. And so you just like, it was like we were in slow motion and it's gradually beginning faster. Google search made it a lot easier to find stuff and stack overflow made it like even easier.

Like you start asking people stuff and getting answers each one of those was a big accelerator. Yeah. And each one of them made programming more fun. Absolutely. I mean, if you went through any of those transformations and and cloud computing, I think made programming more fun. I mean, it's a pain, but there was a lot more fun than running right in my own computer in a data center in downtown Seattle.

And when my game would go down, sometimes it was because the computer like wouldn't reboot and I had to like go in the middle of night and get a key card and reboot a server on the third controller, this horrible hot building. I mean, cloud computing was a real, real level up for us, right. And this is no different, right. This is like, wow, this is a huge level up. It's a toy. It's a fun toy. It's like, you can make it do things.

And you're like, I've been doing this for six months now. I still didn't know it could do these things. You know. So, so that's kind of like where I'm at right now. That's why that's why it's like time my life. It's like because programmers we like to build things that people use. And that gives us that don't mean cycle. Right. And so if you're speaking that up, it's getting more addicted.

Speaking of that history, so you were actually retired before coming back to a lot of socials. And you mentioned before you've transitioned from like, management to IC. I'm curious like, what were some like, what are some of the skills that are really useful in just picking up like a completely new domain, right.

All this stuff didn't exist, right. Like the last time you were like, you know, back to coding. So then like, yeah, like, how do you go about when diving into like a completely new domain? Like, what do you do? You know, there's an interesting question behind your question. I feel like lurking there, right. Which is the police help me out there. If that question is so well, I mean, like it's related to something we talked about earlier, right.

Which is if I'm going into software engineering, should I be going into software engineering, right. So, I guess when you just make absolutely sure. Stay your question one more time. Make sure I'm going to answer the right one. Sorry. When you go into a new domain like building AI product now with Cody, like, how do you go about picking up the new skills that are relevant to building it?

Yeah. When AI came out, a lot of the folks, you know, that I worked with didn't want to have anything to do with it. Like, they just didn't, they weren't interested. I think source graph or actual graph both both. And they just, you know, because maybe maybe there's a misconception they didn't know that it wasn't like right a bunch of pie torch anymore.

It was an engineering evolve now. But for whatever reason, yeah, a lot of people feel like the entry into this new domain is actually so steep that they're scared of it. They're worried or they're just putting it off. And so like seriously, I mean, like getting into a new domain, I mean, the best thing you can do is be young. I'll tell you that right now. It's harder for me.

And so like when I got to learn new programming language, I can't just like read the book in two hours and like know the programming language, like I could 20 years ago. So that helps like just spend the time while you're, you know, while you're young, learning as much as you can. But, but seriously, I mean, like don't be afraid of me domains. I mean, we're all kind of reinventing our jobs kind of all the time anyway.

And this is, this is, this is necessitating that we all read into our jobs as prompt engineers. Yeah. Even if you're doing regular programming, you're going to, I mean, if you're going to have to do a fair amount of prompt engineering from now on. So how do you learn it? Well, I mean, gosh, there's so many more resources now than they're used to be, right? But you know what I would do?

I'd use chop to figure it out. I'd go into, I'd go into Emax or Chad's, you could do whatever or go into Cody when I'd start asking there. Because why bother with Google searches anymore? You can get summaries from AI, right? AI knows how to even like present it to you at your level. You can say, I'm a complete newbie. You are actually more similar domain compared to that. So like the way, so the answer to your question is different than it was a year ago.

How you ramp up into new domain is you go ask the hell I'm about it. That's my opinion. It'll make you a better engineer, right? Because there's a lot of, there's a lot of nuance. There's a lot of, there's a lot of trips. You know what? There's a lot of people out there today who still possibly listening to this podcast. Yes, you who think that like chop is not a thing because they asked the LLM to write some tests for them and the test for wrong and therefore,

which is just like saying, you know, Google, I did a Google search and it didn't come up with the right answer. So Google's dumb, right? I'll tell you who's dumb there. It's not Google. And it's because people don't realize that the fundamental, I mean, you have to still be patient and persistent and making your own when you're doing chop.

So you have multiple questions around trips with the LLM, right? People aren't doing that. They're not doing that. But that's, I mean, fundamental, I mean, prompt engineering and and and. I'm not a scared. If you go back and look at my, if you seriously, you go back and look at my chat history and Gemini and Claude and in chat, you'll see that I am ramping up on new domains right there in place as I'm coding.

Having used web sockets in 12 years, let's go through the fundamentals again, right? Oh, man, I haven't used Windows in 15 years, walk me through how I like install it. You know, I mean, literally anything you want to know, you literally asked the LLM while you're coding. And so, so everything that you're doing becomes this like, you know, it's an a blender, it's blurred together, you know, you're working and doing all the things that you need with the assistance of your coding assistance, you know, your LM.

And you're cranking out a bunch of code that you're not the one writing list of it. Do you guys do this? There's like a bit of a cycle for me as well, where at some point I was kind of like, you know, the meme of like, can't tell if the LLM really bad or I'm just really bad at prompting the LLM. It usually doesn't matter. So you took me a while, I think, to learn like, oh yeah, I need to, you know, really be strategic.

And then I think to a point like in terms of reducing red lights, one thing I really realized was like, oh, if my prompt is bad, I can just ask the LLM to improve the prompt. I think that was like the biggest unlock where I was like, oh shit, this seems like way better than what I would have asked it. So yeah, it's been, it's been pretty big. Yeah, I read papers on prompt engineering, you know, all the time, and that's a new one, right, is a rephrase and expand.

Tell it and you get a better prompt. But what I found, honestly, truthfully, okay, is that I get good results without resorting to special prompt engineering techniques. The models are good enough now that just a regular conversation with the right context gives it everything that it needs. And the context arguably is more important than the prompt.

Because it's going to get her, I mean, like you want prompt engineering, like you want it to be really, really, really good if it's doing some very specific task for you. Of course, you want the prompt to be really good. Like if it's the right unit test prompt, we're going to put a lot of work into that. And then you can give in random problem that you're doing during the day, it's a throw away prompt. So you don't really need to put that much effort into it.

And instead what you're going to do is you just got to recognize that some of the time it's going to just make stuff up. And you'd be like, no, no, no, that's made up. And they're like, oh, apologies for the confusion or whatever they tell it to say these days, right. But you just, you just, you just steamroll past that. You're like, no, that didn't work.

Until the thing get does the thing you want. And if it starts to go into that circle, you mentioned this. Sometimes it'll start giving you the run around and realize the model cannot make any forward progress. Just treat it like a chess match, right. If you get the same position like three times and still go take the whole conversation and take it to the next LLM over, right.

Just go on a circle and say this LLM's having a lot of trouble. And the other LLM will be like, they sure are. Here's your problem. And they'll like tell it out to you, right. It's so cool. It's kind of like the magic property of second hashing, right. Second secondary hashing, second hash function almost never, ever has a collision.

Just because you know, because the odds and I just just awesome. You can go to a second LLM and pretty much always get the answer that you were missing the first time. Oh, never. You go to third LLM. Interesting. I need to try that. Oh, very cool. Dude, this is a chop technique. There's a bunch of it. I'm starting to do some videos on it and whatever. It's like there's an art to it. Should do a post on chop techniques, I think that'll be super helpful for a lot of people out there.

That is one of my completed drafts that needs to be rewritten to be funny here. But yeah, it's ready to go. Awesome. So one thing that you mentioned, there's a lot more engineering to this now to building air products. I was curious. Generally, when you talk about a lot of people who are not building air products or software engineers, they also have this formal of like, hey, kind of missing out.

But you know, I'm not a machine learning engineer and they may not have the right opportunity either at their own job or they can't find one these days because it's generally harder. So you put things in context, building air products at this point where you're not necessarily training in LLM or so, but let's say you building something like Cody. How much engineering work is that versus how much research she work is that?

You get to decide where to turn that dial. I mean, it's it's it is a dial and I mean, look, and why, why is that the case? It's because there is sort of like infinite low hanging fruit in the space right now.

I mean, really low hanging fruit like we're like even just I think a two weeks ago I was in crack on Poland for we did a hackathon, a year of hackathon right. And I realized what one of our one of our teammates we were chatting and we realized that nobody has gone and fixed our chunking for embeddings and it's been a year and we still don't.

Like an embedding will literally like the chunk will be from the middle of one function to the middle of another function, which is just totally useless for the LLM right. There's a big low low low hanging fruit right. Of course, we fixed that. Yes, sorry, you know, but.

And we have lots of other things besides embeddings that get us indexes, but you get the idea right it's just like the road is so clearly set out for you. It's so obvious what to do. There's too much to do. And so you got to decide, okay, all right, where how are we going to allocate our resources.

This is a strategic sort of battle early on in the game like last year, I thought we were going to like really double down hard on, you know, on the AI side, you know, we've got a great AI team and they're doing great stuff and fine tuning and eliminating bias from models and they've got a they've got a trained context ranker because what we found ultimately was the biggest bang for the buck you get in the coding assistant space.

And it's aside from the LLM itself, okay, the biggest bang for the buck you're going to get is. It's not the fine tuning models right it's it's and it's not the prompt engineering. It's your context. Okay, it's making because because you get a lot of tokens. You don't get anywhere near enough, but you get enough to put a lot of context in there right.

And if you're doing raw chop without a coding assistant, if you can just shovel screenshots and paste in logs and stuff right that context means everything because every time you talk to the LLM and I think people have trouble getting their heads around this every time you're calling a phone like you know around the world and the first instance you never met you before.

Hey, I'm a programmer and you got to explain your problem to them and give them all the context that they can give you the answer every single time. Yeah, which is why coding assistants are actually so useful in this game. So you had this block post cheating is all you need. I think it came out some time last year again. One of the other amazing block posts will link it in the show notes and recommend people check it out.

You had this very simplified diagram of how code works, which could be a diagram for any coding assistant for that matter. It's like you have a human developer on one side, you have the assistant in the middle and then you have an LLM and you have some source that is giving you embeddings, search, etc. And you tie all of this to track. How much of that has changed since then? How much of this change? Okay, so like in other words, yeah, go ahead.

The right way, right, because the right context put into the, which is to use basically local local searches. Now, it's definitely expanded in terms of like the techniques and the approaches and things that go under rag. And the customer of ours introduced me to a really, really interesting new kind of source code index, for example, that I think we can bring in.

I got to go right it up. It's just mind-bending. And so I mean, like people are, I mean, people are innovating like crazy in this, in this space. Yeah, so it's fundamentally hasn't changed. So that's the question, well, if they're all doing the same thing, like how is your coding assistant differentiated from the others, right? And to some extent, you know, we thought we were going to maybe differentiate on like local AI, smaller models, you know, like we do with our content's ranking.

So to the extent that you can do that with rag, yeah, we can differentiate and we are. But but for the most part, the effect of the actual foundation models in each generation is so overwhelming compared to the other stuff that you're doing. That you pretty much just got to decide like, okay, how am I going to ride this chain to maximize value for ultimately myself, I'm building this for me, right? For programmers. And yeah, what we've decided is that chat is the future.

It's chat is where it's at. Everybody's been so vocal, so narrowly on the completions. And I never understood it. Completion right. Yeah, and car, completion acceptance rate, right? It's just been the industry's an obsessed with it. When completions are like, they're okay. I mean, they're kind of gimmicky, right? I mean, your code along all of a sudden goes, well, and it finishes it for you know, it's pretty clever.

But that's not, I mean, like most of what you're doing, most of the time you're not doing code that's conducive to completions or in a way that they would help. Whereas if you could plus you, it's hard to guide the completions, you get into this millibere like putting comments into the code and helping it picks up on the comments and right.

So like there's clearly what you need is sort of like prompted completion, right, which which we have with in line edits, which is a really cool feature where you just type make, make these changes to the code and it'll do it. Sort of like a directed completion and we're even working on some modalities kind of between those two like super completions, right, things like that. But again, completion, I do.

They're almost a niche compared to how often you use chat. Once you realize that chat covers all phases of programming, right, from the generation of the code to the fixing of the code and the diagnosis and the troubleshooting and every part of the life cycle. Chat can help you with it and accelerate you and at that point, completions just fade into this gimmicky background.

And I think most people haven't made that transition yet, but they will over the next six months and they're going to realize, oh, wow, the coding assistance that focused on chat are the ones that are going to help me the most. And that's what you have. So you mentioned this in your blog post tool like offered chat, GPD photo and Clark 3.5 sonnet. We saw this huge jump in the performance of coding assistance again, like just the understanding of the code base.

You could ask them to do complex things like at times I've asked it to refactor certain methods and before this you could try. I don't know how many loops with how many other limbs and they would just not make progress, but now it's been very different. Not exactly how amazing either my prompts are really bad or the limits are in that great one of the two, but they're not exactly where you want it to be.

But in terms of the improvements that we're seeing in some aspects that you mentioned, like focusing more on chat than autocompletion. There is an aspect of your foundational capabilities are just improving by so much and there's a step function improvement with every new foundation model. And on top of that, now you're differentiating with features because models in a way are commoditized to an extent. Everyone has access to it over an API.

So then how do you leverage that model becomes a differentiator? So it goes back to regular product building fundamentals. How easy it is to use. Are you providing the right things at the right time to the user when they need it? So a lot of the improvements that we see today, apart from the models, what else do you think plays it all right now? And how do you envision this would be better? I'm really glad you asked that. I almost could have prompted you to ask this.

So I think that the piece that I'm most interested in because obviously the models are going to continue getting better and better. And the tools are all going to continue sort of like copying each other and converging tools will do right. So the piece that I think that's really critical that's missing that everybody needs really desperately and asking us for and we're making good progress is the platform. Yeah, you know I was going to be a platform at some point.

And so to me, the real value and Cody is that we are building it as a platform. I mean, the internal the guts are very platform me. So we've got a command line tool you can use to integrate with your workflows and we've got sort of like a set of curated platform APIs for connecting clients, but also for connecting your own context. Protocol and like there's so many surfaces right because you've got to bring in all your personal context, your wiki's, your private code, whatever.

You may have your own personal model. So Cody also becomes a clearing house for you to like mix and match what you want like because we got. We got users and customers that really love one particular model or another for whatever reason and a lot of coding assistance. They're pretty much a bundled deal like you don't have to think and it's great. They give you a model and that's the model you get. And if for people who just want to like just start going they're great. They're fine.

But if you want a little bit of control over it. If you're the kind of person that builds your own PC, you know, you want to like build your own tools. If you're the kind of person that has a bin directory with scripts in it and you like to you know, like tinker with your tools to make yourself even faster. Then I think you're really going to like Cody because we're building it for that kind of user.

It's actually a necessity because enterprises have such whack requirements, right? So you kind of have to build a platform. What are some of the use cases do you see that could be built on top of the platform that are probably not as obvious to many people? Well, I mean, that's a good question. I mean, like there's some of them are kind of feel like science fiction, but I'm sure we're going to get there. Oh sure. All of this was science action three years ago.

Right. Yeah, right. I think the Holy Grail is batch. Right. You got to be able to basically like run the L and you got to be able to drag the LMS knows through all of your code. Yes. And right. So what we do now is we do it in entry and ingress ingress, right? You know, we're like, OK, a PR just came in and have the I'll look at it, you know, and see if you do the linting or whatever else. And it's great.

So that when codes and you stopped the bleeding basically like codes that are your code, but what about all that existing code, right? That's that's the Holy Grail. You know, our product will have a really cool batch engine. It's similar to one that Google built internally that Google really should have open source. It's called Rosie, but they didn't because they never do. And it's pretty cool to be able to like like know everybody that you're about to break. Right.

Right. At least in the open source world and be able to like even go and re factor them and send them a full request saying I'm about to break you, but if you accept this PR, you'll see that it fixes you. That's nuts. Okay. You know, we do that. Now imagine being able to do that with code. Imagine being able to do that with AI, right? Because now I mean, it can do.

I mean, these things can be remarkable thing. You can basically give it high level instructions for a file and it'll go ahead and port it or whatever it is you need to do. So yeah, that's the Holy Grail. You know, when do we get there? I don't know. I don't know. There's so much to full. It's like I could you got to pick where to spend where does where to turn that dial.

We've got people chasing the AI side because that's important. We've got people in the context. We've got people making the clients really, really slick and good. We've got people working on the mixing and matching of services and models in the back end. We're spread then everybody spread then and we're just we just hope that we're making choices that are going to make the most people the happiest. That's kind of what's great is right now.

So one of the aspects you mentioned like the Holy Grail of being the batch, like today, if you think about. Elements or many of these coding assistants, they're essentially assistants and I think you said it well they're safer in the hands of a senior engineer. I was probably going to go where you just already prompted it is like you don't know when it comes but the thing that I was thinking about is it would be really nice to have not a coding assistant but a coding partner of sorts where.

It works on your instructions. But you trust its output as if you were off a senior engineer so the part about that you said on the legacy code or the existing code base and I think that's where a lot of. Well, Eli's two and I would say majority of the organizations struggle there. We know there's a lot of code which is dead in many cases. No one goes and cleans it up.

You want to do a big refactor and just updating libraries of all the clients is extremely painful in a large code base being able to do such operations which require. Chain of thought breaking tasks down into pieces but then executing or hours on days sometimes. Is this a direction we're coding assistants would eventually go on your perspective. I mean, dude, your whole your whole question was tainted with the t word you said trust.

You know trust right and so when that happens because it will happen somebody's going to have to go through every single step of it and validator right we're going to be very was it linen trusted verify. Or is it pre-shev I don't know. Right. Don't trust and verify. Very good. So so yeah that verification you know it happens in the small when you're doing chop you validated yourself your ID can provide you a very quick feedback loop because you jump it in and it goes that doesn't compile that

that you can't that doesn't exist and you're like right. But you know there's larger feedback we have larger and larger ones that go all about to production and then come all the way back and yes and logs or whatever. And those things they're going to get looked at by models and models that are trying to do those kinds of things workflows and it's going to be just like chop is right now which I wish more of you were doing it and I'm doing some videos but basically.

It is an assistant and you're still doing the work and you need to make sure you're the look if you build guitars or cars or whatever in your garage right you can bring in assistance. You can train them up that ultimately it's your guitar it's your car right you got to make sure that the right things happen and assistance drop stuff right so I would never get out of that mind so I know you want to part I know you want someone you can trust.

You're not going to get it. Okay settle for an assistant for now. Oh, hopefully keep my job in a way.

It is job security absolutely and you know what I mean despite the doom and gloom nature of that obviously I had a provocative title in my blog but I mean like you can do that validation yourself as a junior engineer right because it'll spit out a bunch of stuff and it seems plausible but you can use other tools like other LLMs or compilers or whatever to go in and do the valid tests to do the validation you need to see.

Yeah, it did work now you're going to miss things you know probably like you know it's inefficient or whatever something that it did that it didn't need to right if you're a junior engineer but if it meets the basic spec right so I mean I don't think I don't think it's all doom and gloom for

junior engineers at all they're just going to have to get really really good at validating and you're saying eventually you would have models which would also help you validate so keep feeding yeah basically you pipe from one into the other. I mean as soon as you start doing a set of validations that's the same you know like every time then you need to train them you need to embed that whole thing right train a model the most how to do that.

Yeah so chop is definitely a useful skill and not everyone is doing it and even the people who are some are better than others do you evaluate for this an interview is anymore or at all not anymore oh my goodness no we haven't started asking you this is too new right that's that's interesting.

And you know I mean we fundamentally we would still be we would still be asking people that would be like asking people can you type like we might ask people some prompt engineering questions like just to see if they have any common sense about it I guess but mostly yeah you're always still focus on fundamentals right computer science timeless stuff ideally in interviews.

In terms of hiring junior developers versus senior developers this is at least a shift that I see are playing then and I think it's probably true across the industry right now where you all that are fewer positions open and the ones that are are for experienced engineers and when I say experienced I like how you described it where a person who has an idea of what needs to be done has a framework of how they would do it and they can control the element this case.

Do you see that affecting productivity of organizations that you speak with. You mean the shift towards your yeah I mean sure yeah they're getting more productive but you know what what that's short term gains right what what long term consequences are we're going to reap from not hiring enough junior engineers.

We're trying to get it right. I mean like have you ever been in a company where everybody there was you know like retirement age I've you know been around companies like that you know it happens and and it's really really difficult it's at that point you've got Japan's problem right you know the it's some point it's it's kind of not reversible.

We got to have you got to have young people entering into the workforce and learning the stuff so continue doubling down on that and hiring higher junior people and then just I don't know I think I think we're just going to have to figure out like a new workflow but there's been a lot of people suggesting that maybe the junior folks can crank out a lot of work and see your soul kind of like more do that.

You that's a little bit better but I haven't seen any fall in the that most of the time we've been really close and sorry they can and really planning for the future. It's just such a big race. Yeah it's a huge problem. And there is this thing called Devon there's a company called cognition or AI I'm curious if you have seen it what's your take on it.

I didn't see the demo I heard people talking about it but I don't think it comes down to the T word again trust right I ain't going to trust agents until they're trustworthy not before not a job before them which means I'm not going to meet an early adopter of them unless they have a mode that allows me to use the agent as an assistant or it does part of the job and then I tell it where it went wrong with chat and then it continues the job if they can do that.

But human in the loop driving it along and steering it then I think they'll land a lot faster and people will trust them a lot faster but my suspicion is that people are they're chasing investor dollars and so they're trying to do something that's hitting home runs and they're not going to hit it not not next year 18 months I don't think I've been wrong before.

So similar to coding assistance there is another aspect that engineers care about which is understanding how the existing code works and you mentioned batch in case of Korean source graph.

Is it a direction which you're pursuing where one could ask questions off the court to understand how it works and I recently saw a demo I forget the name but they had this thing called which would basically generate a wiki out of open source code base so I could just go and say how does react work or how does Kubernetes work or aspects of it.

Yeah, this is it this is an idea this is the idea I mentioned earlier in the talk that one of our customers pitched to us and they actually got it from Google via another customer of Google's so this is spreading what you're talking about is I view look it was hard for me to get my head around it but I view it as a new kind of source code index that's complementary to a search index or a graph right that I can probably would build or embeddings any sort of index right they all have different you know strengths.

There is a new kind of index emerging which is generated by all M's and it's a semantic index and I think they're going to have incredible they have incredible potential I know some people who are building them now we will be looking at them for sure I'm excited about that direction but that is that is smoking hot new that would be amazing by the way like.

Wiki's and dogs get out of date this would be pretty neat we spent all this customer spent a whole afternoon educating me about it why because they didn't want to build it right they want me to build it which kind of makes sense right because it's kind of red so so yeah I'm off I'm this is I'm getting chills this this is this is a new direction right it's so weird you're basically building this document store the semantic index that doesn't have to be human readable it just has all the semantics that are unpackable by the.

And then you have business owners who can't look at the code necessarily when they can be going into legacy code basis of 20 million lines of co ball or whatever you know your company doesn't even understand anymore but you're dependent on it and they can go in and actually start making like informed not just questions but potentially even start the company can start doing refactorings now that are.

Like tractable when they weren't tractable before and trust me I know a lot of you out there that have intractable migration problems trying to get your monolith split up or whatever like this is going to be a game changer for that.

Teenie tiny benefit of this would be developers getting less things from their managers about how does this thing work they could just go ask it is there's one of many benefits that this this structure has and I'm just it was weird it was it's like a discovery you start realizing this is this is this is an inevitability like people are all going to figure this out you know is use the element explain all the coding great detail and then now it you know how it works in ways that you just couldn't before.

Now I mean like how to build that index is still a wide open question what's the format of the index what's the granularity do you run it at the function level the file level the module level at the graph level you know we'll see right but it's really cool there are a lot of people are innovating on this right now and a lot of developments we see right now are on the left side of the developer cycle in a way which is court assistance to help them develop code or this summarization in a way that is going to be a good example.

So the summarization in a way or the code understanding semantics of the system and then helping you understand how it works. There is a center world of things on the right side which is when system is running in production and you can't reason about a system specially in the service oriented architecture where you have like 2000 things talking to each other at any time with configurations in some way I'll secret somewhere else and they are just connected via this network.

Do you see any movements on that side which would help one understand the production environment better. I mean I'm sure there's like startups and stuff to and that but we view that as a subset of the context problem like people companies came to us and immediately they're like oh Cody can index our stuff and then we can ask questions about it.

Then can we throw our wiki in there and we're like yeah and they're like welcome we throw our issue tracker in there and we're like yeah and all of a sudden they want everything logs and whatever right and so that's why I said we've building a platform and with an open context that allows you very simply to to to to to basically like allow other context to get put into your workflow doesn't have to be a query can be a CICD workflow that you're building right.

That's the platform lecture I mean it's gonna be huge. Okay completely missed then what you meant by platform because this is fascinating so essentially Cody's while the way you interact with it is through the coding assistant interface or the chart interfacing a IDE but the context it has is beyond just the code in the repo.

It is it is it's it's it's it's negotiating you can at mention URLs and stuff so you can slip down you know now and Cody right you know it's actually quite it's funny it's not the way we thought it was going to go and our AI people are a little bit indignant just a little bit right because they're like well technically you shouldn't have to figure this out the model should be able to figure this out and I'm like well there's your operative word should all right so why don't we just go ahead and work with reality which is that we're going to work with you.

We need to be able to swing it around like gunslingers I need slice it dice it prune it go back and rewrite it at a history your conversation becomes the dynamic context right that that's going to help you solve your problem whatever it is and so yeah you need full editing ability to go back and rewrite that thing you know and we we decided to go down that path with our AI people kind of running after us to try to you know model it right and that's just

just tremendous flexibility that's what I mean by platform is it's it's an it we can integrate in on many different surfaces we can integrate on the model the back end we can integrate your context we can integrate like at the repo level or like with your particular tools that's a lot of work right it's like three times much work to build that way but

right because that that then then people can build stuff like what we just talked about like that that that magical document store you know and and integrated into your tools and this is why I'm telling people that the growth is exponential that it's only been what 18 months since GPT 35 came out you know or whatever the 3GB4 and and and that's kind of garbage compared to what we have today the next 18 months isn't going to be you know

like as much as happened before it's going to be significantly more than happened in the last 18 months because we're growing exponentially so it's a little bit dizzying actually now is it it's like now do you see why I'm saying this time with seriously I think everybody should jump on this.

So you mentioned earlier that while building code and also using code a surprise to you in ways we're adapted in a way you're not dissipated to act that way are there any specific instances that you can recall where it's surprised you.

Yeah I mean just sometimes it just has no you feel like it has no business knowing what it what it figured out right and this this happens a lot especially because Cody does a lot of magical context fetching for you can see it you can open it up and look at which ones it picked right it'll it'll rank them it pulls some out of the embed it'll embed your query and see if it comes up with some relevant stuff that way and it'll do a search and check our graph and so on.

And those sometimes that context is completely irrelevant we need to work on it and sometimes it's so relevant that even with like it's incomplete and you look at it you like I'm actually even like a senior engineer probably shouldn't have been

able to figure that out and yet here the all and generated the right code maybe it's just really good at guessing right but I mean in the end you know if it's doing the right thing you know I'm still regularly blown away by these things so I don't really have products this question might be a dumb question but I was just trying to visualize in my head in terms of how those things functions makes visualizing things makes it easier for me to understand stuff.

So you have this model in one place which is the code of them call it GP40 car 3.5 son or something else. You have Cody in the front and then context you said is where the magic happens or that's the differentiator. Yeah. Cody can pull context from different places. Now context here tell us a little more about what that looks like so you in I'm just imagining in one case you have all of your source code in one case you have your wikis.

And this is something I see when I just search on source graph by the way source graph search is outstanding that's my go to place to search code I recommend you to most of my teammates I work with. So in that case it's say something like search based it'll actually show that tag. But in terms of fetching this context can you elaborate a little more on what that looks like.

Yeah I mean it's not a one size fits all problem right it's it's a it's domain specific so if you're generating unit tests your context is going to be. So if you're a company's testing style guide whatever framework classes you're using maybe some examples of tests that are particularly representative of how you want them to be written in your organization.

So we actually allow for things like this by allowing you to put in pinned context and you know having administrators actually point in context that everybody's going to be using all the time for specific use cases. Custom custom prompts basically and build up prompt library so that you know because the prompt can help bring in some of the context as well or interpret it.

And then it's going to be a completely different set of context for the use case that you brought up in your earlier question about how do you bug in production. Right and the answer is the LLM can use tools and they can read and interpret the output of tools.

And so you know our context there is can we get a what information can we get out of prod that we can get in front of the LLM and since they're multi modal maybe that's as desperate as doing a screenshot but hey right that may work too so context can be can be anything context is context it's literally like it's it is defining the problem you know

that's the most fundamental thing so in the rag part of this equation this is the triangle part and how much of the classical search techniques apply to this part what is this part also way to different nowadays.

It's very similar actually like if you think about Google search right it's not one back and it's not one size fits all just like this is the I domain right if you're searching for baseball cards there's going to be a very specific set of you know things in your query that are very different from if you're searching for you know a pet store.

And so there's going to be a bunch of back ends you know hundreds read thousands of models that all all report back on what they think the relevant answer your context query is search results say. And then somebody has to go and arbitrate right those those those answers and multiplex them and that's that's where some of our key differentiation is is we're not just a context fetching engine where context assembly engine.

And there's a lot of there's an art to that as well right because it's it's a been back in problem. So for someone who has some knowledge or experience with information retrieval systems or search in general I can see them transitioning into this space relatively easily what are some of the other transferable skills in terms of software engineering to just building a products.

Typing right because you're doing chatter in a programming that means you got to like chat and and so if you're really a slow typist you're going to be fresh you're going to be the bottleneck right. Although it would be great you know if we could I think I think this is it's finally time for you know speech to text to to really shine we need to get it wide up so that you can talk and type and whatever at the same time.

I think that was slightly tongue in cheek I get it but you know but yeah in terms of software engineering I would say look the skills are the same but all for my entire career the low level skills have been becoming obsolete one after another as we level up okay so when I started you know in college like we were literally like using put pixel like to put a colored pixel on the screen and we would use the R on Z buffering and whatever to build three.

Graphics and by the time I graduated from college it had already gotten to animation courses it was accelerating that fast right and at some point we didn't need to know how to render polygons into brezen hams algorithm and right and because it's down in the hardware and this steady March of pushing software down into the hardware layer over and over again as met we've been focusing on higher and higher and higher level things there still computer science

and there's still things you still have to understand algorithms and data structures and big O complexity and all that you have to right but the specifics of technologies that you know are staples like I don't know do you need to know relational databases anymore maybe not for long if lm is no everything about the additional databases and all you need to do is tell them what you need right you know you need somebody to go in and fix it one person on staff again this is the problem right but yeah

so we're coming towards the end of our conversation and so in one of your YouTube posts you mentioned that having something to truly push up a mountain is fulfilling we like struggle and struggle is necessary for life you've had a very successful career or this period what have been some of the struggles that you are the proud most proud of am I the most proud of or most which have been most fulfilling in a way

yeah I mean like I guess one that I would love to do is almost like a PSA is your mental health is really really an important resource that I think that maybe like so I'm Gen X and we don't talk about that

that's a tough ever okay it's like you know yeah just see the football game yeah that's as close as we ever get to talking about mental health and sharing in my generation all right but I learned the hard way like wrestling with the innovators dilemma you know head on very very difficult problem you know my mental health really really took a hit you know and I learned how important it is and that was a huge struggle a huge struggle right to get better you know what I mean

like get back to my jolly old self you know it was not an easy thing and so I just want I want people especially the older programmers you know to recognize that yeah stuff can get so hard that you need to go get help so there's a struggle I'm proud of telling you about is that count that that totally counts the totally counts Steve this has been an amazing conversation

we highly encourage people to check out all of your blog posts we'll link to them in the show notes and we learned a lot through this conversation and I'm sure all of our listeners will to not only learn they'll find it entertaining so thank you so much again for joining the show

well thank you so much for having me thank you Steve awesome all right guys hey thank you so much for listening to the show you can subscribe wherever you get your podcasts and learn more about us at software missadventures.com you can also write to us at hello at software missadventures.com we would love to hear from you until next time take care