Well, hello, hello, and welcome back for another wonderful episode of Jobscript Jabber. On today's show, we have first our guest Connor books Bronston. Oh, I'm sorry I spelled your name wrong. I spelled with two wins. It happens, man, We'll fix it and post. All good. Not something I'll worry about too much, given that both of my names you could assume I'm spelling them a little wrong, so peple usually assume double ends
the first name Bronze sun flast name which just happened. Don't worry about all right, cool, and then you give us the thirty second who we are and yeah, pardon me here? So I'm the coast the Devin Rupted podcast. We talk with engineering leaders every week like AJ and Steve and leaders at you know, Slack, et cetera, about topics like debex. Jen Ai.
I also work at Linear B and one thing we've really done is dived into data reports around house software engineering and software to the life cycle works.
So I'm excited to share some insights early insights actually from our Jena I report what's out tomorrow, along with talking debax and some of the impacts we've seen from things like the Dora study which we worked with Google on this last year, so it should be incredible conversation and excited to dive in these insights to you two, all right, And then we also have trusty Rusty Steve Hello, Hello, Hello a j. We got to work on your posting skills
here. You always do the panelists before the guests say the best for last, right, so we got to work on this. You're kind to me on that one. We'll take. And then Connor, I had to keep a straight face when you put me an engineering leader in the same sentence. You know my general approach with work and podcasts, I always bring on people smarter than me, and so my I am the leader in good dad jokes.
But we'll leave it at that, and that's a crucial contribution. So that's very crucial, Yes, and I will show leadership at the end of this episode in that area. That's right, leaders joke last, But no, that's not how it goes now anyway. But with that all the way, we'll go ahead and introduce today's topic. We're going to be talking about jen AI, which is like Generation Z except not at all related. It's just generative AI. And then developer experience and you've got those insights you're talking
about, So where where should we start? Connor, Yes, I think a great place to start is a definition phase. So you mentioned generative AI. Let's just make sure we level start with the audience and showcase that. Yeah, when we say JENAI and software development, we're talking about generating code through the use of an AI tool like Copilot, like these other tools out
there. And the cool thing that linear B has done is we have done a study following up to the great work that the Dora team did this last year and said, Okay, you know the Dora team when I met with Nathan Harvey and we've partnered with them on their door report in twenty twenty three, they found that there weren't early efficiency impacts from Jennai that were showing up
in their survey audience. However, what they did see was that teams that were using Jenni and about fifty percent of the respondents were, saw improved happiness on their deb teams. And we correlated that with, oh, you know, maybe we don't have to necessarily spend as much time making tests. We can have the AI helveles with that generating those or maybe the annoying parts of the code can get kind of abstracted out, so I can then focus on
these strategic pieces. And that's where I think there's this huge opportunity. Initially is like, hey, how can we remove these blockers? How can we improve here? And now what we've done is we've followed this up and we've done targeted surveys with over one hundred and fifty different development teams across the world and are now adding in quantitative data as well from our linear By tooling.
Linear By for Context is a platform for software delivery management and software engineering insights. We helped both provide quantitative and quality of metrics and automation for improvement.
And one of the things that we're going to be rolling out here shortly is a beta of how you can actually track gen AI pole requests or partially generated partially GENI generated pole requests and then compare them over time to code that is not being affected by GENI so you can actually see are we getting efficiency gains? Is my GENI initiative at the executive level making an impact or is this kind of more hype than anything else? And we have some really interests,
Yeah, please define GENAI poll requests. Does that mean that the AI wrote
the description or that the AI wrote the code or the code contribution. So we're we're working on a tool to automatically detect geni's contribution there, so linking up with Copilot or other different GENAI code generation tools to say, hey, this pole request is partially written or was written in concert with kind of a GENAI code assistant, and then we can evaluate code quality down the line and say, did we have to refractor it more with our issues that were caused
by it? Do we need to maybe add an addition test early on to make sure that everything that is working. How does this fit? And we're seeing some fascinating early insights. Okay, so before we get going too much farther, we got to find some terms here. As a parent, when I hear Dora, I'm thinking swiper, no swiping, you know, Dora the explorer? Absolutely, So who is Dora? What is Dora? Why
is Dora important? And et CETERA great question. I am using a lot of acronyms, and I apologize to anyone who's fallen along and knocketed them. So Dora is the accelerate state of DevOps team at Google Cloud, and they've done more than ten years now of seminal research in software development and software development metrics to understand how productive engineering teams are, how to make teams happier and help build high performing engineering teams. Okay, so moving on, we've established
dora. So what were you about to ask aj? So you Connor, you just said something about results or insights that you were kind of summarizing that. Yeah, it was probing for more detail. Happy, happy to dive into some more. So I'll say a couple of things. One, across our survey set, we saw that eighty seven percent of organizations are already planning to invest or have already invested in the Genai tool in twenty twenty four.
So the idea that this is something that is very broad across the industry is absolutely true. We're seeing that rapidly accelerating our data set. Compared to fifty percent in survey respondents last year for the Door survey, We're not seing this continue to increase, and it's it's a key area of focus, and I think part of that is that efficiency gains people are starting to see, and then the happiness gains. I mentioned that are clearly being shown across dev teams
that are labaging this. You know, a great example of a success around this is Uber, and Uber has actually started to help not only generate code internally using Genai coding tools, but also to generate comments for codes. So, what's one of the most annoying things that happens in a software development team's life cycle? It's like, Oh, I have to go review someone else's code, right, Like okay, Like I have to, you know, pause what I'm doing, maybe jump out of a meeting and I'm like,
hey, this, what do I have to understand this PR? Or what's the context I need? We can start leveraging automations like the guests through automation that linear B has to provide context, and then Uber's actually saying, hey, we're going a step farther and we're actually going to help generate comments, suggested comments that people can then evaluate immediately in that PR to help improve the
PR's quality. So huge opportunities there, And I know Ubers Developer Experience and Developer Productivity teams have estimated they saved about ten million dollars in time saved for engineers last year alone. And that doesn't even consider the potential positive benefits on
that happiness piece we talked about earlier. Of like, if I'm a dev on a team and these annoying tasks are starting to get automated away, or at least it's getting easier for me to do them, whether that's tests or reviewing code, We're having better tools to help me that are enabled by generative AI. This is a huge opportunity to improve retention, improve de have happiness. And we've already seen a million correlations across various studies with happiness and high
performing teams. So it's a really exciting thing for the industry. Okay, so we've got a we've got a gambit from there are people who literally do not start coding without some sort of AI integration. And then on the other side, we've got people who vehemently refuse or are banned from opening such such tool. So can you tell us anything about that spectrum and it was any
of that type of data captured in the report? Yeah, And this report will be out for everyone to read here the next few days, so definitely go to linear beat on io to check out more information here. It should be out forty eight hours, is ish from now. But the exciting thing here, as you mentioned, Aja, is that you know, there's these opportunities for improvement, but there are risks. And this is one of the key things that we captured in this report, which is the risks are perceived
as very much compliance quality. IP security is a top concern for devs broadly, but particularly with generative AI tools, where they can be kind of a black box of what's this code that's coming out of here, We're you know, giving access to our code base, what's the risk level here? And what we saw is that risk levels or perceived risk levels drop across the board as adoption grows. But IP and compliance are major concerns for frontline managers and
executives. And interestingly, something we saw is that the frontline managers are actually more concerned about these risks, maybe because they have more in depth understanding of what the tools are actually doing, maybe simply because they know they feel more accountable to it than the directors and executives, who I think are feeling a lot more pressure to go, oh, hey, I have an initiative I need to do. My board is telling me we have to have an efficiency
initiative, we have to do more with less. I'm sure we've all heard that terminology thrown around, and so there's this odd change between you know, the business side which is really pushing this and says, you know, we have to improve our software development with AI and the managers who are are to your point at AGA concerned, and I think frontline devs as well in some cases, who are saying, hey, is this the right thing to be
doing? Does this actually help me? In and AJ Before we started recording, I know you mentioned that in your experiments with genet of AI code you felt like there are times where maybe it just wasn't getting things right and you weren't as tightly aligned to what you actually needed here, and that is definitely something we saw as well. So there's a huge opportunity for improvement where like basic tasks, automations, tests, these are things can really be helped.
But as code gets more and more complicated, it's harder to get your GENAI code assistant to actually match the in depth needs of very particular coding needs. So there's very much still a need for senior developers to come in and say, hey, I understand this code based, let me help guide this,
let me make this work. And you know, one of the things that we're predicting is that that need for senior dev is actually going to really increase, because, particularly as junior devs start leveraging Genai code to just move faster deliver more code into the system, we're going to need a lot more senior devs who can help parse through that and make sure the quality is there.
And there's a huge risk that as we just simply increase the speed which which we generate code, that there are quality issues in that review cycle, in that actual delivery cycle, because we designed all of our software development systems to have a certain amount of code going through them, and now we're saying maybe
we're have twenty percent more code, maybe have fifty percent more code. Are the different systems of review, systems of tests that we expected to have here going to work or are we going to have these huge blockages that maybe then people don't spend as much time reviewing. And this is where there's this need for us next level out of me like what I mentioned Uber so, I like what we're doing with our free get stream tool, say okay, how
can we help these blockages in the software development life cycle? These area where the pipe kind of tightens up, particularly code review move faster. So how is code review going to be affected by AI? Because to me, like, okay, so my experience with it, I use both Olama with the code Up I think it was code up and code Wizard. There's two models that I've switched between. With Olama and then with GPT chat GPT I use
just the main website. I don't have any VIM integration for code gen in part because I've from what I've seen and experienced thus far, having it pop up bad code half the time and me having to do the mental context switch of like am I writing? Am I correcting code? Back and forth? That's like that, I'm that's frustrating to me. So I'd rather just take a few extra seconds to type things out and just type out what I know I need to do rather than like approve or deny and retry on something that's
relatively you know simple on cogen. So for like little snippets and stuff, I'm not doing it, but I will. I'll ask a GPT a question. And again it's like it's like fifty percent of the time it's it's worth it, I guess, because you know, it'll get it right, it'll get it wrong. But I'll ask you the question, and it seems like the closer my question is to Python code to start, the better. It's
going to be a generating go or job. But but there is some value there because there are some things that are relatively common boilerplate that are you know, it might be thirty lines of boilerplate and I'll ask it to do something that I know is maybe in that realm, and it can generate it. It's very good at that. Yeah, it is. It is. I also asked it to write a backup routine and it wrote a function that would delete all files in the folder. Yeah. That tracks as I think we've
both experienced here. The more complicated your use case, the farther away Jenai is from being ready to really solve it for you without deep fine tuning. This is where major enterprises have the opportunity to say, hey, we really want to fine tune this tool internally so that it fits our code. Basic it fits our needs, but right now it's really good at the basics helping
solve these kind of pain points. And where we're seeing it. The interesting code review, as you asked, is that we have just more code moving through the system. Because the basics are all easier now, people can now just deliver code faster. And so now we're seeing this major pain point which was frankly already a pain point for many software development teams, which is, not only are there more code reviews to be done, there's more code to
be reviewed. People are still spending most of their time on generating code. So what we've seen is this already existent pain point get exacerbated by the increased speed which code is generated. Yeah. Yeah, but oh so what I was gonna say is what I see it do is I'll see it add comments to code and asking it to summarize code. It can it can often do a really good job at that, even even if it's something pretty niche.
Like there's been cryptographic stuff that I didn't understand and I was working in some cryptographic library and I paste in some code and I say, you know, what does what does MKW stand for? Here? And it's like that's the Mills Buiser kill Awowsen curve. And I'm like, oh, okay, is that a real thing? And then you know, it turns out it actually is, and I'm like, oh, okay, that's that. And so it's done really good at generating comments, but then most of the time it's
pretty dumb. It's you know, because it's this adds two and four to get six. I'm like, yeah, yeah. And so when I'm when I'm thinking about in code review, I just most of what it would do.
I you know, if it was something for a cryptographic library, I would you know, want somebody who's at a really high level to be handling that because it's something that's really sensitive and we don't want to take a chance that it's a hallucination, right, And if it's something that's really basic, and then it's just going to spit out, Okay, this you know, this really benign or banal description of code or something. Where where does it
Where is it helpful? What is it doing? That's that's really what's the sweet spot for that. So I actually think you nailed the opportunity here, which is context and and saying okay, how can we autogenre generate the right piece of context. So maybe it's things like something that we're doing is PR labeling. How long is this PR going to take? We can estimate based off of a little machine learning on the back end and say oh, this this PR is x number of lines of code long, one hundred four lines
of code long. This will take five minutes, or this will take thirty minutes to review. And it'll can also give you, hey, here's a few things that you might not know what this means to your point, you know, the MKAV or whatever, and give you that context so that you can hopefully jump in. And it's a lot easier to get acclimated because too often, and I'm sure you've experienced this someone you know, you get assigned as a reviewer for a PR for someone else and you used to go and
you're like, Okay, what are you doing here? You know, maybe they didn't give you a great comment from the start with of like, oh, here's the context, and that's where I think, Jenny, I can start start to help us. So it's helping us very much in the Hey, let's code let's code through the kind of annoying boiler plate faster. Let's code test faster. Great, and then let's give context faster so that we can then make more strategic decisions and spend our time on kind of the higher
leverage opportunities of Okay, does this all work. Let me apply my knowledge. Let's talk about test for a second. My experience in testing code when
I really feel like I need tests is usually one of two situations. It's usually either a regular expression or it's something where I actually am not certain what the result is supposed to be. So I'm writing a test kind of to say I think the result is supposed to be this, and then if I find a case where it falls outside of those parameters, then I need to manually review and say, Okay, is it that the code is not handling an edge case or is it there was an edge case in the input that
needs to be handled in the test I'm typically not doing. I I don't think that the type of when I'm when I'm reaching for tests, I mean, I guess the regular expression stuff? Could I mean the test is really there more to explain what the regular expression does. It's more like a comment at that point. But then again, so is you know that other thing? So? So what kind of test generation again? Is is where the
value is being brought to the table. Yeah, so I would say two things, And I'll admit here that I haven't spent a ton of time generating test with Jenny Echo, this is very much something that I'm hearing secondhand from people that I'm seeing in our data. I've heard it. I've heard it.
I just haven't seen it. I haven't seen I totally will say, oh yeah, I'm generating my tests, but I haven't seen the tests that were generated to to be like, oh yeah, that brings a lot of value to the table I should I for two examples, so so one that regular expression when you mentioned totally a great use case like this is something that's very parsible. It's I don't want to call it basic, but it's like, okay, it's boiler plate that we've kind of talked about. They can
get this, let's let's help new things faster. Gave me a little time, and the edge cases I think are still figured out. Though maybe you'll have someone who's going to come on next week who's going to be an expert in edge case testing with Jenna I and they can help help lead us there. But the other area here is that when you have a large enough database, So I'm going to keep using this Uber example because it's something that they looked at and they're saying, Okay, now we have thousands of devs.
We have an idea of basically what our codebaseys we know a functions. Let's take all of those tests and make them artifacts that help train this AI. It's particularly saying, okay, let's suggest tests for you right off the bat for some of those EDU cases. Here's things that we saw as potential educases that are issues in previous similar areas of the code base. Let's just suggest
some right off the bat for you. And that's an area that I think is a huge opportunity for companies that and particularly like B to B companies that are working with a variety of clients. We can say, hey, can we train a great database to improve testing off of this? And I think there's a lot to be done there still, but we're hearing great results from individual devs and small dev teams, and we're seeing some enterprise companies succeeded that.
So I expect there to be an explosion of these tools moving forward. Yeah. So, I think one really interesting thing we saw on the report was one of the things we asked Serve of respondents was hey, do you prefer qualitative metrics or quantitative metrics? Ie, do you want hard data of like this is ten percent better or do you want Hey, we saw developer
happiness improve. And what we saw is that, particularly with larger companies who are usually farther along in their adoption journey for AI, they just have the resources to do it and add it more than the software development process. They were ready to start saying, hey, we want these hard metrics. We want to see. Eight percent efficiency gain is the ROI of our Genai initiative,
you know, seven percent of efficiency gain. Whereas earlier on the initial signals all, hey, our devs are happier, they feel like they're moving faster. We see this gain in total lines of code created, but those are all kind of bad mesh. Well, I mean, death happiness isn't
It's an amazing depax measure. We can definitely talk about that. But like lines of code for example, is I think we all know not necessarily indicative of great code or you know what you actually want and there are some terrible favorite there. My favorite prs are more red lines than green lines. Yep.
And this is where I think companies now looking to say, Okay, what's the actual impact of the code that we generated with gen Ai, And this is where we're running our beta with linear B we're saying, okay, let's label every PR that has been code generated with genera of AI or fully generated with generative AI, and then track its life cycle to see its impact on the codebase. And we're we're very much just starting to get data there.
We're going to be putting some out in a workshop here in a couple of weeks. But I'm excited about the long term potential there because I think we're going to have this transition over the next couple of months from you know, survey data which is super useful, to survey data backed by these hard data metrics, these quantitative data metrics that that can really dive into it. And this is where you also see this in developer experience, which kind of
like was the early indicator here of success. So, as I mentioned in our work with Google Clouds State of Software Development Report last year, we saw these efficiency gains for teams because team members were happier, and it was kind of correlating this positive feedback cycle for dev teams based off of the leveraging gen
AI, whether it's for tests or coding, those basics. And now I think we're starting to see this wave and are going to continue to see this wave of here's the actual impact of that org long term, here's where you know they delivered X number of futures faster. And that's what we're really excited
about, is that next step, and that's where we're building towards. Okay, and then the developer experience, and I want to allow us to take this conversation outside of just the AI portion totally, but with the developer experience. What were the areas in which that happiness is improving or that and well and also I want to know, I want to know what what tools actually before I go into the question about the happiness what what tools are people using?
Because I'm I'm familiar with Chad, GBT and Olama those are the ones that I use the most. And then I know that a lot of people are using vs code with the Microsoft uh CHATGBT plug in, the co pilot plug in. What what else is there out there that people are using.
Yeah, so obviously Copilot's the one hundred pound gorilla here. They've got I think one hundred and fifty thousand organizations across the world using it Chat GPT being the more broadly available for every individual dev and everyone else, code whispers and other that's out there that we saw. It was quite popular in our survey. Codium, dot AI and TAB nine were a few others that we saw.
But you named both of the key ones there, okay, and then Tab nine man, oh they were doing they were doing a marketing push. And then I think two weeks later, uh, Microsoft announced the GPT copilot.
I was so sad for them that their timing was tough. Yeah, yeah, but it's hard to compete with that, a company like Microsoft when they decided to roll something all like that, well, and I mean, and you don't know that it's coming right, like, yeah, they they had they probably they've been working on their platform for however long, you know, a few years, finally getting out of beta and and then just overnight.
Oh that's so rough. That's so rough. I did use Tab nine and I kind of had the same you know, feeling feeling about it like it was. It was. It was cool, it did. It was so good with refactoring, so good in terms of you know, I need to go change the next ten lines to be from I don't know grams to kilograms or whatever, you know, some sort of I need to add sql tags to the comments I needed whatever, like, it was so good at that sort of thing. And I imagine code Pilot probably is too, although
or co Pilot, although I haven't used that. And then Codium I think I've heard of this one. Is this any relation to VS Codeum Yes, Codeum not AI is its own website, but I do believe it's related. If I'm not miss speaking here, maybe Steve's gonna correct me here. Well, no, I just said a panelist co panelist to mine on another podcast was using Codium and he really liked it. He didn't get into details, but it was just the way that AI was just sort of built into everything
in the ide was was the way he described it is very helpful. So I haven't used it myself for seen it, but I just remember his description of it. I know they have uh well the jet Brains and VS code Free versions, so I do. I do believe they're related there, Oh, je brands, I use PHP storm. I checked that out. Well, you know, I'd be happy to pay for their their their VIM plug in when that comes out. Okay, is this one more privacy focused or
something? Or what makes it? Do you know what makes it different I'll be honest, I don't. I don't know a ton about what makes that makes them different. I've been pretty focused on I'm looking at copy it because the data sets a lot bigger, and kind of evaluating our metrics across three areas, So like adoption AKA, are you opening more pole requests? Are you getting more poll requests merged? Are we just seeing more code flowing through?
And then the benefits piece of are you merging more frequently? Is coding time going down? Are more stories getting completed? Are you being more accurate when you plan stories? And then also we're starting to look at the risks too, like our PR size is bloating, is rework rate up? Is review depth down? Because people are just kind of skimming our PR has emerged that review that kind of thing. What about do you have anything in place
for license violations? Because I think that it's it's been shown that in many cases it's almost a direct copy and paste from the source material, And I think that I haven't experienced this myself, but I've heard that people have complained that it's generated GPL license code as like a whole block, which you know has which is really weird because Microsoft, for the longest time, you know that they wouldn't even they would not allow their employees to look at GPL code.
And to think that we've got an AI scanning all of GitHub where I don't know what five percent of the code, I'm just making up a number, you know, But it's like MIT and Apache. You're probably the highest BSD, but there's a lot of GPL code out there. Any any thoughts or observations on the licensing concern, I think it's a huge concern to your point, like, this is one of the areas where we saw significant serns this compliance side of things. We don't have a full solve for it yet.
I think this is something well, you can expect us to have an automation out on our get stream platform within linear b here in the next month or two that will hopefully help track that. But it's a it's definitely a risk that I think companies are going to need to solve in it. I mean, we're seeing lawsuits over this kind of thing, not just in the code space, but look at the New York Times suing open ai for use of their content and models. So the legal side of this is going to
be a massive thing. To unfurl over the coming months, no question. Yeah, I can remember. I can remember the big stink when when Gethub's copilot first came out that I remember seeing on Hacker News or other places that there were authors that had private repos you know, with non GPO licenses, so it should not have been annexed by by copilot, and they would type in and here comes their code, just you know, straight out of the way they read it coming up from copilot when it wasn't supposed to be.
So that was I can remember that stink, and I haven't really followed it since then as to what actually happened with those instances. But it's just struck me as odd as that I would think that, you know, co pilot being a GitHub product and GitHub having access to all of the repos, how you don't have a flag in there says okay, if this isn't private, don't going into exist. Code didn't seem real complex if you own the whole back in. But you know, obviously I'm not sure how that works,
but that was a big issue. Yeah, it's a great question, Steeve, and it's really interesting too because we've talked about these like three use cases for it right of you know, writing new code, writing tasks, assisting in code reviews, and then there's this whole other side of i'll call it like generating documentation, and some of those it's totally fine. Like genering documentation.
Sure, I don't I don't care if I'm copying something probably like maybe there's some compliance concerns, but that writing new code piece, there are absolute clients concerns. And and I know Copilot is working on this, but to your point, it's surprising they're not a little farther along given that it is
a Microsoft for Lata product. Yeah, and I do wonder, Okay, I've seen one of these reports that came out where it generated like entire paragraph of paragraphs or text, or it would generate images that looked almost identical to the original but with basically noise on top of it. And in that case, I think that that is truly a copyright concern. But I also wonder how much of this is boy cried wolf, where how many ways are there
to implement the you know, area of a triangle function? And so I do want to, you know, just put in my own caution there, even though I'm the one that brought this up. I haven't looked at what these people are saying to verify, Oh, that's a legitimate copyright problem as opposed to, well, how many ways in Java can you implement a JWT
siding function? You know, maybe it pulled from some quote unquote pulled from GPL code, But maybe that's because the GPL code is also the same as the MIT code and the Apache code for those three lines that everybody has in
common. And I think that that was one of the things in that Oracle versus Google lawsuit over Java that if I recall correctly, ultimately they ruled in favor of Google, saying yeah, like for common mathematical functions and you know inputs and outputs of an API that you can't copyright pat in that stuff because
it's just too basic. So you know, there's there's probably some mix in there of people crying wolf and making a big della nothing and then and then some actual you know like what Steve was saying, where you know, there's some sort of proprietary code but regular to a company, and then boom, all of a sudden, it's been shared. And actually, that's one question I had, is that how much of this code because with chat GPT, the license with chat GPT, and I wish they would make it easier to
get a business account. I'm trying to get a business account and it is it is really hard to get a business account with OpenAI. But anytime you use chat GPT, anything that you type in or paste in goes into the pool for model learning. But on the business account, if you use the API and then the it doesn't go into the pool for model learning, you
keep it proprietary. Do you know anything about that? Well, I would just say that I think this is a huge opportunity for open ai and all these other companies here, and this is why you're seeing them get these crazy valuations, is because every business is going to want to secure their own code base and data and they don't want to extend it into the main chat GPT
data. So to your point, AJ like, this is a reason for every business to look at that business business style version of it and paid that extra dollar. Is that compliance use case? And it's a classic B two B one right where it's like, oh, you know, anyone can get our basic tooling, but you know you have to get our enterprise package in order to add the compliance piece add you know, role based access control,
these other things, and I think you're absolutely going to see that. I mean, we already are with all these AI companies, all right, So now now let's take the turn. Unless, Steve, do you have any more questions on this topic? Now, let's let's take the turn and go into the developer experience the developer happiness topic genre. So we go ahead. Let's start with the AI what what are the biggest improvements, what are people
excited about, What's what's making them happy? Yeah, So I mentioned Dora's report from last year, and I kind of I'll just say the twenty twenty three accelerate State of depop Support is what I'm saying when I'm referencing that. So I interviewed Nathan Harvey, who's the head of Dora's kind of outreach team, on our dev Rupted podcast last year and we were also partners Lenard View
as partners on the report data. And one of the really interesting things we saw in that report is set they saw among the three thousand plus organizations they surveyed that last year that about fifty percent of them were using AI tools already and most were already considering an adoption, but of those folks, there was a correlation with improved developer happiness on dev teams, and so this is where we've started to see these great examples of teams that are saving time on documentation,
saving time on code review, saving time on tests, saving time on codgeneration, and it's helping devs get rid of these annoying blockers in their time and spend more time on the strategic inputs. And so it's a really key piece of how you can potentially make your team both more efficient and happier. And there's plenty of research, whether from Google or many others, that has
shown that happier teams perform better. There's this like positive feedback loop of hey, I'm feel good, I have more energy, I maybe feel like I'm being more impactful on my work. And this is where I think a lot of companies are starting to wake up to the importance of that developer experience idea. You can look at LinkedIn, which uses a developer experience index, which
is a specialized metric. LinkedIn provides to teams like aggregate scores based on a number of different metrics such as you know local build times and you know happiness scores on surveys and a variety of qualitative and quantitative measures, and they use that to help kind of calibrate the complexity of developing within their orgon. And there's plenty of the companies that Google does this, Peloton does this. Obviously
they're different pieces. Like for Peloton, for example, they look at engagement, velocity, quality, and stability. And it's really important that the organization starts to think about this in the context of am I making my devs perform better because we're giving them the right tools, We're giving them the right time, we're right frameworks, and am I going to be able to recruit and
retain dabs because of that? And so whether whether you're an individual developer or you know, a major corporation is just looking at some of dollars and cents. Because STEBAC should be something people are thinking about. I would like to categorize this under the category of the sky is blue and the grass is green as well. I mean, if you're you have developers and you want them to do your work, it would seem logical you're going to do best when
you give them the tools that they need. This is I mean it extends to more than developer tools. There was a recent article that came out and this has been along the same lines, although it's not specifically code related. There's whole studies that have been done about the workspace that you give engineers, and that you will you find that engineers need an office space instead of being
crammed together in cube forms for various reasons. Has mostly to do with focus and you know, not being interrupted and so on, and that's one of
those things that that to this day is a no brainer. You know, anybody who's been an engineer or has worked in environments like that will tell you you, oh, yeah, that I can focus better, I can concentrate, I can get more done, and yet you still have people that cram them all in the queue farms, and there's you know, there's various reasons for that in terms of office space and having only so much space, you
know, versus being able to give everybody their own individual office. I can still remember being out at Intel. Intel has a lot of their engineering offices here on the West side of the Portland area and being out there one time and holy cow, you literally had to have a map to know where you're going in this gigantic queue farm, and I was like, how do people get stuff done in here? So, you know, sorry to go off on a tangent, but you know what you were saying struck to me?
Is you know it struck me as Yeah, Okay, that goes without saying it doesn't have to be a dead job. Any job here, Go do this job, but I'm not going to give you the tools you need to do the job. Well, I output it. What kind of output are you going to expect from them? Then? The challenge, I think is that too often, particularly lately, software development and other R and D teams are being beholden more to I'll call them CFO metrics without thinking about what's actually
happening on teams. How can we improve it? And so it's saying, okay, well we can save money by doing X. We can you know, cut hat count on this team and that'll make us more efficient on our cap table and we can return more value to shareholders. And you want to know the classic the poster child for this right now? Everybody knows about Boeing. Yes, okay, Portland two weeks ago, plane takes off, door panel comes flying off mid flight, two seats get checked out. A twelve
year old sitting next to them has his shirt ripped off. The plane comes around and is able to to make a landing. It was an Alaska Airline's flight, which happens to be an airline I fly quite a bit. And this has been an ongoing thing with Boeing ever since nineteen ninety seven, especially with well, especially with the seven thirty seven Max line of planes. Where
it used to be. I've seen articles written ad nauseum where Boeing used to be an engineer driven company and everything was about quality, didn't matter the cost, Let's get a good plane made, and that's how they developed reputation. And then in ninety seven they merged with with it Pratt and Whitney or who was it that they merged with. There was another giant AERONAYM. Mc donald
douglas. I think it was McDonald douglas. And the joke was that McDonald douglas bought Boeing with their own money because they came in and pretty soon everything was driven by cost and cost accounting and let's save money. Let's not do anything that's going to cost a lot of money. So Boeing decides, hey, we need to keep up with Airbus. But we don't want to take ten years to develop a whole new product that's going to actually compete, So
let's just take our seven thirty seven and modify it and tweak it. So right out of the gate, you've got planes crashing because of software that overrode the pilots and forced the plane to dive into the ground. That happened at least two times. And now you've got they started outsourcing production, outsourcing building of a lot of this, which is where this particular problem was tracked to, and a lot of QA stuff that happened there all because quality went out
the window. And we've got c suite people who weren't engineers making decisions solely based on money and not on quality. So sorry, that's my rant, But no, that comes right to mind as soon as I hear that.
That's an incredible example, Steve, and I think it really correlates to something we're seeing a lot in software engineering right where we could name layoffs that have happened in January ad nauseum, and a lot of the reason for them, you know, whether it's Amazon layoffs, YouTube, just score, whatever else
might be. Because Section one seventy four of the US tax code got changed, which has now made it so that you cannot expend right off R and D expenses in the United States as part of tax You have to instead cost capitalize them over five or fifteen years, making it more expensive to develop. And companies are saying, oh, well, let's just we can outsource this, Well, let's not actually pay attention to the quality piece. They're making
those same decisions that Boeing did. To your point, see Eve, and I think you brought us something else that is really interesting, which is the idea of the space that we built for developers. And I think it's important to think not only about you know, the office space, the meat space i'll call it, but also the digital space. Like platform engineering teams can significantly enhance developer experience by building a tight information loop that identifies pain points and
gathers DEV feedback. And if they use you know, the right internal developer portals to do this, you can you know, actively solicit develop or input via surveys and use quantitative measures like local build times and cycle time to under
standing track blockers and broadly improve developer experience. So like thinking about both how we're setting up devs in person and setting up devs in digital space is really crucial, and I think you guys are doing great job highlighting that, because if we instead spend our time thinking, okay, what's the most cost effective solution, well, it's probably you know, pushing our dev team off to some other country and cutting our US team entirely. And there are major risks
that come with that, but they're not usually risks in the moment. They're usually long term risks. And too often companies are thinking, well, I'm gonna I can cut costs right now, and I think Wall Street or my investors are really going to like that. You know, my vcs will love seeing our cap table look better here because we're spending less money on our R
and D costs. And you have to grapple with the whether or not that's going to be good for your company and the product you're building, whether you can actually succeed long term. And this is where I think a lot of companies are making a major mistake is they aren't tracking developer experience metrics. They aren't tracking their engineering metrics, and oddly so they could see, hey, there's an efficiency drop, or you know, the pr quality that is being
merged is lower now. And yeah, this example you gave Steve of the McDonnell Douglas merger with Boeing, I think is a perfect example of this in an engineering heavy field. And I'll also say, like I was supposed to be flying a seven thirty seven Max nine the next morning that got canceled, so right there with you on the Alaska flights getting canceled and the impact there, and they then ended up finding like bolts loose on multiple planes. What
could happen? Again, very scary yep. And then there was another one in Japan where uh, there's four layers apparently to the windshield in the cockpit, and one of them was cracked as they took off. So they came back around and said, uh, yeah, no, So as far as I know, all of the that particular seven thirty seven Max not als like them seven thirty seven Max nine something I think had been grounded. But anyway, yeah, I think they're bringing some back into service once they've been like
fully checked over. But definitely it's a super scary thing, and I think we need to have that same risk concern in software engineering, like sure, maybe we're not going to have a plane blow out its window. But maybe the software you're building is impacting you know, whether that plan will dive into the ground, as you mentioned, Steve, or maybe it's impacting people's bank
account information. They're they are very real world concerns with both the compliance and security risks that we have to consider and also the quality of our code based generally, and this is why I think it's so crucial that every software engineering team above a few people needs to have some basic visibility on how is our team performing? How is our company getting value out of this? And it shouldn't be about, hey, is this individual developer delivering X lines of code?
Because it's a terrible metric of success, but it needs to be are we, you know, delivering on our promises to the business? Are we having to refactor a bunch or or in fact, are we, you know,
how, helping drive revenue by what we're doing. And I think any engineering leader today, any dev leader, needs to be stepping up and saying, hey, like, we have to help influence this discussion at all the way up to the board level, or else we're going to see more CFOs make that decision that you mentioned, Steve of saying, hey, let's cut
costs, let's just focus on shareholder value and not necessarily customer value. And when you start getting away from customer value, that's when these huge risks come into organizations. All right, So you've you've kind of touched on my favorite topic there, which is in the world of sas, the customer comes last, because you're taking money that was almost free printed from the federal reserve that's given to venture capitalists that are only interested in growth and don't necessarily care about
profitability. So what happens on the black mirror side of things here? Do
you have any thoughts on that? How do we how do we how do we uh incentivize that we actually care about the customer and the developer or you know what risks are there with AI being Yeah, Well, so for example, you know, people for hundreds of years have been saying, oh, this technological improvement X is going to make everybody's quality of life go up without having to work more and we're gonna, you know, we're gonna work less hours. And this is you know, this has been going since you know,
probably the era of the factory. What actually happens every time, well, oh, by the way, Steve, do you have a thing? Did you put that thing up on the screen? I can't take that down? I don't think Okay, yes I did, right? Oh sorry, anyway, I just got distracted because the the I was expecting comments fade away after twenty seconds or something, and it didn't. Oh no, I got to take it down. Yeah. So so the you know what, what's how do we what what might the abuse situation look like? And how do
we prevent it or curb it? Oh? Man, that's a that's a
big one. So I'll say I think what you're talking about here is that a lot of the value that gets generated from technological innovations like AI or you know, improved machinery that helps assembly lines or anything else has happened over the last one hundred two hundred years, and during industrialization has accrued to capital because uh, you know, shareholders are are driving this and they accrue most of the gains, and not most of those gains aren't necessarily accruing to the working
class. I know, I'm swear, I'm not a Marxist, y'all, I'll tell you, but I'm just this is we're talking about here. To be fair, the rising Tide raises all ships. Like the total Amazon exists, I have access to products. I personally have access to products that I would not be able to get access to otherwise without having much more specialized knowledge
and having to drive much further to get to a specialized store. And as your points to fake crap, and I can't always distinguish between the product I'm trying to get in the fake version. But like, I have access to more products, So it's not it's not like it's all bad. But in particular, what I was pointing out is that they said, oh, you're going to work less and you're going to have more. I do think that we have more. We're certainly not working less. Are we work differently?
Yeah, I don't know. If we're happier, well, I think has a lot of other impacts, right right, Yeah, sure, I think we I think we work differently. Is really what it turns out to you right where it's like, Okay, maybe now with these AI changes, we're going to spend less time writing tests, We're gonna spend less time generating basic code. We're gonna spend a lot more time strategizing on the deeper pieces of code, or how does this all fit together, or understanding making sure these
systems don't break. Maybe like we're seeing an increased need for platform engineering and internal developer portals and you know dev X teams at major companies because we need to make sure all these systems we've built to improve development speed work. And the other piece I think of what you're saying is like, you know, this kind of goes back to our discussion of hey, dev teams need to
have these conversations. Dev leaders need to have these conversations with their CFOs, with their partners in finance and their partners in sales and all these other place parts of the company, and we need to be part of influencing the business discussion and saying, hey, make sure some of these benefits do accrue to dev teams, like we're helping provide more efficiency and more value, Like we
need to be able to work on things we're passionate about. We need to be able to be you know, and sure we're have DEVAX metrics that look at whether those teams engage and talk to the value of that. So I do think you're spot on there, and it has absolutely allowed us to create these incredible technological improvements But what I think we need to be really careful on
is assuming that will just mean we work less. Usually it just means we work more efficiently, and then we try to increase overall value over all GDP through that and hopefully you produce more value that then gets you the next pay raise, the next piece up. And I think as an industry, if we aren't helping shape that discussion by talking about DEVX, by talking about developer productivity, by being part of that discussion, we're gonna let, to Steve's
point, the accountants be the ones who make the decisions, and that's where we get into major trouble. Shout out good accountants. Ever, by the way, I'm not saying accounts are bad. So one thing that was funny. You're talking about devs teams. Originally that was called DevOps. The purpose of DevOps was to make the developer experience better. Then we just rebranded ops to DevOps without changing anything. I do agree with you. So what's the
new it's gonna happen with THEVAX team stuff? Well, now, platform engineering. When you heard so, I mean to your point, it's all it's all a bit of you know, six of one and a half dozen of
another. We kind of use these new terms to speak to new eras of how we think about soccerlent, but we typically retain it right where it's like, hey, the agile movement this huge thing, and then DevOps this huge like layer on top of it. Now we're talking developer experience and platform engineering and building on this, and really, I'll say my take is that it's it's basically, how can we improve on the last standardized way we did this
and how can we keep leveling up? But is the role very similar? Absolutely? Are many of the concepts the exact same? Absolutely? All right, Well, it is about time for us to wrap up here. Is there anything else that you wanted to share? Any any topics we didn't discuss. No, I really enjoyed the conversation. AJ Steve has been super fun. Thanks so much for having me on. Well we do picks at the
end. Hopefully you were informed, which is, you know, basically can be anything just stuff that you're interested in, could be tech related, not tech related. I'll go ahead and start to give you some time to think about something. If you if you hadn't because I I'm guessing if you if you got the email with the wrong calendar link. You probably didn't get the
email with the description anyway. So there's a couple of things that I still that came to my mind a few weeks ago that I still haven't gone through all of it. So let me see if I can scroll to where this was. So one thing that I was going to talk about that I mean, I've mentioned, just like like I did today, this whole issue with you know, we've got the investors and they're focused on growth and and that's
a lot of what's driving SaaS companies is not profitability but just growth. And then what happens when you've reached the market that the full market potential of the growth? You know, what what happens when everybody who wears shoes has a bluetooth smart shoe. What happens when everybody who needs a calendar link has calendarly? What happens when everybody who wants to scroll infinitely and starve to death is on Facebook? Right? And there's a video that let me see what the
title is. The thumbnail was something like ninety eight point nine market saturation. Now what oh whoa, whoa, whoa whoa? That was really loud? Sorry about that I I need to reconfigure my don't auto play plug in. Yeah, okay, So the title of the video is market saturation equals ninety eight point nine percent? Now what? So? I think this is an interesting video and it just talks about this issue of these these SaaS companies are
starting to kind of cumulative. We all get to the point where growth is no longer possible, and many of them still haven't figured out how to be profitable. So what's going to give as there's no more investment to get and there's no more people to reach with the product. I'm not it seems like there should lead to a collapse. I don't think that it will. I think that something else is going to proxy and take it place. I fear
it might be something government related or you know, I don't know. But but it's this is a question that I think is I got I don't know. I guess it's not really important thing about because nobody can do anything about it. Nobody, you know, unless you're unless you're on the investor capital side. We're getting really existential today. You're just going to experience it.
But I don't know. It's it's something that that I I wish more people were aware of and and I really would like to see us get back to having profitable businesses rather than having growth businesses, because certainly profitable, profitable businesses are better for the customer by definition, because then, I would argue, is the one who benefits. They can often be better for the employee too, because you aren't having these kind of wild growth expectations. Instead, it's
how do we build a strong business. So I've been thinking about as too lately, actually, AJ, which is you know, I've considered kind of
in the process of starting my own business. I've previously been a founder business owner, and I'm very much looking at the bootstrap side of things and saying, I don't want to take VC funding because the pressure that it provides to me as a founder, to me as an employee is so substantial and some of the goals that get set by our kind of perverse incentives as you put out there, and there's a there's I think a big push now for folks to say, hey, I'm going to bootstrap, I'm or I'm going to
take very minimal funding, or I'll take and I think this is a new way for starting to see I'll take funding, but you get to share future profits instead of hey, I just get a percentage of the company. And I think we're going to see a continued wave of that going forward. Yeah, I think that that. I hope I hope to see that. I hope that the dah revolution uh kicks off strong and hard and we get back to making products that deliver value to customers rather than than deliver poker face uh.
For for investors, I mean, it's it's really a Ponzi scheme. It's a hot potato hot Ponzi scheme between investors right now and needs to get back to profitability because because it's just going to lead to government bailouts and anyway, I'll go further down that route. But other thing really cool keychain, keychain pin tool that I got that just turns out to be useful all the time. I don't know if I pull this out real quick, not show my keys on video where you can easially copy them. But I got a
little keychain tool. It's got little silicon wrapper or the pentle like sim cards, reset or router, you know, hit the reset button on the game controller. You name it. There's all sorts of things that have that little reset button on them. And now I just have this on my keychain. It's a little and and because it's in a silicon sleeve, it's not poking into my you know, like ripping my jeans or anything I need one,
or poking in my skin. So yeah, I mean they're they're overpriced only in the sense that in order to ship it, like the cost of shipping is more than the cost of the product. But you can get a pack of three or five of them on Amazon with the silicon sleeves, so I'll put a link to that. There also open audible. I don't remember when we were talking about audible last time, but thank Heavens, that's not a trademarkable word for whatever reason I don't I don't quite know, maybe because it's
a dictionary word, and I don't think you can trademark dictionary words. But anyway, open audible is you can you can download your audible books and then you can convert them into a format so that you actually own them. So rather than being a rental where when the license with that author ends you lose
access to your books, you can give your grandchildren your audio books. And so I often buy the MP three CDs when they're available, and at a at a you know, good price, within the same range of you know what i'd pay on audible, the fifteen dollars, twenty dollars, ten dollars
something like that. But I really I would not be using Audible, and I would not be listening to as many audiobooks as I do if it weren't for open audible giving me peace of mind that this stuff isn't just going to be ripped out from underneath me one day, that I do have it preserved in a permanent format. I check that. Lastly, Yeah, yeah, it's it's it's twenty bucks or so, but totally worth it to be able to have that peace of mind that you know, the stuff you bought you
actually own for you know, for real. And then the last thing was, because we were talking about the plane stuff, there's this YouTube channel mentor Pilot, and he he goes over every accident report of every plane incident and all of history across the globe, and I just love listening to it. It's one of those things that you can also listen to while you're falling asleep, you know, kind of half pay attention if if you want to just
need need to hear something. But it's so it's so interesting to learn about the different plane accidents and the different things that go into play, the different parts of the report, what people on the ground and the control center reporting, what was going on in the black box, you know, what other pilots were seeing on the radar. Whatever. You know, he takes It's not like that for every single incident, but you know, for all the
incidents that have the external data points, he links it together. He talks about manufacturing process. He talks about you know that the different grades of lubricant that are used on a particular shaft and why it you know it failed because it wasn't they ship that they switch from this brand to this brand, and now it should have been on a six month rotation, but they kept it on the eighteenth month rotation. You know, any of that stuff. It's
not too technical, I don't think. I mean like it's enjoy I'm not a pilot, and it's very enjoyable to me. It is technical, but it's not like you don't have to be a pilot to appreciate it. But he does go into detail on things and I really like that. So anyway, those are those are my picks for the day. Steve Harmley, I left you a little time for dad jokes. Hopefully I'll take the time whether you left it or not. That's the most important part of the episode.
No. Along the line of pilots, I'm working on getting my drone pilot certification and for the FAA. It's called Part one O seven And I've been taken a class that was recommended to me by somebody else who's a fixed wing pilot as well. And remember our drone program for our fire district, and it's called Pilot Institute and really really quite good. The guy's name, I forget the guy's name is Greg something. He's French that he lives in Prescott,
Arizona and does a lot of teaching and training. But everything I've been seeing from those who have taken his course and then taken the Part one oh seven FA tests, he said, you just need to know the course. Everything you need is in there, and it's been really good. A lot of a lot of information to cover, for sure. But pilot Institute dot
com is the u r L really quite good. All right. So now for the the jokes of the day, I got to give Kudo's shout out to the best stand up communitian of all time, without dispute, and his name is Stephen Wright, deadpan comedian, was actually actolutely hysterical, and recently he was on with Stephen Colbert and you know leveluce a couple jokes there that were that were fantastics. So he thought this one that he told, this one that had me laughing out loud, and he have to imagine him telling
it in his deadpan style. I'll do my best, he says. My friend Jimmy brought an electric car and he was very excited about it. Then he bought an electric blanket, Then he bought an electric guitar, and then he bought an electric chair. And I haven't heard from him in a while. Oh that loop got turned on again. Sorry, So right, So one time when I was in school, I went to my teacher and I said, would you find a me for something I did not do? She said no, of course not. I said, okay, I did not
do my home. And then a simple question, what do skunks say at church? Let us spray? All right, I got to give you prophecy. This are some solid ones there, thank you. I have high standards when it comes to the dad jokes I tell, so I'd do my best. I don't have a good dad joke to share, but I do have a couple of good carve outs, so AJ shout out for the audiobooks. Huge fan also recommend, in addition to trying to get your open audible licenses
set up, public libraries have a fantastic audiobook selections. I'm in Seattle area, Seattle Public Libraries has an amazing connection. Very much recommend it. Most local libraries, whether it's a county library, city libook library, will be part of a larger collective that has a ton of audiooks available to you. So if you want to get an audiobook through your public libraries, fantastic way
to go about it. I'll also show out to Ali Abdell, one of my favorite YouTubers who I'm currently reading his new book which is called Feel Good Productivity, How to Do More of what matters to You, Very much enjoying it, talking about a lot of the concepts that we've kind of brought up more broadly around de experience and those other pieces of Hey, how can I be happier in what I'm doing and get to do more things that make me
happy and really enjoying the book to kind of start off my year. And then I'll close by just saying hey, if you haven't checked out the Devinrupted Substack or our podcast, would love to hear feedback from everyone on this. Always great to come on these other shows and really learn from Aj and Steve here. It has been so much fun. I really appreciate you guys having me on, and you can check us out at Devenrupted dot substack dot com
and maybe Ajarstevil join us for a guest article this year. I don't know you talk you guys into it. We'd love to feature something from you guys if it's interesting later in the year. Sure, I'm up for that. Like I said, I'm not exactly what you call an engineering leader, but I could do my best. Well. I'm stoked to continue to collaborate with you guys, and thanks again for having me on. It's been a ton
of fun. Yeah, thanks for coming. I'm I'm glad that we had you and we got to talk about these topics and and get a preview of that report from Linear Body. Yeah, super exciting stuff. It'll be out here probably by the time this pod goes fully live. On all channels and lots of cool insights coming. So yeah, linear b dot io to check out the report, and then Devin op dot substack dot com to check out
our weekly guest articles and the podcast. So yeah, thanks guys, been a wonderful conversation, really really enjoy all right, well, have a good one. Audios ideos ideas,