#198 - Better Software Faster: Measure & Improve Developer Productivity with DX Core 4 - Laura Tacho

Now more than ever, engineering leaders are being asked to be more transparent with how work is getting done. Every single thing that an engineering team works on needs to benefit the business. And your job as an engineering leader is not to say, well, just trust us, we're in the code all day. Developer experience and developer productivity exists in this very strange area where what is good for your engineers and what is good for your

business are the same thing. Because the bottom line is that poor developer experience just cripples innovation. We've really nailed it down to speed, effectiveness, quality, and business impact as the four pillars to kind of put a story together about developer productivity and developer effectiveness. So there's not going to be a single metric. So for speed, that is diffs per engineer. These are like PRS or merge requests or whatever it's called.

That's the first one I always talk about because it is the most controversial. I think even a few years ago I would have said like, that's a bad idea, don't do it. It's not about ping pong and beer, which is often. I think developer experience has a like quite a marketing problem. Abby has this thing that just

like makes me laugh every time. He's like, Can you imagine if a new sales leader, like a new CRO comes in and says, well, we're going to measure the impact of our sales organization based on sales Rep experience? Like, there's just no way that that would ever go over. Hello, welcome back to another new episode of the Technical Journal podcast today. I'm very happy to have another DX person in my show. Laura Taco is here with me today. So she's the CTO of Get the X or the x.com, right?

So if people who are following into these developer experience, developer productivity, you might have heard of or seen the X materials out there be research paper or their own products, right? So happy to have you again in the show to talk about developer experience and all the stuff that you published recently. So welcome to the show, Laura. Yeah. Thanks so much.

Nice to be here. Right, Laura, I always love to invite my guest first to maybe share anything from your career, any turning points that you think we all can learn from that? Yeah, what a big question to start with. A good one. I think there's really 2. I mean, there's lots of points that have defined my career, but the two that stick out may be interesting to your audience. 1 was Docker and getting in

extremely early on a new thing. I think as technologists, it's really hard to know like what's just a new shiny thing that might be a little bit of a distraction and like what's a new thing that's like fundamentally going to change the way that we build, ship and run software. Even thinking back to like back in 2013, I'm not sure that I fully understood just how big of a difference Docker would make. But to me, it just felt different than like a new JavaScript framework, for example.

Not to put down new JavaScript frameworks, but you know, it. Just like my intuition, which I think a lot of leaders, we are taught not to follow our intuition because we shouldn't act on emotion, we should act on logic. But truly what your intuition is, is just hundreds of thousands of data points that you've accumulated over your career that allow you to make a judgement about something.

So it is, it's not just fully emotion, but working on developer tools for Docker from very, very early on, getting started with it before it was really heard of anywhere was hugely transformative to my career. So I guess one lesson, intuition isn't just emotion, it is data points. So sometimes you have an intuitive feeling about something and you should follow it. It might pay off in a big way.

The second big turning point was like I left my corporate job, AVP job that was like had a lot of responsibility to start my own business and go into coaching and consulting full time during the pandemic. Like on paper didn't look great. At the time, I was working with my own executive coach and I really realized how much of my career I had let someone else's career framework decide. And that was really the first time that I said no, if I want something different, you know, I

want more flexibility. I want to be able to go climbing at 11:00 AM or whatever it is. I need to fundamentally do something different with my job. And so I took a very big leap to do that and it paid off with a ton of hard work, a lot of opportunity and hard work went into creating my own luck to make that successful. I still do coaching on kind of on the side. Now I'm back sort of in full, you know, full product building mode with DX.

But I think the short lesson there is like, don't let someone else's career ladder define how you're gonna, you know, your trajectory and your own career growth. You need to at some point take responsibility for it and be in the driver's seat. And it's surprising how long we all go in our careers without actually being in the driver's seat. Well, thank you so much for sharing the story. I think those are really great

insights, right? So the first thing is about the Docker containerization technology, right? So I think these days almost everything is run based on container. All right? So be it, you know, the CICD and all that. So I think that kind of like revolutionize the whole thing, how we get things done. And the second thing about, you know, defining your own career path, I guess. So thanks for sharing that as well. So maybe some people also are looking into, you know, more

meaningful career, right? So don't follow other people's paths or maybe define on your own. So maybe you mentioned something about intuition, right? So the engineering leaders have to kind of like listen to your intuition, you know, gather from past data points probably that you have gathered in your experience or maybe whatever learning things that you have done. So what can you advise us to actually do more in terms of, you know, following our intuitions?

Because sometimes, you know, technology change so fast, there are so many things out there, but your intuition is not something that we always, you know, try to sharpen, right? So maybe anything any tips from you about this? Yeah. You know, this is an exercise and two things can be true because on one hand I say, you know, you should trust your intuition.

Your intuition is data that you've accumulated throughout the course of your career that allow you to make a snap judgement about something or have, you know, a gut feel. It's not just purely emotional. On the other side, I'm like a extremely data informed decision maker, but I think you know, if you get your intuition and actually I think practicing making data informed decisions helps you refine your intuition

skills. So changing your mind when new data is available to you, being able to broadly look at lots of data sets and draw a conclusion from them, but not just draw the conclusion, but then know what data you're going to look for to know that you're wrong or what data you're going to look for to know that you're right. I think those are all skills to kind of practice that will help you feel more comfortable about

intuition. I think the other part of it is like, just like in a recipe, time is important. Like you can't just throw the ingredients in a pan and like, wait a minute. And they're like, there's a time element of that. And I think careers are the same thing. You have to just get the cycles and get the reps in and let you know experience will teach you a lot. So time is a factor in career growth. And I think, you know, over time, your intuition gets more honed and more honed.

So look out for opportunities where your intuition might be telling you something to figure out. Is there a way that I can use data to see if I'm right or wrong? What data would I look for to do that? And then when you can get some more practice in that, it will just become more natural to you. Yeah, I like that. You mentioned time, right? So I think one thing that I would add also like your past mistakes, right? Sometimes in our career cannot

always go up, right? So there'll be ups and downs, maybe situational, you know, change of culture, leadership in the company, or maybe you made the wrong decision to take some career path. But nevertheless, those things are like data points that you can use to, you know, hone your intuition. So maybe let's go into our main topics of discussion today. So the first thing that I'd like to discuss is something that you actually highlighted before we

had this conversation, right? You mentioned one thing that piqued your interest lately. It's about engineering leaders not necessarily being able to become part of the so-called business leaders, right? So almost in any company, technology is kind of like enablers, right? Even though if if it's like technology companies, right, it's like one of the key pillars of the business. But many engineering leaders after you mentioned that, yeah, I kind of like make some kind of like reflection.

I think many engineering leaders are not able to kind of like switch their technology mindset into more business oriented. So maybe tell us a little bit more like what made you come up with this kind of, you know, thoughts and what do you see from your side there? Yeah, I think right now we're in, I mean, there's lots of things going on macroeconomically, right? We've got like the end of the

zero interest rate period. We had this like big hiring boom and then a bust and like maybe hiring starting to come back. I think now more than ever, engineering leaders are being asked to be more transparent with how work is getting done, how much it's costing. There's just like lots of things that before engineering was really kind of a black box. Like we got away with a lot of stuff. Let's be honest, you know, I think back to the, you know, like, let's bring this back to Docker.

But you know, I remember the, I mean, the heyday, like the height of the Docker craze, when it was just like you, if you knew Docker or knew Kubernetes, like you could just hear pretty much, you know, no one got fired for learning Kubernetes, right? You know, that's the saying you could demand a higher salary, you could do a lot because companies were afraid of losing your talent and having that business knowledge walk out the

door. And because of that, it, there was just, I think a lot of, I don't want to say that, you know, there was dancing around engineering, but maybe it, you know, we just weren't expected to show the bottom line for stuff that's happening in engineering org the same way that marketing, sales, other functions would be expected to. I think that's really changed. You know, I spent all my time in developer productivity and engineering metrics. And so I know that that's the case.

I know that engineers are being asked for metrics in a way that they just have not been asked to before and they're not prepared to be able to tell that story with numbers because now we have to answer those questions. I think there's just a heightened sense of accountability from engineering. We're no longer the as charity Majors puts at the artists and the organization that we can kind of not work without

accountability. But there's just there's more questions that need to be answered now. And I think because it hasn't been expected of us, a lot of engineering leaders just haven't had to develop that skill even on the broad level. But just think about an interaction of engineering wants to do this tech tech project product manager says we need to build this new feature. And the conversation in a lot of companies is like, well, engineering says that we need to do tech, that we're going to do

that before we do the feature. I mean, a lot of companies it's the other way around. But there was never this like, let's actually build out a business case and see what customer impact this tech tech project is going to have. That is something that engineering leaders need to do now in a lot of companies in a lot of circumstances that we just haven't had to do before. And it's a really missing skill

set. Yeah, you mentioned something about, you know, the situation about the economic, right. So the end of 0 interest rates, I think in the past, I don't know how many years. You know, they are a lot of tech startup booms, right? So many engineers being hired and you know, the engineering teams are keep growing, growing and growing. Right now, I think it's like a questionable kind of a decision, right? If you still want to hire more engineers, right?

And that's why engineering leaders are being asked to actually kind of like be accountable. You mentioned, right? So how you optimize your kind of like resource versus the impact that you bring. So you mentioned about tech debt, right? So it's always very interesting, right, whenever engineers talk about tech debt, right? Because obviously the first question is like how to quantify that.

And I think you also mentioned things like, for example, engineering leaders are not able to explain things in the business sense of business metrics, be it, you know, for example, number of users, number of growth, right, number of revenue, dollar that is being brought into the company.

So how do you actually will advise leaders to actually think about, you know, switching their mindset, but you know, from tech debt, which is probably more like, I don't know, like complexity or, you know, maintainability and all that into something that is more quantifiable. Yeah, I think there's kind of two, maybe there's like two ways

to answer that. I think for the broadly, I want to say just as a a general statement that I'm not saying that individual engineers should be thinking about how much revenue that their features are going to bring in, right.

That's not appropriate. What I am talking about though, is engineering leaders who have a product manager counterpart or like a business stakeholder being able to come with the same information that you would expect that business stakeholder to bring about a new feature, come to that conversation with that same information about a tech debt project.

So I think a, a way to get practice here is like, let's just take tech debt for example, which is a term that I actually really don't like because I think it's too broad. It's not very descriptive but if we could pick AI, don't know. Why don't you pick a hypothetical tech debt project? Can you think of 1? The typical one these days, especially if you're in the big setup monolith versus microservice kind of thing, you know? OK, great. One monolith versus

microservice. Like let's break up this monolith into microservices. Or sometimes what I'm saying now is like let's combine these microservices into a monolith again, because it's just, it's just too complex. But let's think about that like what's the actual benefit of doing that? So let's just say it's lower complexity. Whichever way you're going, if you're breaking up the monolith or putting things back into a

monolith, it's lower complexity. So what would we expect from an environment where developers have lower complexity code? We would expect that features can be delivered faster and so we can pick some measurements to measure that. We we might expect fewer bugs and higher stability because it's less complex. We might expect build times to go down because it's less complex. We could probably find four or five more examples. I think for each of those examples then we can actually

tie that to customer impact. So how much does it cost to have an outage? We can look at contracts. If you have an SLA of whatever, however many nines you need, usually if your service goes below that, you need to pay back some of your subscription fee. Or maybe there's customer churn. You know, there's lots of different places that you could look for. What's the actual impact of incidents and stability problems? I think the other thing here is opportunity cost.

It's a little bit more difficult to calculate, but let's say, OK, so every project that used to take two weeks now takes only 1 1/2 weeks. So we're able to accelerate our delivery by half a week for every project. Like over time, that's a lot of time saved. And of course, we can do the back of the napkin calculation on like salary, you know, just like pure monetary, like how

much salary money did we save? But let's actually talk about what you could have built instead in that time and what's the potential economic impact of the stuff that you would build instead. And so now it becomes a question of like, well, if we wait, we could do this now.

And that means that features AB and C will be delivered more quickly, but minus the cost of being able to or the, the technical debt project, actually the monolith micro service project, or we could delay it and do it in 1/4 from now, which means we know we're OK with paying that tax basically. I mean, we're talking about debt here.

We know we're OK with things moving slower, but the upside of those features going to market first before we do this tech tech project outweigh the negative parts of waiting for it or whatever. So you know, that's just a kind of simplified example of how we want to start thinking about the business impact, the customer impact, the user impact of something that traditionally has been seen as engineering's problem and engineering's kind of self-directed project.

It's not every single thing that an engineering team works on needs to benefit the business. And your job as an engineering leader is not to say, well, just trust us, we're in the code all day. It is to figure out what is the way that this actually benefits the business. And it's just something that we haven't had to do a lot in the past. So it's a new skill for a lot of leaders. Yeah. So you mentioned that I'm also guilty in the past, right.

So whenever we talk about engineering challenges, engineering project, I think we always kind of like explain it in the engineering complexity manner and then it stops there, right? We don't actually translate further into how it actually impacts the business or how can it actually bring more capability into the business, right. So I think culturally in the industry, I think it's really rare to have an organization that kind of like be able to balance between these two.

And even you mentioned in the in our pre composition as well that you have seen two different extremes, one maybe organization that focus a lot more on just business impact, you know, being able to churn new features fast, as many features as possible. And the other one is actually focusing a lot more on engineering and maybe doing a lot more this, you know, rewriting micro service and all that.

So maybe tell us about the danger of these two extremes and how can we actually balance between these two? Yeah. You know, and I think the career change that I went through in going to be a coach versus, so now I get to go extremely broad, not necessarily as deep, but like very, very broad across a lot of companies, which is a huge benefit to do like pattern matching. You know, sometimes I think about myself as like a human regular expression for engineering problems.

Like I just, I'm just like figuring out which things belong and which things don't. I've worked with companies who really have extremely hands off approach to say like, OK, say every engineer, you just like you're hired to do this job. Here's the product you work on, do the thing. And I think what we get in that situation is like a bunch of people moving in different directions. I mean, this is an extreme example, but you know, people are vectors. We have like a speed, but also a

direction. And so people can be moving really fast, but if they're not aligned and going in the same place, the benefit just isn't going to be as great as it could be. So that's on one end of the extreme. I think the other end is I've worked with some clients where I mean their engineers need to know not down to the cent, but like how much impact did that

bring to the the company? You know, I've worked with engineers who really like that's their gratification centers light up when they know like, oh great, you know my whatever optimization I did in, you know, usually it's more like e-commerce type situations. But like, oh, I optimized this page load and like now our shopping cart drop off went from, you know, X percent down to X percent and that accounts for 1.3 million in revenue and whatever.

And I'm like, it's great that you know that, Like, I love that

you have access to that. But I have also some clients that take that to the farthest extreme and say if we can't quantify it and if the number isn't high enough, then we're not gonna do it. And what that leads to is just like a slow degradation of their systems over time 'cause you know, if you cook all day and never do it, a single load of dishes, it's not going to take long until it's just very difficult to operate in the kitchen. And that's sort of what's happened.

So, you know, where in the middle is right for your company is really an exercise I will leave to the listener. It's got to be somewhere in between the two of those. Like how can you save space for self-directed engineering projects with also, you know, alignment and working toward common goals for the business? How can you have both? Because you definitely can have both.

It's just hard to know. Every business context is a little bit different, but you need to find the place in the middle that's right for your team and your company. Yeah, definitely. I think every company has a different situation or context. So maybe time as well, right? There's a pressure to increase the business impact and there's a time probably that is more relaxed so you can do more self-directed engineering projects or maybe equally balanced in any product road map.

Typically it's like, I don't know, yearly, quarterly, right? You kind of like allocate what kind of percentage to work on business related stuff and also the engineering related stuff. I think those are definitely good tips, right? So for people don't go to the extreme, right, I think it's always challenging if you just follow one approach versus the others. And I think the other thing that is quite typical these days is about developer productivity and

developer experience, right? So I remember in the past when I work in a bigger scale up, right? So this is always the bigger question to the engineering leaders, Like tell us about, you know, what kind of metrics do you want to use for engineering? And I think this is always difficult, especially if we want to bring up the topic. We want to improve our developer experience, but building a business case is always difficult.

So tell us, how can we actually build a business case for improving developer experience or developer productivity? Yeah, this is one of my favorite questions to answer because I think that developer experience and developer productivity exists in this very strange area where what is good for your engineers and what is good for your business are the same

thing. And they're so few opportunities as leaders that we get to like, do the right thing That's, you know, the right thing that like we as engineers also want to do. But like, that's also the right thing for the business. Because the bottom line is that poor developer experience just

cripples innovation. It makes work and value flow through your company to your users at very slow speed and reducing friction, not just makes engineers more like fulfilled and happy and satisfied with their working environment. It is really good for your customers and for your business as well. And so it makes sense to invest in it because if you don't and your competitors do, there's only one outcome of that situation, which is that you fall behind.

And so developer experience is this interesting thing that is sort of like the overlap of the Venn diagram right there. And I think it's such an enjoyable problem space to work in. Very interesting that you mentioned developer experience is something that both business and engineers, you know, incentivize to make it better,

right? I think this term is kind of like new in the industry, maybe, I don't know, in the past three, 2-3 years, probably it's getting hotter and hotter, right, in terms of discussions and research papers and all that. And more companies are getting into this space as well trying to understand what is developer experience, right? Because in so many different culture, different types of companies, experience and productivity can be quantified

differently. And I know that you have dealt with this engineering metrics for quite a number of years. So maybe in your point of view, right. So what are the good engineering metrics to use these days? Yeah, well, I can answer that question simply with a new framework. Actually. Then Abhinoda and I published along with a collaboration from quite a few other people. So to give a kind of a richer answer, I think I would have said it depends.

And my answer would have been it depends for a long time. I've been working in developer tooling for over a decade, you know, always trying to like make things more efficient, make them better, make them faster, make them more stable or whatever. And like how to quantify this and how to bring that to the business has always been kind of an open question.

And because every business is so unique, usually the story that we've needed to tell with metrics to make them matter to the business has also been slightly different depending on it. As you said, the last couple years, like developer productivity and developer experience are really like having their moment in the spotlight. There's so many companies that are practicing improving developer experience, like really investing in developer efficiency and effectiveness.

So we have just more data to look at more examples. And from all of those examples, we've really nailed it down to speed, effectiveness, quality and business impact as the four pillars to kind of put a story together about developer productivity and developer effectiveness. So there's not going to be a single metric. I think that's an important thing also to call out here. We've been looking for a single metric for decades now.

Maybe we thought it was lines of code before and now we, I don't know, maybe we thought it was number of commits or number of PRS. We're just not going to get to a single metric because software development is just inherently very complex. There's a lot of knowledge work, there's a lot of different processes and social systems that also feed into it. But I think those four really cover comprehensively the most important bits. Yeah, the new framework that you mentioned is called DX Core 4,

right? So it's speed, effectiveness, yeah, quality and impact. So it's very exciting to always hear new research, you know, new kind of metrics, you know, especially, right. So how do you compare this with, you know, the other metrics that were available before? So I think many people would have heard about Dora metrics, right, the four P metrics and also space framework. And I know DX also published a paper called DFX, you know, a

few years ago. So how would you differ between this DX Core 4 and the other metrics? And is it just another metrics that we have to study and implement our company? Yeah. So the aim with DX Core 4 is that these, the metrics that we have in the DX Core 4 framework are like they unify the other frameworks that come before it. And also the research that went into DX Core 4, it's all building on top of the research from Dora, from space, from the Dev X framework, etcetera.

So this is not meant to necessarily. I think there's all often it's like, which one are you going to choose one or the other? When in fact, they sort of all kind of coexist in the same universe and have different purposes. The DX Core 4 has all of the four key Dora metrics integrated in it. It is an implementation of the space framework and it uses the concepts from Dev X. So the Core 4 is sort of everything altogether, but simplified enough that it's easy

to get started with. Yeah. So I think that's a very good thing, right? So now we can consolidate our effort into this just the X Core 4, right? So maybe if you can explain a little bit about those 4 dimensions, how did you come up with just these four? Are there any kind of research or you know, something that kind of like brought you into this conclusion? Yeah. I think fundamentally the backbone of the DX Core 4 is a concept called oppositional metrics.

And, and you'll see this also in Dora and in space and and you know, many metrics frameworks. But the idea is that when we look at many metrics next to each other, we want to have metrics that are sending signal and work in in conjunction with each other, that if one goes up, we want to see the other one like also go up. And if the other one goes down, we know it's a, a signal that something is wrong.

And so, for example, we could take speed and quality because that's an often the classic trade off is like, are we going to go faster or do we want to make it good and, and quality. And usually it's seen as a trade off. And what core 4 and a lot of these others metrics frameworks suggest is like, it should not be a trade off. You should be able to move faster while increasing quality, and so we want to see those things move together.

The same for effectiveness, which is like the the measure of developer experience. How effective can developers be in the system in which they develop software? If we see speed go up but the developer experience degrade, we know something is wrong. Or if we see a really great developer experience, but quality goes down, also not good. So we want to have these sort of levers and checks and balances built into the framework.

And so speed, effectiveness, quality and business impact really cover the biggest corners of software development and keep those things in check. For the individual metrics, what we're doing is suggesting a key metric and then several secondary metrics if you want to go a bit further into your measurement. So for speed, that is diffs per engineer, these are like PRS or merge requests or whatever it's

called. This is a metric and it's, you know, it's the first one, it's the first one at the table. It's the first one I always talk about because it is the most controversial. Definitely. I think even a few years ago I would have said like, that's a bad idea, don't do it. But we have seen lots of companies use this metric to provide helpful but limited insight into the health of their software development cycle.

One really important thing about this metric of diffs per engineer is that we're not trying to see, oh, how many PRS did Henry close and then how many did Laura close? We're not looking at that. We just want to look at a developer in your organization, you know, are they closing on average 3 PRS a week or 10? You know, what is that?

What does that look like? Because that can provide us information about how easy it is for developers to develop code just like fundamentally what we're doing here, right? So we wanna keep track. Like are developers able to actually produce code that has a lot to do with effectiveness, which is the measure of developer experience in the

system. So companies that have a great developer experience have developers that you know, just don't lose time to things that are wasteful, are friction, you know, a lot of drag. They are satisfied with their tools, their processes, etcetera. They, you know, they can make room for innovation, those things. So we want to measure that and make sure that that's again one of the four pillars and that it's keeping balance with the rest.

Quality is change failure rate. That's a classic good old Dora metric. If you recognize that change failure rate. And then impact is the kind of the business number here, which is percentage of time spent on new capabilities. Thanks for sharing this overview about those 4 dimensions, right? I think definitely the first thing that always kind of like controversial is counting the

number of something, right? So this time in the DX Core 4, it suggested that we should count the number of diffs per engineer, although in the aggregate not like individuals, right? So this this diff could be the PR or merge request that we are accustomed to these days. So tell us why this is something that is more prominent versus the Dora metrics which previously advocated something like number of deployments or maybe the lead time, right time

to bring value to production. So why diffs become more prominent rather than those? Yeah. So Dora metrics are designed to measure software delivery capabilities. It's really focusing on like once the code is written, how can we get that code out to our customers? So you'll see things like lead time, deployment frequency change, failure rate, time to recover from a failed deployment, which used to be called MTTR. These are really focusing on

that delivery side of things. The diffs per engineer focuses a little bit more on the code authoring side of things like the, you know, everything that happens before the Dora metrics kick in more or less. So the activity measurements. So you mentioned the space framework before. For those listeners who haven't had a chance to work with the space framework or read about it, the space framework suggests that there's five main dimensions to developer

productivity. We've got satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Those activity metrics are usually what we think about when we think of developer productivity, like commits, lines of code. It's not a great metric. Other things like that number of PRS, for example, those all fit into the activity side of things. We're not saying like we should ignore those activity metrics. That's not what space came out to say.

They're saying activity metrics exist. They provide limited insight into how well the system is performing. Also, you need to make sure that there's all these other things accounted for as well. And so that's what we've tried to do with the core 4 is to not

ignore the activity metrics. It's still an important part of system performance, but give it more context so that it's actually meaningful and gives us information, which is why Disper Engineer not in an individual level, but not as the only number. We never want to look at that number on its own. We always want to have it accompanied by effectiveness, quality and impact. Yeah. Also not to mention that you not just have the key metrics, right, the one metric for each

dimension, right? You actually also have secondary metrics, which I think in this speed dimension, right, you also have likely time and also a number of deployments as well. So I think again, like coming back to what you said, there's no one single metric. So it's always kind of like combination of a couple of them. And the other thing that I find also interesting is about impact, right?

You're not actually measuring in terms of dollar revenue or number of user or number of whatever, right? But actually it's kind of like the number of new capabilities and kind of like new innovations probably. So why is that more important rather than, you know, the BAU stuff, right, bringing more revenue and profit to the

company? Yeah, part of this is like every business is different and I guess like when we get into it depends territory like we are not able to provide outstanding benchmarks about like what is a great business performance for your particular company in terms of like monthly active users or daily active users. That's not really something.

But what we can do is look at in general how, what's the ratio of like what's revenue per engineer, for example, for every dollar of revenue that we get in how much are we spending on developer salaries. That's something that we can kind of look at standardized and see and compare, you know, companies or like R&D as percentage of revenue.

We can look at different ratios and see, OK, how do we like how does this look across the industry, but then more importantly is like, how do we look compared to ourselves and is this number going higher or lower? We're not suggesting that you shouldn't measure daily active users or monthly active users, like please do that if that's the if that's the metric that you're looking for.

But when it comes to kind of broadly speaking about business impact and how organizations are approaching this across the industry, these are a bit more easy, like they're a bit easier and more straightforward to standardize on. You mentioned about the IT depends answer, right? I think this is quite typical in any kind of discussions about developer experience. I even have a friend telling me that, hey, in your past episodes, why is it always kind

of like it depends, right? So I think it's a more opinionated answer is something that many of us would kind of like appreciate or kind of like want to follow, right? But in this case, the X Core 4, I think it's kind of like a new new, kind of like a new paradigms. How can we actually start measuring this, maybe in our engineering team or engineering organization? Is there some kind of different mechanism that you would

suggest? Yeah. So in the DX Core 4, we'll go over the key metrics and secondary metrics, but we also have a third row, I guess or column depending on how you're looking at in that table, which is about data collection. So I'll just like I'll get right to my sort of opinionated answer. Everything in here can be collected via a survey. So surveys are super powerful. They're not just about gathering people's opinions. You can gather quantitative data

via a survey. It's self reported data, which quite honestly is high enough fidelity to make the decisions that you need to make with the data. So whether your lead time is 46 minutes or if it's between 30 minutes and an hour usually doesn't matter when it comes to what do we need to focus on and what do we want to improve. So you can gather all of these metrics with a survey and self

reported data. You might have to ask some different people for different surveys, especially for different survey answers, especially when you get into the business impact stuff because individual engineers probably don't have information about R&D as percentage of revenue, but they can tell you how much time they spent on maintenance work versus new features last week, you know, So my opinion, the

answer is start with a survey. You can get all of this via a survey and then figure out what would be useful if the collection was automated via, you know, scraping data from an API or integrating with some tool and then go build that. But don't delay getting a baseline because a baseline is going to give you direction into what you should start improving and then help you measure your progress.

And so delaying a baseline to build something that you're not even, you don't even know if it's going to be useful because you haven't gone through the process of getting the information, making a decision, changing something, measuring again to me is a a bit of a waste of time. So I would start first with the baseline, make your changes, measure them, and then figure out what you want to invest in when it comes to automated collection. Yeah, getting the baseline, I agree.

It's I think it's very important as well because in the past I've used the X as well, right. So we were actually like having a lot of challenges, right. We know we have problems like anecdotally, we can hear from engineers, OK, we had built issue, built time issue, you know, quality issue, but it's all about opinions, right? Rather than kind of like, you know, quantifiable, right? So I think get getting the baseline across different teams, across the whole engineering.

In fact, I think it's something that is super critical, especially if you're a leader, right? Because without that baseline, I don't think you would know what kind of things you should improve. And plus there are so many different dimensions and problems, right? I think it's also difficult to kind of like improve everything, right? You're gonna, you know, put focus on some certain areas rather than, you know, throwing all your effort into all

different dimensions, right? I think, I think the baseline is definitely super critical. And I think another thing that is one of the dimension that I would like to ask you is about effectiveness, right? So I think in your DX Core 4, right, you mentioned about Developer Experience Index or DXI. Maybe this is something new for some of us who haven't heard about it before. Tell us what is this DXI? Yeah. So Dxi is a developer experience index.

It is a predictive benchmark of developer experience and all that it is, is just a mean average of 25 developer experienced drivers that you can find online. It's like the DX 25, we can find it on DXS site. What we found over time, now that we have a lot of companies using DXA, lot more data is that companies that score higher in these Dev X drivers have less time wasted.

And so that again, that time waste is not just in terms of like salary and, you know, frustration, but really the main benefit there is like, what would you have built instead? What could you have built instead? Because even if like so for example, the average developer wastes 20% of their time each week on inefficient processes, meetings that are not useful to them, waiting around, waiting for a feedback, you, you name it.

I mean, if you've worked as a developer, it probably probably doesn't take you long to list out all the ways that your time gets wasted in a week. Or when you're just kind of spent, you're, you're just waiting for stuff. Even taking that down to 19%, I mean, this is like a minuscule, but like, let's give developers back just like a little bit of time every day or like don't take them out of their flow a little bit more. It does not take long for that

to add up to some. A huge amount of time of money, but also opportunity like what else could you build instead? And so that's really what the idea of this developer experience index like it's not about ping pong and beer, which is often I think developer experience has a like quite a

marketing problem. I don't know if there's a, we should name it something better, but you know, when Abby has this thing that just like makes me laugh every time he's like, Can you imagine if a new salesperson or a new sales leader, like a new CRO comes in and says, well, we're going to measure the impact of our sales organization based on sales Rep experience. Like no way. Like you would not like there's just no way that that would ever go over.

And so I sometimes I feel like, you know, when developer leaders, CTOSVPS directors and we talk about developer experience, the perception is a little bit like, oh, these whiny engineers want to play ping pong and drink beer after 3:30 PM every day. And like, that's not what this is. This is not a ping pong and beer problem. This is like a crippling, you know, this, this stops innovation and it's going to cripple your company's ability to innovate if you don't fix it.

And so putting it front and center in the Core 4 along with the other things sort of emphasizes like this isn't just a ping pong and beer, it's truly like a fundamental business problem that needs to be solved. But the DX I I guess to go back to your original question is just a mean average of these developer experienced drivers that are correlated with lower time loss, better performance and engineering generally. I think it's very funny when you mentioned that, right?

Yeah, I think it speaks true to the situation, right. So why there is a developer experience but not, you know other departments experience, I don't know like sales experience, operation experience and things like that. So I think it's very important that we kind of, I've put perspective on this, right? I think productivity is kind of like general term, right? To kind of like measure or improve. So another thing that I would like to ask you, since you have published this paper, right?

Has it been implemented in maybe some of your customers or maybe other organizations be big or small? Is there any kind of impact? So once somebody has, you know, used this framework, new framework, right? Is there any impact into how they improve their developer experience? Yeah, there's quite like hundreds of companies using the DX Core 4 right now. And I think like when you start using it, you shouldn't expect that, OK, now all of a sudden developer experience is going to

improve. What it does is it shows you or or gives you some models to figure out where do I want to focus my efforts and where should I prioritize our resources in order to fix problems, to see the impact that

we want to see. The other thing that DX Core 4 really does for an individual engineering leader is it just gives you better set of vocabulary words, common language to take the problem of developer experience and the problem of developer productivity, quantify it, but also tell a better story so that you can walk into your exec team meeting or whatever. Meaning that you have to go to and draw a line in between developer experience and innovation capability, for example, in that impact

category. I think before with other metrics like Dora. Dora is really great, but it only measures software delivery capabilities. It's not tying it all together and really bringing it back to the business, which is what we've designed Core 4 to do so that an individual engineering leader, whether you're a manager or AVP or CTO, you can have better conversations about why productivity problems or, you know, lack of investment in things like tech debt or whatever it might be, why it's

bad for the business. And then specifically, when you change something, what are the results that you should expect to see and then hold yourself accountable to that prediction? Yeah. So when you mention about, you know, not expecting sudden, you know, improvement, right. So I remember in the past when I, you know, used to do this as well, right? So after, I don't know, like two quarters, we only saw like a mini improvements in some of the

numbers, right? So I think some of these problems are very complex, especially the social dynamic aspect. And plus, you know, people change, you know, direction or maybe the mission of the organizations also change, right? And I think don't forget about that aspect as well. It's not like something you just quantify and you know, you put a lot more effort and it will be improved after a while.

So I think the other aspect is how do you actually motivate engineering leaders to actually try bringing business case for improving developer experience? And do you suggest also doing it quarterly or is it something like a yearly thing or is there any kind of a, a best practice that you would suggest us to do?

Yeah, I think always all the time, I think, you know, I don't have a great answer to the cadence part of it, but let's take a maybe a, a bigger picture answer, which is like, who are your enemies and who are your allies? So I know we're going to talk about like my tech leadership wisdom at the end of this episode. But like, you know, adjacent to that answer is like there's always going to be someone who benefits from an inefficient

system. Doesn't matter how bad something is, there's always going to be a group or an individual that's going to benefit from it. And so you really need to figure out like, why is the systems like, why do we tolerate developer experience as it is? Or like, why do we tolerate poor productivity or like bad tooling?

Like who is it benefiting? Is it benefiting us because we don't like change and it's just easier to not change than to go through the discomfort of having to change, You know, why is that? But try to figure out like, what are the, the organizational factors that are keeping the status quo in place? So that's like your enemies kind

of category. There's also your allies and those are like your Co conspirators, people who are going to, you know, just as fired up about developer productivity and about developer experiences. You are that might be another engineering manager or leader. It's really great if it can be like a product manager or someone from a cross functional function or a senior individual contributor, like staff level, principal level, like those are

really great allies to have. Because then instead of it being you going to your VP or your CTO and saying, hey, we want to do X, it's like a bunch of you. And there is strength in numbers. There is strength in numbers. I would also look for the person at your organization who's going to sponsor this. So like an exec who is going to pave a path for you. You know, you might need more budget, you might need resources or headcount.

You're also going to need sort of like a mandate to be able to get everyone on board with these types of projects being on the road map, so to speak. And so who's gonna sponsor you? And then concretely, what are, you know, if you do establish this sort of like working group, like what are you really responsible for? What are your KPIs, so to speak? And that's really where the core 4 comes in, which is like, how do you measure the impact of what you're gonna do?

So that's where I would encourage if, if an engineering leader is like interested in pursuing this is like first figure out like your, you know, like your tactics on the ground, like who are you working with? What are the factors working against you? What are those objections? Who is an executive sponsor that you can, you know, bring this up to and try to kind of make a plan and then pitch it that way

and then take it from there? Very interesting advice you have there, you know, figure out your allies and kind of like your detractors, right? So the one who benefited from the inefficiencies or maybe status quo, right? Sometimes we don't like changes, especially if it's too drastic, you know, coming back to the example of monoliths micro service or the other way around, right? It's kind of like a big change, right?

It's something that not everyone is incentivized to make it successful or make it happen, right? So I think figuring out allies and work together in order to come up with business case definitely is a very, very crucial. Another important thing that typically is always discussed when we talk about developer experience. Lately, it's about introduction of internal developer platform or even AI these days. So maybe in your view, do you have any take of these two,

right, IDP and AI? How do you see them as a core components these days to improve developer experience? Or is it something just nice to have for some organizations? Yeah, I think developers see it as a core component. And So what I'm interestingly seeing, I've had quite a few conversations about this in the last few months, is like, how do I as an engineering leader, make a business case for investing in an IDP or investing in Copilot, for example?

Especially with AI, some of the promises are very lofty. Let's just keep it that. Like, let's just say that diplomatically. I mean, you know, I don't know who really believes that, but like, AI in general, you know, is just sold as like, oh, it's going to make everyone twice as productive and you can reduce half your staff or, you know, whatever, whatever the promise is. That's just not really what's happening. It does reduce cognitive load and make developers work faster,

help them work faster. But you know, it's still, it's a tool that needs to be operated by a skilled individual and using an irresponsibly can lead to more problems when it comes to maintainability and creating more problems for yourself. So figuring out what purpose these tools should serve and then figuring out a way to measure that impact and tell a story with numbers and make a business case for it again is a

very important thing. So, you know, one thing we haven't really touched on is like attrition or being able to attract talent. A lot of developers, if you ask them like if they were interviewing for two similar companies and one of them had a copilot license for them and the other one didn't, most developers are going to go work for the company that has a copilot license for them. And So what does that actually cost you?

And you can figure out the numbers like you might have to ask some other people, but like how long does it take the average from job requisition to be posted online for that role to be filled? Then how long does higher like training take? Like on boarding, you can figure out how much expensive that is. And then how expensive is it when someone quits and goes works for someone else because they're not happy with developer experience. Like we can put a number on attrition pretty easily.

It usually it costs like 75 thousand U.S. dollars, I would say at a minimum to replace an engineer like that's mid level or above. And that's not nothing. You know that that's pretty significant when it comes to like what are you going to pay for copilot licenses? But these are some of the ways that I might put together a, a narrative around something like Copilot or, or an IDP.

It's not just about like what's the tangible benefit in terms of productivity gains, but also let's look at space. What about satisfaction well-being? What about communication and collaboration? What are those other things that are positively impacted by these tools being in place? Yeah, thanks for highlighting that. We have to understand the purpose before we implement those kind of thing, right?

Because many of us, we read from the research out there, people raving about, you know, IDP or AI, right? And we just follow, right? Implement it without actually understanding the the impact or the purpose or the even the the things that actually sometimes like in the past episode, I have this Andrew Boyaji, right? So he highlighted the state of developer experience report. So he highlighted that leaders are just following, you know, maybe the trends, right?

Implement AI and thought by doing that they actually can improve developers, but developers thought differently. They just find like maybe a few percentage improvement, but it doesn't really actually necessarily improve the whole developer experience. So understanding the true purpose behind implementing those I think definitely is a

key. So we discussed a lot of things, you know, maybe is there anything that unintuitively you figure it out but many people are not aware of in terms of developer experience, is there something that you want to highlight? So for us who may not have heard before in the industry or in the research these days. I think that maybe the most surprising thing is that your teams just, they know where the problems are. They work in these tools all the time.

And I don't care what metrics framework it is like if it's the TX Core 4 or Dora or any combination of metrics aligned

to the space categories. Do not expect that capturing workflow data and looking at GitHub data, JIRA, GitLab, whatever, is going to point you to some like unknown about developer productivity that like suddenly now that I have all this data about commits and story points and number of PRS and how work is flowing through the system, I'm going to be able to find this magic bottleneck that I'll be able to fix and like unlock productivity for my team.

I have never seen that happen. I have worked with literally literally thousands of engineering leaders at this point. I have never come across a single case where engineering team, let's just say, has discovered something new based on workflow data or quite honestly even like survey data. The thing is like your team is in these, they're in the tools all day long. They know where the problems are.

But surveys can be so helpful to just tell you how big the problem is. Like we know the problem is there and then we can put some kind of formality around measuring how big the problem is and if we're fixing it or not. So I think this myth of like I'm going to implement, I'm going to get some developer productivity dashboards and like suddenly I'm going to be able to forecast and see ahead when my team has

productivity problems. Or I'm going to kind of find this like secret bottleneck, like it just doesn't exist, doesn't exist. Your team knows where the problems are. Just believe them because they are professional adults who do not have incentive to lie to you unless you give them incentive to lie to you. So gaming the system isn't actually as as big of a problem as we think it is as well. That's another surprising thing. Wow, I think thanks for highlighting that, right.

So I, I, I left when you explained all that because, yeah, I think many people try to automate, you know, you know, getting the workflow data, getting metrics out out-of-the-box, right? Only to find that if you ask maybe the engineers or the teams, right, you could get the signals master. Other than spending all the effort to actually come up with the dashboard, maybe.

And in the end you kind of like, no, that's the the like the problem exists from some people who may have shared it with you. So you don't need to actually spend a lot of effort. So Laura, it's been a pleasant conversation. So I think we all learn a lot about improving developer experience. So as we reach the end of our conversation, I have one last question for you. I call this the three technical leadership wisdom.

So you can think of it just like advice that you want to give to us. Maybe if there anything that you want to share with us today? Yeah, I have so many things. I'll try to keep it to three. On the topic of metrics, I think the saying that like all systems are wrong and some are useful is

really important. And so I would encourage anyone listening to have a healthy dose of skepticism for any type of data that you're looking at, making sure you understand where it's coming from, understand the purpose of it, all the things around it. Because measurements are going to be wrong. Even a broken clock is right twice a day. So just don't let data push you to incorrect conclusions. I think that's a very dangerous area in this developer productivity metric space.

Yeah, I think, again, as I said before, things work poorly because it sometimes benefits us to have them keeping, like to stay working poorly. Maybe that's ourselves and we're the enemy there because we are resistant to change. And maybe there's another factor in there. So I would figure out what those things are. You know, you'd really try to debug the system and maybe to follow on this like, you know, metrics thing.

A lot of us that are, you know, were previously individual contributor engineers are going to try to pattern match or bring a lot of the skills that we had as an engineer, for example, debugging skills and try to debug our teams and organizations in the same way that we debugged like our Kubernetes clusters or whatever. The thing is like, your nodes can't talk to you and they don't have feelings, and talking to people because your teams are made of people isn't being

illogical. It's actually the correct way to approach measuring a system that's made-up of people. If we fail to recognize that people make up teams and we try to treat them like a Kubernetes cluster, we're actually missing sort of like half of the whole system. And I would just say that's sort of categorically incorrect to approach it that way. So make sure that you're not forgetting that software teams and software development is a

socio technical phenomenon. So we need to have the social aspect and the technical aspect together. Both of them are equally as important. So don't forget that humans exist. Developers are adults. They're professionals. They don't have reason to lie to you. Treat them as adults. Believe them when they tell you something is causing them friction in the deploy or the development process. Wow, that's really beautiful.

I don't know why when you explain that, right, I always come back to this phrase, you know, pet versus capital thing, right? So I think for engineers, we try to make everything kind of like uniform with big container Kubernetes cluster, right? And we treat everything like capital, right? But actually all individuals are different, right? We have our own needs, our own challenges, right? And don't forget about the people aspect. So I think that's really

beautiful. Yeah. So, Laura, if people want to follow you or you know, maybe discuss with you further about some of these topics, is there a place where they can find you online? Yeah, go to laurataco.com. That is where you'll find more about me. My blog. I do have a course on developer productivity metrics. I'm going to be running it more frequently, so you can check that out as well. And you can find me on LinkedIn. Just search for Laura Taco. I think I'm the only one.

Right. I'll put that all in the show notes. Thank you so much for your time, Laura. I think we all can learn a lot about developer experience from you today. So thank you so much for sharing. Wonderful. Thanks for the lovely conversation.