From localized to systematic speed: How Spotify deploys AI in prototyping, strategy & maintenance w/ Tyson Singer #245

So as we go through its AI transformation, everybody knows it's like making some task. Easier, faster, and they're starting to disappear. Now that's great, but we want to think bigger than that. We want to think actually, how does it disrupt the paradigm of how we do the work? And to do that, we need people from all the different parts of our business that make our business work come together, reason around that, think about that, and be supportive.

Hello and welcome to the Engineering Leadership Podcast brought to you by ELC, the Engineering Leadership Community. I'm Jerry Lee, founder of ELC. And I'm Patrick Gallagher, and we're your hosts. Our show shares the most critical perspectives, habits, and examples of great software engineering leaders to help evolve leadership in the tech industry.

Everyone right now is focused on using AI to quote unquote build it faster. But at Spotify, they're looking at the other ends of the product development lifecycle, prototyping, strategy, and maintenance. In this episode, Tyson Singer, SVP of Technology and Platforms at Spotify, joins us to unpack how Spotify is transforming its product development lifecycle across creation, experimentation, and maintenance to shift. From localized speed to systematic speed.

So in our conversation, we explore why the industry's current obsession with this build-it phase of development is short-sighted and how Spotify is aggressively deploying AI in prototyping, strategy, and maintenance. Tyson breaks down the internal tools driving this shift, including ICA for context engineering and honk for automated fleet management. Tyson breaks down the internal tools driving this shift at Spotify.

We also discuss the looming bottleneck of human code reviews and how probabilistic reviews might be the answer. Let me introduce you to Tyson. Tyson is SVP of technology and platforms at Spotify, where he leads technology infrastructure, developer experience.

cybersecurity and finance IT. At Spotify, Tyson leads a global team of over 900 engineers, product managers, and designers. He's also the executive behind Spotify's internal developer portal, Backstage, and Spotify's experimentation system, Confidence. The product development lifecycle is evolving rapidly. So if you're looking for strategies that go beyond simply build it faster, this is the conversation for you. Enjoy our conversation with Tyson Singer.

Tyson, first off, just want to say welcome. Thanks so much for joining us on a Tuesday morning. How are you doing? How are things going in your world? Uh things are going great. Uh we're wrapping up the year. Uh hoping to go on a little break and decompress in a in a bit over the the winter holiday. So many things.

uh going on at the moment. Uh I think a lot of it this year has really been driven in by the the AI transformation. I mean it's sort of getting ready for the next set of transformations that are gonna go on uh next year.

Well, and I think what's fun is like Spotify kind of wraps up the year in a really big way. Like with like Spotify wrapped and everything like that. And I think for l for my social circle at least, it it sort of marks the beginning of the holiday season and the beginning of like reflecting on your year to come. And sort of kind of creates this like really interesting bookend. So um it must be really interesting from

from a team perspective, to have that be like a big milestone endpoint and then sort of open up the next season for people. Yeah. Um we have some fun at Spotify and uh we share a lot of that fun, I think, with everybody, the the wrapped parties. I don't know if you found that feature in this year, but you can, you know, get together with a lot of friends and go through your wrapped experience and it matches people well. This is always super embarrassing for me because well and it's working.

at Spotify. I'm not like a power user of Spotify. And so then people see like, all right, great. You listen to a tenth of the number of artists that other people did and all these sorts of things. So I get a lot of rousing at this time of time of the year.

Absolutely. I feel like the th the item that was really fun for razzing was sort of your your listening age. Oh yeah. Uh and and that was a r that became really contentious for a lot of my my friends as well. That that was awesome. I was seventy seven.

And I I'm not seventy-seven as you might I was twenty-four and I was extremely surprised. I was like the I'm not listening to, I don't think, like particularly youthful music. So it was it was one of those fun things. But you know, Tyson, in our conversation, there there's sort of a few directions we I know we want to go.

And you know, your focus since you joined Spotify has been enabling and helping the engineering org evolve and scale. And you've seen sort of different, I think, iterations of this or in different like distinct moments of this.

So maybe I was thinking we could unpack that journey a little bit first and set some context and then start to talk about how Spotify is evolving, some of the different areas that you're focusing focusing on and thinking about. So talk to us about your journey. Like what's your journey been at at Spotify so far? Um and how's it starting to frame what you're focusing on at Spotify right now? So I've been at Spotify for about nine years.

And what I discovered very, very quickly was that Spotify is a bit of a crucible. As I mentioned before, it's like a playful, fun place and there's really nice people around, but like it's very intense. So when I joined, we were growing the the engineering and RD teams by thirty to forty percent year over year for a number of years.

So we're seeing this like incredible growth in seeing that both in terms of the growth of our business and the growth of the the staff. And so we needed mechanisms to help manage that scale and to continue to drive our innovation with speed.

So I started out actually leading the experience organization, which is our con uh our customer uh facing uh experience organization, but quickly pivoted into leading an organization that we call platform, which is really about supporting the productivity of all of the workforce with a very significant focus on uh the RD organization and needed a a bunch of structure really for a company that was not structured at that point. It was all about autonomy and sort of moving fast in the small, but

things were starting to slow down. Yeah, because everybody was doing things very differently. And all that differentness was resulting in a lot of tech fragmentation, a lot of duplication. So while local teams may have moved fast, when we started to do bigger things and try to coordinate those, really it started to to slow us down.

So for the platform organization, we needed like clear guidance on how to take us forward. One of the sort of simple things I did, I like alliterations, I said like, let's articulate what we need to do. and came up with this thing called S cubed, uh which is speed Scale securely. That was a lot of what we were doing at the time. Now scale was uh happening both on our business.

We actually didn't have much of a security team at that point. It was a few folks that came out of our core infrastructure team. There wasn't a lot of structure around that. And the most important thing was speed. On top of that, that's always been uh important for us and continues to be important. But what we needed to do was pivot from this like localized speed to the systematic.

speed. So there was a lot of change that we had to sort of lead the organization through. One of the things that was really early in my journey was looking at our mobile code base. So we have a surprisingly complex uh app when people

say like, oh, you know, you work at Spotify. You know, back then it was like, oh, you you have maybe a hundred people, right? Or whatever. It was like we were already well, well beyond that. People don't understand the complexity in there. And so we have one of the largest We had even then one of the largest mobile code bases. And it's just continued to grow and grow. And it was chaos.

Like there it wasn't it there wasn't a lot of like really structured engineering approach to that, breaking down the software into components and and whatnot. It had a really good release process at the time. So we're releasing it relatively well. Um, we we sharpened that up. But there was a lot of work to just like simplify the architecture, uh support the tooling around there so that as it was growing, we could continue to move fast.

So that was one of those things that early in our process we saw. And then we were just seeing the same thing happening in our back-end ecosystem, our data ecosystem as well, where we needed a lot more structure around those ecosystems.

This quote that you shared, I think really captures where a lot of people are starting to focus. Their time on right now, where you you mentioned this idea of pivoting from localized speed to systematic speed. And when I think about how AI is impacting.

product development and the product development lifecycle. I in like the last like 18 months, I would say like the earliest iterations of it are like people applying and using different tools to sort of accelerate their productivity. But like right now, it seems like the major focus for people is to shift from, I guess like that version of localized speed to

how do we use this to impact our entire system? And so as you were kind of sharing that major, major moment, I'm like, ah, like there must be principles or lessons like from that initiative that map to this kind of dynamic challenge. I don't know if if I'm

Making uh sort of assumptions or or anything there. Is that is that something you've noticed or Yeah. So and maybe it's worth mentioning some of the principles that we came ba up with back then. And then we can talk, you know, as you mentioned, we can talk a little bit about how bottlenecks and how that sort of fits in. But like starting from a principled perspective has been really important to the way Spotify has worked.

We put in a set of five strategic principles to guide the platform organization. One of them was we partner with our customers. So instead of just like we're customer centric, we partner with our customers because this was really around driving adoption and change of both my organization to make it a more customer-centric organization, but also drive the change because culture of autonomy that I reference.

Was very, very strong. My role was not to have all the technology folks report up to me. The joint point was a CEO. We don't have, we don't have a CTO. So I needed a way to really drive change in that. So I had to use a whole bunch of tools. And one of those tools was to have people embed in my organization and go out and embed in the organization and get a lot of the technology standards both.

worked out and adopted by that that embedding. We have this other one, which was sometimes contentious. We call it we take the pain away, sometimes it's in parentheses, but like we want to ensure that the feature teams aren't Taking a lot of the pain of things that people hate, like migrations. And so that helped us uh really focus our attention on how we can prevent.

the cost of all those migrations. And then I mentioned speed before. Speed's always been really important and accelerate those from a handful a year to like literally setting that to orders of magnitude larger, like hundreds. of of migrations, which eventually led to our fleet management solution, which today is also tightly integrated with our AI tooling. So that's sort of another example. Maybe one other we have this thing which is Uh this principle uh we call we ship one platform.

The core concept behind that, which everybody I think talks about now, but I don't know everybody was talking about then, is to reduce the cognitive load on developers. So it's a pretty common part of how people talk about platform engineering now. Uh, but it wasn't then. From that principle we can draw a pretty direct line.

to our IDP backstage, uh which was like, hey, how do we take all the complexity in the developer ecosystem and still it down into a single pane of glass that helps people you know manage and understand uh their software ecosystem in the sort of runtime view of things and then also kind of control the fragmentation, understand the fragmentation, visualize the fragmentation in that ecosystem. So those are some of the things that we started with, which continue to actually inform us today.

Well, I was gonna say as soon as you start to share these and and some of the examples of how they manifest, I'm like, oh, I I totally see the direct map to how these can maybe manifest in AI driven product development and developer productivity, like different elements like that. And so one of the questions I wanted to ask was like. I particularly admire and respect

Spotify's thoughtful approach to pretty much all different ways that you operate. And I've had a chance to talk with a a lot of different people kind of looking at that thoughtfulness across a couple of different use cases and and launches. Particularly like right now, the level of thoughtfulness around how product development and engineering workflows and operations change. I'm like,

There's a there's a lot of need for that right now and to understand how people are thinking about it. So taking some of these principles, like how are they starting to show up in more AI driven product development lifecycles? Or what are some of the ways that you're seeing this sort of map? to n how things are evolving for how we build right now.

Yeah, so maybe that's worth mentioning. sort of the very I would say simplistic framework we use for thinking about the product development lifecycle. We think about it as a loop and like everybody else, we think about it as sort of the inner loop and the outer loop. But we also use a like a different terminology, which people may be familiar with, which is just think it, build it, ship it, and tweak it. And in that last context, I would mention kind of tweak it also includes maintain it.

In an AI context, there's just a Ton of folks focusing on the building. context right now. And it's been super successful everywhere, I think. For Spotify, we've doubled the amount of code that we have now in the last two years relative to the previous 18. So we are building code like mad and and we haven't really grown our engineering base in the last couple of years.

That the tooling around that is super successful, but you have to think about the full cycle that's going on there and how that really impacts. what's going on because just take away some tasks that bottleneck moves to to another space. And so for us, we're thinking about really that the full value stream and looking at the ends. Yeah. No, there's it's a cycle, so maybe there aren't any ads.

But you get my point. So on the the think it side, everything we do starts with uh experimentation and learning built in is one of the like core values we have. I've talked in previous podcasts with some of our folks who are really focused on on that. The way we we look at that is a lot of the time that is actually spent in uh developing a product is actually spent in a research, design, and product management. Then once you have those ideas.

Testing whether those ideas are out there. You know, there's a lot of products out there like lovable that's helping you get to sort of take your very ambiguous ideas and removing that ambiguity down into a like a really core concept. We have found that in order to do that effectively, we have to build our own AI prototyping tool that brings in Spotify appropriate context.

to that uh environment. So off the shelf tools aren't quite good enough for what we need to do because if you really find that we have to get high fidelity capabilities out in that context. So that's becomes like a job of my team, which is like great, create the the tooling to help with that end of the product development cycle. Uh we don't just think of ourselves as, hey, we're helping out engineering a lot of

you know, platform engineering has talked about engineering, but again, we're trying to think about the whole flow. So, you know, our R D process, you know, we've got these folks who are really trying to do the initial process of going from ambiguous To less ambiguous. Uh and then going through our experimentation platform, uh, which you've talked about, confidence. Is our external product there?

So that's our kind of one side. And then of course on the in the middle, you need context to do a build-out. Uh and so we've got a product that we've uh included in backstage and our on our portal product that's associated with that, which we call ICA, which helps. Provide context.

not just to humans, but to the bots in that ecosystem around the Spotify way to do things and the Spotify software catalog. And that's really helped in a lot of different ways and love to talk a little bit more about that. And then sort of at the end, you've got your your maintenance activity.

Your maintenance activity really is sort of the what we look at as kind of the next set of bottlenecks that will come out there. So double your amount of uh software uh output in a short amount of time these double amount of products, which means the double amount of things to maintain. But building software is just not that hard. Uh maintaining software that you've built over time and improving it, that's where the challenges come in.

This is a great scaffolding to dive into a few different elements here. So I think from like a workflow perspective, what are some of maybe the near-term ways that you all have experimented with and evolved? the workflow around thinking it or or that ideation phase and that that research phase. The sentiment I'm I'm guess getting from folks in our community is that like they've experimented with the build-up phase like you like you were talking about, but you can't uh really automate the elegant

thinking and research phase. Like you have to do the cognitive labor to get to a good product idea. And like you can't really hack or accelerate your way to that. Maybe there's some elements around researching, but I'm curious about like, is that research

and ideation phase changing or evolving or other short term sort of experiments that you've done uh in terms of like the workflow? I guess how is that how is that particular part changing? Yeah, I I see it actually changing subtly all over the the organization. So that AI prototyping tool that I mentioned is really the built first and foremost for our consumer experience teams, uh, because they're the ones who really need to do the tough work of that.

So they're using it very, very early in their strategy articulation. So before we spent a lot of time on strategy discussions and strategy documentation. So typically what we would see is like people would write docs and they'd talk and all that sort of stuff. And now that visualization part, since it's so easy, is starting to go right into those things.

And again, that gets you to more clarity and context in your strategy because you can visualize it so much faster. So that's like one way. And then in our sort of internally facing teams, I see my teams using the exact same tool. to do this ideation around how they're going to improve the tools for our Spotify users as well.

So they're using it. It has our design system uh both for our external products and our internal products built into it. So it knows how to generate things that are high fidelity and correct in the the Spotify context. And so you see it uh like manifesting very clearly in how people are working already. I think it's a really interesting interesting look because it's both internal tools and external products and it's changing the nature of the strategic discussions that the team has together.

Maybe the question is how to map this?

to other teams and other organizations. Like, do you have recommendations for like when you're changing a workflow in this type of way to facilitate this or operationalize this type of of ideation workflow? Yeah, I'm always uh reticent to provide uh recommendations outside of the context that that we're in. So I mean, I think that it maybe it just distills down to figuring out how to get the right context for your business into those flows uh and to do it early.

That's kind of the the the simple part of the answer. I mentioned this uh tool that we built internally called ICA. It's uh it stands for AI Knowledge Assistant. And uh initially it was only in a sort of chatbot format. And really the value of it was a lot of context engineering to ensure that it understood all of our internal knowledge base, our code knowledge base, our documentation knowledge base. And uh, et cetera, what's going on in Slack, et cetera, all those communication channels and

to represent that just through sort of the chat bot inside of our backstage ecosystem where the engineers were operating, got really fast adoption across the organization. But then that was sort of stage one, which is providing that context to the humans. And well, now, of course, our humans use agents to do some of their strategic work as well. So they need that plugged into whatever they're doing. And you know, some of our product managers and designers are using agentic tools.

whether it be Claude Code or or whatever, to do part of their work. And we need that to be plugged into that ecosystem as well. So now it's more of a an MCP infrastructure and provider for all of those tools and providing that context. And that kind of sounds Maybe obvious and simple at this point, but it's not in order to get really high quality results. You actually have to do a fair amount of work and iteration to get that right so that the answers that are coming back are quite valuable.

In looking at the framework that you share with us in terms of Spotify's framework for building. When you were talking about like the build and maintain element. So I guess the end point of the cycle. I need to continue a cycle, but like the end maybe the end point of this of this loop, like how is that workflow Evolving, like within the context of some of the tools that you mentioned or how the teams are operating. What are some of maybe like the short term shifts and evolutions going on there?

Yeah. So one of the things I mentioned early up front was uh this idea of fleet management. So we have well over a hundred million lines of custom code uh that we have to sort of manage at this point. And in order to really get the feature engineers not bogged down by that management process. We want to make mass-scaled changes to the code. That initially started with very simplistic things about like, oh, upgrade a library, you know, make fairly well-defined code changes.

But now that we've supplemented that, we've built our own uh framework. Uh originally it was based on a s uh open source uh framework called Goose, so we called it Honk. to uh represent that. Now we actually don't use the the goose part anymore, but we've tied that into the coding agent. So it's basically part of our background coding agent ecosystem to make really large scale changes across our coding base.

And to do that in very sophisticated ways. And that has really become an important part of this sort of tail cycle in the sort of the tweak it, maintain it uh context. And I gotta say, engineers love it. You also mentioned sort of the the middle part. In the middle part, we built out backstage to be a lot of the thing that was important for the middle part.

Captures the SAF software catalog, captures the relationship of the catalog towards ownership. So Spotify has a very ownership-oriented culture. Uh where if you build it, you operate it. Uh we don't really have an SRE uh based or organization. Um we have this as I mentioned before, very autonomous culture that still persists. in a in quite a good way, which means that discovery and understanding of that software ecosystem is really, really important.

And uh in the AI world, the standardization of that has become even more important. So we've always believed that um defragmenting our technical ecosystem has been an important thing to do and that backstage is a really great way to. Define the standards, visualize uh whether people are uh aligning with those, but within the new AI ecosystem, what we've discovered is that those standards actually haven't been enough.

Because as the agents come in, they still have some of the same flaws that we do as humans. They only can understand so much context. When you give them a context w with so much variability, they don't know what the right choices are to make. And so we're finding that we have to add more and more constraints into our sort of build it phase and the the the the middle to allow the agents to do work In a way.

that we feel will actually result in long term success because I I think we're not unusual in this. Like as we look at a lot of the metrics for code quality in our ecosystem, we're starting to get a little worried, right? There's a lot of code being created uh very fast. We have something called a maintenance in index that we look at. I forget all the details of what goes into that. It's sort of slowly going the wrong direction. We see that PR sizes are going up.

And there's these indicators that the next downstream thing that we need to worry about is like the high quality operations of things. And so we kind of want to get ahead of that. And the way that we think we get ahead of that is to really have a lot of rigor and a lot of constraint. On how we do coding to ensure standardization and clear guidelines for all the engineers and all the agents and how they do the work.

You know, we're talking about sort of right now in the moment, some of the ways that R D is evolving, some of the workflows and and how AI is playing a role in in this type of system. And I'm starting to like maybe take us to thinking about how this is going to change or like the next

challenges that you're thinking about and some of the operational changes that you're preparing to shift in order to prepare for that. And I think this code quality element is a big challenge that a lot of folks in our community are facing. When we did our conference, in September, every organization we did which was how are you, you know, becoming AI native and changing your workflows.

Almost everybody was like, What are you doing about code quality? And every single person who were spotlighting for this was like, That's the next thing we gotta figure out. We're preparing for it. I don't know. And so I I think like when you're thinking about the future and how these workflows are changing and maybe the next

next phase change that's that's going to happen. Like where is this going and what are some of the experiments or ideas you all are discussing to help prepare for that next shift or prepare to maybe address something like code quality. Yeah. I mean I this is also one that I know everybody's uh struggling with and this maybe isn't the next one, but I think we started on this journey almost two years ago and it was the PR review challenge.

And Sarah I'm sure other folks have talked to you about this. But you know, we we we kind of anticipated this uh because we are a very metrics-oriented uh organization. I have a I have an insights team specifically focused on qualitative and quantitative insights. Inward towards our RD organization. So we have lots of dashboards that track all sort of the details there. And so we started to see some of these signals around the speed of our inner loops and our outer loops.

The fact that PR times were starting to increase. And so we're like, all right, clearly we're gonna have to solve this. And uh so we made uh an initial attempt to solve it with some basic AI capabilities. I don't know, maybe it was a year and a half ago. It didn't work. We've tried again, we're going through kind of the third iteration of those things and I think what we've needed to do is just Step back a little bit and think about, well, uh a few things. What is the purpose of a code review?

And what is the nature of how co creation is changing? So there's I'll bring those two things together. Uh one, you know, there is uh the sort of technical quality aspect of it um that I was talking about. You need those and those better standards. Part of the code review is to check those to make sure nobody put in some silly security vulnerability and and all that sort of stuff. So that's like getting all the ildies right and the Like did you do the business facing logic correctly?

when we put that in a context of having background agents making very substantial, or maybe even foreground agents making very substantial code changes, uh, and the PR is getting larger and larger, then you shift towards thinking about this maybe more as Well, how do we solve each of those problems separately? And can I solve some of those problems even before the code review process goes in place?

through the agents actually generating the right code. And then you think about, well, is this more of a stochastic problem, which is when you get to the the code review part and you want an agent to do the part of the code review, they may uh evaluate the code and say like, yeah, probabilistically, don't worry about any of this stuff. It looks good. Here's the two or three percent of the code that we want the human to actually look at.

instead of looking at all of it because we can see those patterns. I think for the the first problem in terms of the technical quality, that's going to be maybe the easier part to uh to flesh out from a stochastic perspective. And then we're going to have to layer in some logic on the business. Side to say, like, if the code tends to have these patterns, then more than likely you need to start to call attention from the human to look at.

So like we need to do a lot of that automation in that in that context, but we have to do it in a like a thoughtful way.

I know your goal is to make all RD teams effective. So I was wondering if you can maybe zoom back out and talk to us a little bit about some of the bottlenecks that are changing around this goal of making RD teams effective and the process for how you start to identify and shift through those bottlenecks. Yeah, a lot of thoughts on this one. So I'm gonna really zoom out and look at it from sort of how we operate Spotify and then kind of break down towards the the center there.

So uh we have a program uh board. Uh we don't call it programs, we call it uh our company Bets Board uh runs six month six month cycles roughly. The reason we call it bets is because you know we're just kind of acknowledging that the context we're working in was highly ambiguous and we don't know what the outcome is going to be. We stack rank all of those bets. And typically what I've seen a lot of companies do is they hide a lot of their IT or productivity improvement work.

outside of their big program board. It's sort of hidden in the dungeon. And we do a little bit of that. Like we sort of ring fence a fair amount of my team to do that type of work. But this is really important work. And so we don't do that for a lot of our work. We have like two types of bets. We call them P0 bets and P1 bets. The P0 bets are really about our technology health. We call them the the P0 Tech Health Bet.

And they're prioritized by myself and this committee of the top tech leaders. And they get priority over everything else our executive team puts out there. And that helps us. really move the bar with respect to ensuring that we can do things that help with the the the bottlenecks and the challenge in the technology ecosystem. Uh we had a pretty severe incident.

earlier this year in our data ecosystem. And what happened was we had this very subtle corruption in a few of our important upstream data sets. That started to poison the rest of the ecosystem and our ecosystem is very complex. You think, hey, Spotify, you've talked about this amazing backstage thing you have.

It captures and shows like your data lineage and you know, shouldn't you have just been able to see where it was all going? And the answer is yes, except for not everything was in that ecosystem. And you really need everything to be in that ecosystem, otherwise You don't actually know how far the poison has gone. The result is that like this really disrupted our business and c and in some sense became a temporary bottleneck. In the the ecosystem.

And it sort of got back to us and now we have one of these P zeros where we've decided like this is so important because it can be so disruptive, it can s impact the business and kind of goes back to this like idea that you really need to have alignment on standards. to do that. So, you know, we do that with our P Zero Tech Health Board, and then we also do it on our P one bet.

Where we don't just have sort of business facing things. We have very strategic programs that focus on how we improve the effectiveness of the company. I think a lot of companies are doing this right now. We have uh something we call our AI momentum uh bet, which is focused on how do we transform the workforce with AI.

To make them more effective and productive. But that bet is not just about RD. It is about thinking about the entire system. So at Spotify, we can be moving as fast as possible on the RD side. But if our legal teams and our licensing teams don't get the right that has just as b big of an impact on uh our ability to launch as anything else. And we we operate in a very, very complex legal and licensing ecosystem. And so applying AI into that can be even maybe more valuable than I'm not

Um I don't actually believe this maybe, but in the PR like review problem. Like not they're they're hard to equate, but like it can be. Well, maybe it is actually. So again, my job is bigger than that. And we have to think about how we provide technology to everyone and think about what is the biggest bottleneck for company value, not just great, how do I make the RD organization go faster? So that's That that is how I think about it. Well I I think in this question of identifying bottlenecks.

And then applying the RD team either internally or externally to address that, I think is really interesting, especially in the context of this question of where is the bottleneck to company value. And as you're describing the Spotify ecosystem, like you're entirely right. There is a really complex level of

stakeholders and roles and functions that are contributing to sort of this end product experience that people then consume and use and listen to. I was wondering if you could talk a little bit about like the process to go through that question of where is the bottleneck to company value and to expand out outside of R and D and to start to think about some of these other functions.

The reason being, I think a lot of senior leaders in our community are not just the head of engineering now, but they are essentially the head of AI transformation for the internal operations of the company. So I think this question becomes more and more real and material for people to be able to wrestle with.

And I'd be able to help support some of these different things. So in this question of where's the bottleneck to company value, like what does it look like to start to explore that question outside of the context of RD and engineering and some of those specific workflows?

Yeah. Uh maybe I can tell you a little bit about how we've structured our AI productivity program. So as I mentioned, it is a company program and it has a steer co. And one of the worries I had as we I was putting together this STERCO was it to be actually Overweight by R D folks. So it really made an intentional push. to include the head of HR. She's actually my co-sponsor uh for the change because this is not about technology by itself, is about people.

You know, if you're gonna do be a system thinker, you can't just be like, oh great, I changed some tools and then no one uses them, right? So uh it's her and I who who really need to be partners on changing the company across the board.

systems people, technology. They have to be thought about together. So HR, legal, finance, and then of course a lot of our functional leads within R and D and people who have different like lenses on the business. We have somebody who's representing our marketing team. So that that group can get together and we we we actually call it the new X Playbook is the name of the the that

Because we're basically saying like rethink about these workflows and playbooks that you use to operate our business. Don't assume what we have is the right uh thing. So as we go through it's AI transformation, everybody knows it's like making some task. Easier, faster, and they're starting to disappear. Now that's great, but we want to think bigger than that. We want to think actually, how does it disrupt the paradigm of how we do the work?

And to do that, we need people from all the different parts of our business that make our business work come together, reason around that, think about that, and be supportive. One of the quotes you shared, we have touched on on this podcast very slightly, but I think you captured it and crystallized like this area, I think so powerfully, which is that systems change is about people.

Because when I've been talking to a few different folks about bottlenecks and things like that, it's like, yeah, technology is accelerating, but like when you get into some of these other different areas, it's like people's capacity for change is becoming the new bottleneck. And to that the thing that really significantly reduces speed is like can somebody learn fast enough or evolve how they work fast enough to keep up with the capabilities that are shifting?

Have there been any particular insights when it comes to people in systems change, like through this AI productivity program that have been interesting or you've started to formulate maybe a a framework or an approach or a way of thinking about that? Yeah, I I feel like we're still in the midst of figuring all of this out.

When I got to Spotify, what I saw was we had our you know engineering organization very, very strongly set up on what I would call an an I shaped engineer. I'm a mobile engineer, I'm a back end engineer, I'm a data engineer. That was problematic.

to the speed of our business because we would continually see on our bets like, Oh, we don't have enough mobile engineers. Oh, this time we don't have enough data engineers and we couldn't move people around very, very uh effectively. And it was like, Well, there's a solution to this like You should hire engineers and hire people who can actually cross skill across

things. And so we put a lot of energy into improving our golden pass and improving our onboarding so that people could do this and our training. And the technology training team is actually part of my team to sort of help people move through that quickly. So we started to track that as a metric, as like a T-shaped metric, to see that like, hey, people are actually upskilling and we're seeing where it's happening and we're seeing where it's not. And that can help unlock some of those bottlenecks.

I tell that story because, you know, today we basically see that not just within the engineering community, we see that across basically everybody in every role inside of RD. I historically come from an engineering background and I am admittedly a Terrible designer, uh, but I can visualize and imagine what I want, and I can type it pretty darn well. I'm pretty good at typing. And so today, like everybody can just

articulate that and you know as I mentioned before our prototyping tool brings in a lot of our design standards and it can spit out something for me almost immediately. So now I'm T-shaped not just from an engineering perspective, but from a broader R and D perspective. And that

I'm not a great designer, but enough to communicate and drive through that ambiguity that is really the process of doing product development. Going from ambiguity to very, very concrete set of rules encoded in code. Like that's the process.

So as we sort of think through this change that's happening and it's supporting product managers doing engineering work and engineers doing product management work, et cetera, all across all those different uh disciplines, then it's changing the team dynamics.

It's changing the expectations that people have from a career perspective. So at Spotify, we are sort of rethinking about how we we express that to the organization in terms of, you know, career frameworks and whatnot. So one of the things that I drove a refactoring of when I got to Spotify was our engineering career frameworks. And I feel like career frameworks are kind of a trap for big companies because they say like,

Hey, we think that you should be in this box and you should do these things. And, you know, here's a set of tasks that say that like you're uh progressing and whatnot. And it gives people structure and that's what you kind of need in a big company context.

But at the end of the day, if you, you know, if you've worked at small companies, you don't care about any of that stuff. You just I have to have impact. And so I feel this AI transformation is getting us back to a point where we can just express that, but we need to figure out how to express that in people's career framework. In a different way.

So people have the clarity that they need while still understanding that like what we really value outside of like ensuring that people meet their values and behavioral expectations. Is impact. And so if you're historically an engineer and the way you got impact was by doing some amazing design work, kudos to you. Let's get you promoted. The way you describe this shift from I to T standard and thinking about that both from a tooling and infrastructure and operational

challenge. But then also in terms of like I think it's really interesting because it speaks to kind of like the unspoken thing in talking to folks right now is like this idea. Like there was like a little bit of career anxiety of like what is next.

Next for me, like amidst this whole shift that nobody can predict and all of these mysterious or unexpected changings in terms of workflows and expectations and things like that. But I think like Like the roadmap you provided here is like the identity shift, like if you're specialized, like the identity shift is I to T and you as an organization have a responsibility to build sort of the standards and tooling infrastructure to help support that in a way that like meets

the the rigor that you need. And then on top of that, then there's like really starting to speak to what is value and to be able to provide a little bit of articulation around that and incentivize and recognize and reward people from an impact perspective. If you can define impact and you can describe how like that helps At least push people into a maybe an ambiguous direction, but like probably in the right direction.

As you articulated that, it reminded me of one other thing, which I mentioned before was this idea of an embeds that we have in our organization. Part of the reason that we had that concept of embeds was to help people to cross-skill. You know, this may seem like a s a simple thing, but we actually built a specific uh plugin into backstage, we call it skill exchange, that helped facilitate this.

Hey, how can you discover what other people across the organization are looking for? They they need to get more work done. How do you advertise yourself as an engineer for what you can do? And this has been really important part of a few different things for Spotify. One, is to support that career growth, that broadening of skill sets, making it very easy, and feeding it to what the business actually needs.

You know, we're a very ownership-oriented organization and we do our sort of headcount and budgetary allocation at a relatively slow pace compared to the project work we do. So at times we need to be able to move people across the organization outside of like their core teams. And so having tools like this that allow you to expose that business need and the skills, because we've actually

instead of asking people to self report into that, just go into GitHub and we say, like, we can see what these people do. This person's clearly good at b uh uh data engineering and they're clearly good at mobile engineering. So it all just kind of shows up in that ecosystem. And then you're an engineering manager, you're in backstage, you go into it.

you can kind of see, oh great, I need somebody to da da click click click. And then you're like, great, I've got a list of five people. I'm gonna go reach out to them and do that. And so I think that same paradigm just will continue to expand as new things that you can do and imagine become more easy to do. Amazing. I as you were sharing that, I'm like, wow. Like that in itself like become uh it seems like if somebody's listening to this, like it sounds like this could be sort of a externalized

product opportunity? Like this seems like an emerging problem as like companies like continue to wrestle with the the churn in the thrash. Is that the opportunity for me to say that yes, we've externalized that and we are selling? It absolutely it absolutely is. It absolutely is the opportunity for that. We have had a pretty uh vast conversation talking about a lot of the different ways that R D and workflows are changing.

But then also a surprising dive down AI enablement and some of the ways to expand this beyond RD, which has been so much fun. We have some rapid fire questions if you're ready to to dive into those. Ah, let's go for it. This is where I always fall down. That is all right. Some people interpret rapid fire as not rapid. So like, you know, we we can take as much time as we need. Okay, first question. What are you reading or listening to right now?

But I what I like to do when I'm not working is I like to read sci-fi and sort of get away from uh all the context and I probably a lot of people are like, I wanna I wanna hear what like business uh books you're reading and things like that. But I do think it's quite important to like decompress.

some great authors that I read. There's uh a guy named Adrian Tchaikovsky, author of Children of Time and a whole bunch of other books. I just like basically find all of his books, I read them. And then like from a humorous perspective, there's a sci-fi uh writer, his name's John Skulls.

Uh he's got a very playful, sardonic sort of set of characters. Anyway, so that's what that's what I actually read. Of course I read all the blogs and blah blah blah, but like I don't know, everybody can tell you about it. Okay, next question. What is a tool or methodology that's had a big impact on you?

All right. I'm going to take a maybe well with a a broad and narrow approach to answering your question. Like from a tool perspective, this is gonna you're gonna not be surprised if we say backstage. It really has had a big impact on on me, but like I think to back up a little bit.

When I started at Spotify, we didn't have an uh uh CTO, or actually we did right then, but we we removed the role about six months in. So I kinda had to take over the role, but not take over the role. Uh when which like not all of engineering uh w reported up to me. So I needed to come up with all these different like tools. to shape the engineering culture without having direct authority over all the engineering qu uh culture.

So bringing that tech leadership team together, pulling together the the onboarding and the L and D team for technology to like shape and, you know, set the new dogma. of how to do uh development at Spotify. I even pulled together our marketing team. They were a really important tool in my ecosystem and deployed them inward.

to be like, this is the amazing way to do things. These are the amazing products. Like those are really important tools. But they did all kind of have to come together in a place that people could act on and visualize it. And and backstage really was that. That's where, you know, as we were building these things, we were kind of seeing this need to help shape the

standardization work and the centralization and the sort of simplification work that we're doing. And that has also just shaped me from the perspective of like, it's been such a fun journey to take that through into an open source project. And then build a commercial product on top of it. So that tool has just had like a really big impact on me. Especially as you describe like what it took to deliver that. Like,

Incredible, incredible example. Next question, Tyson. What is a trend that you're seeing or following that's been interesting or hasn't hit the mainstream yet? And maybe maybe I just go back to what I was talking about before, which is Like the mainstream right now is Super focused on The the build it phase.

uh of the software development lifecycle and making that go really fast. And I think the new thing that's gonna come out is really focusing on the think it. And there there are people certainly in that that space, but really amping up your ability to do experiments. Uh, your ability to get insights around those experiments. I think a lot of like uh energy is going to start to go into that uh space. And I think that'll be really valuable because.

I often hear people say like the problem is engineering. I don't think the problem is actually engineering. I think the problem is, maybe even saying the problem, but the like the bottleneck is engineering. I think the bottleneck is really early in that discovery and that understanding process. And I don't see that much focus.

right now on that, at least not as much as relative to the problem space that I see uh the and the opportunity there. There there's folks, there's great folks that are working on Great insight. Last question, Tyson. Is there a quote or a mantra that you live by or a quote that's been resonating with you right now?

I I think I mentioned this one already. It's the the thing that myself, my leadership came up with, but I think I coined it is we take the pain. I I always go back to that as a platf uh a leader of a platform organization, to be like and of course. If you add the we take the pain away, uh it's uh it's it's nicer. But like our job is to figure out how to to make everybody else's job less painful. And if we can't figure out how to do that through automation.

We'll do it the the hard way. We we were talking beforehand as a rock climber. Like it's a very painful thing, but allows you to focus. You get rewards out of it. That's the one I I actually.

Incredible. Tyson, this has been this has been so much fun. Going into this, I was like, we're gonna get into some some thoughtful principles here. And this was everything and more of that. So I just want to say thank you for for taking the time to share more about your story and the intentionality and the ability to sort of articulate how you work and think together it Makes a huge difference for the folks in our community listening to this. So thank you. Thank you so much for having me.

If you're listening to this and you're wondering, how can I connect with other engineering leaders in my city? Pull up your phone right now and go to elc.community. Click our Chapters page. You can see that on the menu on the left. Find your local chapter and click join. We're hosting virtual and in-person events all the time. And this is the best way to help you get involved, expand your network in your city, and support your leadership and career growth. So pull up your phone, head to EF.

A huge thank you to all of our local leaders who make community happen and thank you for listening to the Engineering Leadership Podcast.