I think the strength of Surrey is exactly in the ability to bring about that alignment on operation concerns between the product management product development and product operations, whereas other methodologies, and great work that we talked about before, I had bought their strings, even other areas, like I didn't call bit their strength is designed the ID functional Enterprise develops. The strength is in the philosophy of or like delivery,
whereas Sorry, really. I think the strength of the methodology is in the alignment on operational concerns, the
entire organization. Hey, everyone, my name is Henry Surya with Robin. And you're listening to the technology, you know, podcast the show where I'll be bringing you the greatest technical leaders practitioners and thought leaders in the industry to discuss about their Journey ideas and practices that we all can learn and apply to build a highly performing technical team and to make an impact in your personal work. So let's dive into our Journal.
Hello again, my friends and my listeners, welcome to the peclet you know, podcast the show where you can learn about technical leadership and Excellence from my conversations with great thought leaders in the tech industry. If this is your first time listening to tackle a journal, Please Subscribe and follow the show on your podcast app and on LinkedIn, Twitter and Instagram and to support my journey. Creating this podcast, subscribe as a patron at technology node.
Dev slash Patron. My guest for today's episode is dr. Vladislav UK's, doctor Vlad is the head of R&D at Siemens health and years and the author of establishing a sorry foundations in this episode dr. Flat shared insights on how to establish a sorry foundations from scratch based on his first-hand experience at Siemens Health in years and the concepts described in his book.
We started by discussing the basic Sr a concept and how Differs from the other related Concepts such as I tell Co bit and devops Doctor flat then explained in depth. How is our implementation can help to create an alignment between the product management, product development, and product operations teams. He also shared the importance of having internal SRE coaches to facilitate this transformation.
And when an organization can start to realize the benefits of implementing a, sorry in the latter half of Our conversation, dr. Philip walked us through how we can begin. Aracari journey. Make further progress in the journey and measure the success of our SRE implementation. Also do not miss his sharing on how a sorry implementation can help to improve reliability in a stringent industry, such as health care. I really enjoyed my conversation with dr.
Vlad though, there are quite a number of resources, explaining about SRE concept, not many provides practical.
A step-by-step guide on how to implement it especially for the traditional organizations that need to evolve from their legacy operations and practices I especially admired octave left for writing an SRE book despite not having the experience working at Google. The place where the SRE concept and practices were created dr. Vlad successfully, embark on the journey to implement a sorry, in his organization and shared his first-hand experience in his book and this episode So, if you
liked this episode, make sure to also check out his book as well. And as always, if you find this episode useful, please help share it with your friends and colleagues. So they can also benefit from listening to this episode. I always appreciate your support in sharing and spreading this podcast, and the knowledge to more people, and don't forget to
give this podcast a rating. If you are listening on a podcast and Spotify. Let's continue the conversation with dr. Blab, after hearing some words from our answers, today's episode is proudly sponsored by skills matter. The global community and events platform. With more than 100,000 software professionals here members. Can organize their learning experiences around the technology topics.
They care about most you get on-demand access to their latest content thought, leadership insights, as well, as the exciting schedule of tech events running across all time zones. So whether devops our data science is your bus or you are And a functional programming or all things Cloud you can make real connections with people who share your interests head on over to skills method or Cam to become part of the tech community that matters most to you.
It's free to join and you will find it easy to keep up with the latest tech Trends. Hello everyone. Welcome back to another new episode of the package. You know, podcast today, I'm very excited to meet someone who wrote a book about SRE which is titled establishing a sorry Foundation. Action. But he is actually not someone coming from Google as probably most of you would have known as there is a concept that is introduced and widely popularized by Google.
But today we'll meet someone who actually didn't come from Google but wrote a good book about heh, sorry his name is dr. Vladislav yuccas. He is currently the head of R&D at Siemens Health in ears, so I'm looking forward for this conversation. Dr. Flat and hopefully we have a great conversation about SRE in general. Yep. Definitely. I really thank you very much for having me on the show.
I'm really happy that it worked out that we can talk about SRE on this really wonderful podcast which I wasn't aware of before. And now that I've seen all the great speakers from previous episodes, I'm really honored to be able to talk to you and be on the show. I keep for your kind words. So dr. Flat, I always love to ask my guess to actually share a little bit more about yourself.
Maybe sharing more about your career highlights, or any turning points that you think are interesting for us to learn from. Yeah. So I spent my career so far in the healthcare domain with the Siemens healthiest, Siemens held in years is a german-based company that produces Walt solve Medical Imaging devices. So things like, for instance can hit it from older files or magnetic, resonance devices or ultrasound devices. And so on there is lots of software that associated with
those devices. So there is a lot of one premise software. First of all, that's driving the devices themselves so that you can do scans on those machines. Then there is what's a post processing on-premise software which is software that is used by the Physicians inside the hospital departments but not necessarily directly at the
scandalous. Then there is also Cloud software that software, usually manages, the fleet's of those cameras and helps Personnel in the hospitals to around the machines, more efficiently in the inside. So I worked in the on-premise software at Siemens Health in years. And then, at some point, I moved into the cloud software. That was also. The very beginning of the cloud Journey for simultaneous.
Basically, over the years, we built the first cloud-based platform for medical applications for the entire organization and now every This units that wants to build a cloud application, they build it on top. Also the teamplay digital Health platform, which would build. And that was also the product where we started exploring how to, actually operate. Such a platform at scale and this is where it's a reclaim it and this is where the whole story started that led to the book.
Thanks for sharing your journey. So one thing that I pick up you have been in this healthcare industry for a long time and actually you implement SRE Concept in it, I think. Most of us are educated about implementing a sorry for cloud-based systems. So hopefully today we'll be discussing a little bit more, maybe from your journey in terminating it. In the healthcare industry which is probably quite stringent and it's kind of like mission-critical, right? So you can't have it to be
failure. First of all, how did you come about with this as re concept because you didn't come from Google? And obviously, you come from a healthcare industry which is traditional maybe tell us more a little bit on this journey. Yep. Yep, so as I mentioned before, the build up all day, pimply digital Health platform was new for the company.
We didn't have Cloud operations before and then also we didn't have the skills in order to be able to operate such a black pump because so far, the majority of the software produced by the company was on brevis. And as a development team, you don't operate on Brenda software that is done by professionals. Personal services, which is telling another organization that was a total paradigm shift for us. When we understood that actually, it will automatically work.
We need the development organization to do operations. So then the next step on that Journey was to see. Okay, how could we do this? We have try to operate the platform. Just using means that came to our minds. So we started setting up a loads. We started promoting the idea that actually, the development teams. Need to be also to be into the operational aspects and not just developing features and we kind of have tried doing this for
several years. But we saw that, this is a difficult idea to grow in a development organization that has never done operations before. Because in the mines, all the developers who had never touched operations. Their job is to develop something in the minds of product managers than their job is to tell the developers who work to develop.
Basically, everybody's thinking that there is This operations Department that will do operations, and the operations Department cannot do this because they don't have the context and you don't have the internal knowledge necessary to operate services. At some point. I kind of understood that we need to do something different in order to really grasp the operational aspect on the black. So I went to all the que con conference. In London. There, there was a whole track
on a surgery. So I stayed near the delete icon. It's on that track because I knew that was the pain point that the organization had of the type. Then I learned a lot about it. Sorry. But of course, that was just conference knowledge after the conference, I kind of understood
that this is something real. That there are lots of companies in the software industry at large going that way and trying out this Arena. And since you have some success, however you define it with it. So basically I thought, okay, that might be worth it. Right? For us as well, the navigate I'm back in the sort of started learning more and more about this living. The Google has a read books, of course. But the trouble was that that was Nicholas read books while
great. I kind of saw that they were describing the high end of what you do when you're really operate services at Google scale. Then I also read a couple of other books but all they were kind of describing was already at a level that was far away from where we used to be so then I understood.
Okay, fine, this is great but we need to So we're gonna start small and really start seeing whether it's a re would be applicable in an environment like us, which is obviously different culture than Google in different context than Google and basically different everything compared to. So then we kind of started adopting a sorry slowly but surely making small steps implementing a little bit of infrastructure and that them boo to the entire organization.
Then operating the entire platform using a Serene, this is only joy. Ernie was shape, really exciting, right? So, you learn about it in a conference, you went through like one whole day, probably, but you let it from the conference and you started implementing it and you become like an author of it. So I think that really speaks like a tremendous Journey.
But so far, I think many organizations would have been in your position as well, right from traditional company, maybe from Health Care, maybe from some companies, which are not born on the internet era. In the first few chapters of the book. You actually try to compare a sorry with the No way off you doing it operations, which are like, ideal covid and also devops in general as a culture.
So maybe if you can give some context and summary what is the difference of a sorry with all these other Frameworks? Yeah. So when I was writing the introduction to the book, I understood that there is no just a sorry which kind of was my focus before because I was the focus for many years on just doing a sorry. But then when writing book, of course, I needed to zoom out and see what else is out there in order. Has been introduced. I'm sorry.
Well, in the context of the existing methodologies. So then I saw that especially prevalent today is the isil implementation. This is basically a framework for how you set up the operations, the it function of an organization. And then, there are also a couple of other problems like this like, cold meats and
others. What they do well is to basically, How and it function or a big organization can be established and can be around, but then there are other kind of related things like, for instance, Devil's, right? So devops is it very, very high content in the industry.
This is kind of a philosophy. How you run the product delivery, especially with the shorts recycles and feedback along the way from different environments and from different people, including the customer and So on but then generally develops stays at the philosophical level. So basically it's a philosophy or fast. So quick delivery and fast feedback loops especially in the area of operations. But also in other areas it doesn't prescribe you enough to get started so just sleep.
So basically you can think of devops is an overarching philosophy of running product delivery and then you can think of other Frameworks like I tell and Corbett as Frameworks for I think the, I key function of the Enterprise. So now, when does the sorry, can in that context? So actually as a really is a and opinionated implementation of
the Ops part of demos. So you can also take a very narrow, kind of view on devils and say that just based on the name, devil tries to bring together development and operations. Although I think it's brought up, but you can kind of take that assumption say devops based on the main tries to unite them all. Building operations, so that they work well together. But then again, this is kind of staying at the philosophical level development and operations.
They should be working together. But, how should they do? Bad programming? Should they do pay operations, watch your video. How should they work together? So, actually, it's already as a concrete implementation of the devops philosophy. Is this opinionated framework to implement the Ops part of devops so to speak. It's got concrete prescription Souls. Should be put in place and also roughly by room in order to bring development and operation stick.
This is also coming from the origins of SRE which is trying as software Engineers to come up with the framework for operating services. So this is then sort of software engineer lead approach to operations. This is what is the reason then if you take those things like my tail and Corbett for For organizing the ID function of the Enterprise, then develops as the overarching philosophy of drug delivery. And then as a rig, being a concrete implementation of the devops was a bit then you can
say okay? So actually I sorry can live very nicely in parallel with Italian Corbett because with item covid you just set up the idea of action of Enterprise and then with that object sets, the overarching philosophy for program delivery. And then a sari is a piece in order to enable the She's lot of your product delivery organization. Well, well, thanks for this explanation, so we can see clearly where it fits into this overall along with the other ID and operations Frameworks.
So, thanks for sharing that you mention a phrase, which is very popularly. Described as area, a sari is like a way of operating software if you give it to a software Engineers, right? I think it's a code from been trainers. Lost. I'm also interested to listen from you. What is your definition of s? Are you coming outside of Google? How do you Describe a sorry, is there anything that you probably have different in terms of perspectives of opinions? I think you can definitely see
that. It's coming from the software engineering world. I would agree that this is what happens when you touch this operation ears with doing operations and actually very interestingly, this kind of thinking when you Cast Operation ears with doing something and be also applied to other areas. So for example, being specifically in health care with what a lot of agriculture, Button. So we've got a lot of Regulatory Compliance regulations that we have to fulfill traditionally.
These regulations are not fulfilled. From this equation, has point of view so they are basically fulfilled from the original tree person, point of view and therefore there are lots of documents that you had to write in order to demonstrate your compliance. With the regulations, those documents have to be handled in a certain way in order for you to be able to provide evidence
during what it's that. You actually comply with the process and so on. Additionally, if you task a software engineer to design the regulatory function, it will be done something completely different. Certainly, I think you can apply that kind of thinking, just a software engineer with designing X and that leads to a totally different implementation than if it comes from a place where it's all inspired by software engineering.
So I think there's generally a great potential to apply that kind of thinking also in other areas and we also started actually changing our regulatory process with ending. With the maintenance on that. It is definitely makes the whole process. Much leaner makes the whole process much easier to apply it and then leads to actually a great acceleration of products delivery. Because if you are less constrained by the regulatory burden that we can, we'll patch box them.
So we're all definitely would still agree with the original definition by Google posed by been touring has lost that this is what happens when you touch them. Software engineer to design be operations function. Yes, it definitely would be happy to apply that kind of thinking. Also, in other areas, thanks for the an interesting perspective. So if you want to improve something, you can also applies of engineering, 2x problem,
right? So maybe one day we'll see Regulatory Compliance engineering. When you apply software engineering to some area. You mention about last time in it, the Ops team will be its own Silo development team. Its own Silo, and product manager its own Silo. So, in the book, you Today, explain how a sorry actually is able to unify these three different teams. So tell us more how asari actually helps bring alignment into this three different areas.
So, I think the strength of a surgery is exactly in the ability to bring about that alignment on operational concerns between the product management or development and product operations, whereas other methodologies and framework that we talked about before, I have bought their Things even other areas, like I tell in Corbett, their strength is designed the ID, functional Enterprise devops the strength is in the philosophy of orluk delivery, where it's a really,
really I think the strength of the methodology is in the alignment on operational concerns, the entire organization. The cool thing about this. I read that also benefits us a lot in the employee digital Health platform. Is that it prescribes that? See, we need to have Level objectives for your services which are then based on service level indicators.
Then once you've got service, level indicators and service level objectives, then we automatically, you've got so called Arrow budgets, which are given budgets for errors. And then if you exceed your given budget for errors, then you can do data-driven decision-making about whether we invest now injury liability, or whether we continue investing in features. So basically you've got these kind of primitive Oops, that you can work with and that you can
use to guide your thinking. So now, how does as a really then bring about the alignment of different bodies on operational concerns. First of all, under the SRE framework, the developers, they need to go on call for their services, to the extent agreed in the organization. So that means that the developers need to experience
firsthand what? It's like to run their services in production and that can be Just a little bit, if the agreements to provide the services is such, that probably loving is just a little bit involved in operations. And it may be that the development teams are fully only called for their services. So we can arrange, sort of between 10 and 100 percents, can be done by the developers
themselves. The product management typically is not involved in product operations, but under the sari framework, they need to be involved in correct operations in order to actually make those decisions when we invest in there like that. It's you versus what we invest in product features, but these decisions are then done based on sound data frame production and that's based on the error budget
consumption. So, if there is no error budget left anymore, what do we do the product management needs to be involved or the definition of the so-called Arrow budget policies that state service by service and team by team. What do we do if there is no, error budget left anymore? Do we value invest in the ability to invest in reliability than how do we do this? Do we take One engineered wood on the reliability or do we re paralyze the back?
Opens on. As you can see, the product management becomes totally part of this entire conversation, how to guide investment on reliability, the product operations. They usually not involved in enabling others to do operations, but under the
authority framework. That's exactly what they need to do. So basically, they don't necessarily need to run services in production but they need to provide as our infrastructure to enable the Developers to go on call and the efficient to, basically, a traditional organization puts their responsibilities on the head, simply because the developers, they never went on call, but they need to go and call down the operations people, they never enabled others to do operations, and they need to
enable developers. Now, the operations, the product management. They were never involved in operations, but now, they are part of the conversation. How do we guide the investment in the reliability? As you can see, it gives each party. Certain set of tasks and aligns them using those Primitives like is alone zero budget, zero, budget, bonuses, and so on. So that the whole organization then is aligned on those operational, concerns ideally before hitting production.
So I think you refer to this as a collective ownership in your book. So from three different perspectives from development from operations and so product management and I like, when you set that now, the three different groups actually aligned on what kind of reliability that they would want to, Offer for their services.
And if there's something wrong about reliability, for example, your error budget is exceeded, then you make a certain decisions, should you invest in reliability, more, or should you continue investing on features? Knowing that your error budget is succeeding. So, I think this misalignment is pretty common in many organizations, right? So maybe from your journey, tell us, how did you solve this? Because when you introduce a sorry, it's a totally radical
concept. I assume for traditional companies, and some people would even think of this works in Google. Not ask. So how did you overcome that misalignment? And make sure three different groups actually can align on this same implementation. He had specifically related to make healthy years. Was that first of all we placed a sorry into the portfolio of things that we do in the
blackphone. We are in agreement that this is an important topic, it's important for us to improve the operations of the platform and therefore we'll try a sorry because everything that we've Right before wouldn't add success. So we'll give it a try but couldn't into the portfolio of things that will work on with that. Then we are able to allocate resources across the board. We were able to stop the development of the sr&ed infrastructure in the operations
teams. We were able to start talking to you the development teams and literally go kind of team by team and introduced the concept there and take each team won. Its tgb. Journey towards operations improvements using a surgery that kind of team. Based coaching was also key to establishing a sorry in the organization because every team is different. Also, every team is using the server infrastructure slightly
differently. So basically, taking the people who are implementing their infrastructure in going, kind of T by Tau into the product teams, including the product owner was key to actually being Unable to establish at sorry, broadly in the organization. It's very interesting because, yeah, like you said every team is different. Some may be more advanced in terms of understanding.
So for those people who haven't really understood about SRE concept, I do have an episode in episode 96, which covers the basic ideas principles and Concepts in a sorry. So today we will not be covering that because you can refer to that episode. Do check it out if you want to learn about what we are talking about, SLO error. Jet SLI right. So, dr. Flap and you actually tried to implement this. I guess some people may be puzzled about a sorry.
So it's like how do you actually educate people? You mentioned about the role of a sorry, coach tell us more about this role. The significance of it. Like, how do you actually recruit them? Is this coming from external or is the internal? So maybe tell us more about this important role of a sorry coach, he has only to embark on the. Is there a transformation? Want to establish a real organization?
Then you need Somebody who would be driving the transformation and this is, they're all sorry, coach, that needs to be somebody who's really driven to improve operations and who can understand the organization well enough in order to nail what you need to do, to establish it that all the different organization by organization, and especially that'll be indifferent. Also Culture by culture, therefore, I think it means to
be somebody in total. I wouldn't think that somebody external would Able to do this because you need to know too much all inner workings of an organization in order to make it happen. Also, I think there might be in just not enough, trust between the teams and be its own coach in all that you'll make that change. And also another thing typically something like this is a multi-year journey and usually somebody external is not there to spend years with a focused. So I think you need to be
somebody in total. Somebody's drinking too. Group operations and somebody convinced that sorry might be a good approach that person needs to be a mediator between the operations team. In The Mending, their infrastructure development teams, adopting a sorry at different bases and the management team endorsing the transformation. And also providing it on below
budget for this. So, I think this is definitely cure-all and must be there for quite a long time until and sorry, becomes the usual ways of working or For any team and also to the point where even Yugi has started, then they kind of don't think about how to do operations, but they just use the existing as our infrastructure and approach everything from the three point of view from the beginning when they set up new services. And you mentioned, this could be
multi a journey, right? So maybe from your experience, how many years did it take you to actually really start seeing the effect and benefits across maybe one or two teams? If not the whole company? So I think you can see the benefits pretty quickly because if you are coming from a traditional development organization that has never done operations before, then there will be basic things that you can improve immediately just to talk about basics.
For example, you need Uniformly, logging across all services that you've got typically in a traditional organization, you will not have uniform logging everywhere. They will be always, you know, one team does it in one way and the team does it in some other way. And there is this new service where the logging has not been yet enabled and so on. So basically, you can immediately can Elevate the maturity a little bit.
Then the next question is, okay, when did you really start seeing improvements at the management level, where the management team members will see? Yes. So actually before we had this, and before the teams were doing operations, the services weren't enter liable as now, something like, this will take a longer of course, but not necessarily that long, I would say, because, typically, in order for this perception to be developed, you need to see some external
change. And usually, the external change is measured based on the customer complaints, right? So usually would be a management team. Um, notices is that okay? Somebody's complaining because something is not working. So you'd be amount of external complaints gets reduced, this is what the people will stop noticing. I actually, this is already kind of best some fruit. Although from the azeri coaches going to do they of course will know.
Okay, we're just actually at the beginning of the journey but already if the development team is stuck, taking production, seriously, if they start really kind of looking into how this in his surrounding, even if it's not The best and sorry. We then that's usually already brings a huge effect in terms of reliability. Yeah, I can totally relate with
what you said. So previously if you haven't done any kind of a sorry, for example, there are typically many kind of implementations especially across teams in the same company, right? So like logging is one thing, observability, whatever that is you will have so many tools and they are not uniform as well as pretty difficult to trace from one service to the other and I think with the sorry concept that you start, maybe standardizing all this.
Having conventions. And I think, like what you said at the end of the day, it impacts the customers, right? Users have been as. So from the management point of view, I think, once you start seeing reduction in customer, complains of customer related issues, maybe that is the sign when you actually implemented SRE successfully. So you have explained as re concept to everyone. Everyone seems to understood about the concept. So how do we start by laying the foundations?
If I understand SRA correctly. It always revolves around service level objectives when we discuss about Collective, Bishop, I think from the product development for the operations and product management. We also agree on the same service level objectives, right? So, we have the same definition. We have the same understanding about a consequence of not meeting that objective and things like that.
So tell us more what should be the step-by-step foundational steps for people who are trying to implement a sorry in their company or team. I think a couple things need to be done at one important thing is that I think it's important for the initiative to be taken. Seriously that requires a surgery to be put onto the list all organizational priorities that they'll get ization wants to explore.
So every organization has got some list old big topics that they're working on. Ten topics roughly that we are working on right now and that needs to be on the list because this is such a profound change the development team, use suddenly starts going on all for their services and the operations team starts suddenly to implement some free Works in order to Enable the developer team Studio operations to basically the development team start doing some operations for
and the operations team started doing some development work and visit Aunt will kind of transformation for both parties. Then be organized when suddenly is in the middle of the discussions about reliability which is still being used to that. So I don't think you can really bring this about if the organization is not serious
enough about being judged. So I think the first thing put this on the list, Of initiatives, you are undertaking, then the second thing, wind a possibility in the racial team to implement a little bit of a mess, our infrastructure and really the very, very minimum. So I would say just start with one SL.
I say availability, because that's kind of the most common ones in the most fundamental one and then he just Implement some infrastructure that can take some logs and calculate the availability of a service based on those logs and then can alert on say Certain thresholds that third layer is already enough, that big say, one development key that is responsible for some services that would be kind of open to exploring a new way to do operations. From then onwards kind of
operate. This re circulate very frequently between what the development team actually means. And what there's our infrastructure needs to provide, basically bring together that one team and the operations team in the wedding ditra structure and bring it. To a point where that one team would say, we actually make sense. It's an improvement compared to before it's better now and we want from this our infrastructure additional 10 features. I think that's a good starting point.
So that's solid already. You've got an owner of this rail tracks are two new chords infrastructure used by a Regal Team and if they want more, this is actually a good point to say. Okay. Now we know we are on to something useful and let's go find the T then the second team can already make use of this for infrastructure. That was built for the first team.
They will, of course, have some different context and this is where you can then decide, which additional features to implement into this or infrastructure from then onwards. It's about going team by team and talking to them, discussing with them. Whether it's very would be something bought them but you should make the more teams you on both the easier it gets because they are infrastructure.
Is already more mature every time To take on a new team because you've got more features there and, you know, the day useful because also, there is a kind of social proof that this new thing. It's a really kind of helps with the services operations. I would say this is kind of roughly the journey that you would need to take. And this is also where the SRE coaches are essential, especially at the beginning. So I think that totally makes
sense. Why you just do it gradually building the infrastructure as well. Don't forget about that because you can't just tell the team. A you go Implement without Structure or platform invited by the SRE best practices because otherwise people will
implemented differently. I totally can relate as well because I did try, similarly leaving the teams to implement it, but I guess people's understanding varies you would start seeing some themes which implemented properly some themes, which probably just follow the request of the order. So to speak, some actually just finish and then forget about that filter, which brings us to the key concept of SRU. So when your SLO is actually being breached, what do you do
about it? Because I think that is maybe the Tipping Point how the culture change our because if you have an error budget it got breached but nobody cares about it or do any action. So maybe from your experience how do you handle this? Such that it becomes a Tipping Point where people understand. Okay. This we have to take seriously. Yeah. So several step step, step one is involved the product owner into the sorry coaching sessions from the beginning.
The product owner is totally a full member of these discussions. Number two rule. Once the pill you moon is mature enough to come up with the arrow budget policy. That this is something that's really helpful, because it forces the team to think exactly about this question. What will we do when there is no error budget left of this service or for that service?
It does this ahead of time before you exhausted your error budget and that provokes really good discussions that also Just upon other things that the team is doing, for example, the team might be say planning to do some Workshop in order to plan the next increment. What happens if your error budget gets exhausted exactly on that? They do you then this time the workshop or what do we do with our planning, something that provokes a big discussion, you need to basically put it on paper.
I Aro budget policy is if this happens. That's what we do at that happens to that one. So that's already definitely the next level of maturity. T, but I think it's important, it's not to start with the whole error budget policy before the team is actually ready for this because that's a pretty Advanced concept. That's also how I put it in the book. So I'm talking about the basic SRE commendations and advanced
as a repo indications. And the basic answer recommendations are SL is Loz and error budget. The advanced. That's very foundations of the error, budget, policies, and error, budget, based decision making. So, here we are. Day in the advanced space, and we need to be aware of this.
It doesn't make sense for the sorry coaches to force the discussion about the error budget policy, because either people will just follow this fashion and forget about it the next day, or they will just not be able to talk about this because they're just too far away from this. We know that for you to talk about the error budget policy, you need already. Everybody's understanding about PS allies. Everybody's understanding about this fellow's. Everybody's understanding about
the era of budget and the solos. Need to make sense you need. To already have several iterations about the whole thing and then you need to reach a point where it's a point, okay? So now we've exhausted every budget, but we haven't talked about yet what we'll do now. And this is where you bring the discussion not before once they ever budget. Policy is in place, that's done good enough but then the real
test is okay. So we'll that error budget policy, the executed, once it's really reaches the point where we don't have that Arrow budget anymore of the service and that's important. Does the team really follow what They have done. That's the next step. And then, this step after is the error budget, based decision made. I think that's the cool step backs at the talk of the three concepts remain in the book, I put all those Concepts in the parameter and that's kind of
been the tip of the pyramid. It's all basically what you do you start tracking your error budget consumption or your SLO adherents over time, you divide the time into narrow budget periods. So, could be one error, budget. Period is just 1. Month. And then you pour all your services and there is a little red hearings by error budget you. So you can have a group there you know you've got all your
services on the y-axis. And on the x-axis you could say half a year's time line so would be 60 budget periods and then you see where you broke your slos and it's where you didn't break them and Zone. If you break the SLO, for the error budgetary, it's actually useful to see whether you broke it but just a couple of percentages or whether Really kind of broke it up significantly.
The same also applies, if you are fulfilling your SLO, then you can fulfill it with a lot of Arrow budget left at the end of their budget Baron, or whether you nearly exhausted your error budget. And therefore you kind of nearly broke their solo, those, the details are useful as well, but the cool thing about that kind of high-level bash.
What is that? Especially in the product managers, they are enabled to work with this because then you can say all the time from experience, we actually thought that We would have this certain service level but we are not fulfilling this and therefore we need to prioritize some significant work in order to build a reliability into this particular Services. The cool thing is that once you make those decisions for your Based on data and not just on
the opinion of the architect on the Obion, all the developments on which depending on the team culture, May weigh a lot or not, a lot to the product manager. So I can totally understand. Once you have this cool dashboard where you can see each different Services, how they're performing the SLO, their budget, whether they are bridge or healthy, I aspire to go that
way. One day like you said, it's a data-driven approach of knowing where to improve and went to improve your across different teams. And don't forget the SLO. Here is a representation of users happiness, The Summit AT internal metrics that you just want to capture a, but it's actually a representation of the
users happiness. So when you see it's being Bridge, basically, the user should be unhappy As well, which is coming to my next question because in the healthcare industry, I always assumed that healthcare industry must be very reliable. All these devices that you mentioned they should function perfectly, but we know in a sorry that hundred percent reliability is totally impossible. Maybe can you tell us more about
from your experience? Implementing in healthcare is Healthcare, always like 99.9% reliable, or is there any kind of explanation that you can give here there are different device? Since and Associated software systems in health care. There are Hardware scanners, there was your availability and reliability, requirements are the highest.
There are very strict medical device regulations that oblige you to perform certain tests and so on in order to reach that high level of reliability, then there are so bold post-processing workstations. These are still on premise software systems that of course need to also work your Lively because they work in a hospital. Very fast a decently persistent. Doesn't warn them that can block the entire workload. But at least it doesn't harm human life anymore if it doesn't
want. So then there is the clouds or wet which are services that enable you to run your fleets of scanners more efficiently. If that doesn't work. Then at least you don't block immediately the work in the hospital department, but you would block for instance, the operations person we see all of the hospital Who is now trying to optimize the assignment of the patients to this cameras in order to be more efficient? So there are different kind of
levels. All reliability that you need to have that and all that is governed by the medical device regulations. They are also a bit different depending on the criticality of the system to the human lives and interested very much interested in the hardware scanners. When you mention it has to be the most reliable. What happens when the error budget is about to be breached or maybe there's an alert. Being other people scrambling because it could affect people's
live, right? So, tell us, when this situation happens, what actually really happens? Well, first of all, we only applied so far as a retail software, but I think it's a good idea to also, think about how that could apply to on where the scammers themselves. They've got lots of checks and balances in order, not to harm the patient. So for example, you can really think of a complete tomography CT scanner.
That's a discolored injects. Some acceptable radiation dose of radiation into the patient in order to acquire images from the patient. Of course, if the scanner would notice that now, the patient would be hard because there is scary too much. Radiation about to be exposed than the machine will stop the scan. So immediately, the stop will be notified. Please sample. Then remove the patient from the ringing, where the scanner is installed.
And so on. So there are definitely lots of precautionary measures that then additionally, What the cloud software can do where we apply this theory is that we have got? For example, those management software. Those management is a cloud-based application. It can also monitor, although not kind of in real time. What's going on with the scans being done right now but it can
monitor things. Like, for example, imagine you are a patient that requires several scans that means that actually your accumulated radiation those Over time can actually be exceeding a certain threshold that is also about meat by the government regulations. So actually you need to keep track of the radiation not just during a particular scan, but also overall, over a period of
time. If you've got some serious disease, then you might be prescribed to do certain scans within say a year or so. So what that software can do it can actually keep track of your accumulated dose and then also issue warnings. You the hospital Stout. Actually that Beijing already got so much, and therefore, be careful insult. As you can see, this is very interesting industry where the different levels you require liability and that can be on the one hand, really kind of Life
critical right now too. Okay, so it's not like preschool right now, but you still need to have some monitoring. Also, the patient in order not to expose the patients to somehow control treatment Thanks for sharing all this. The reason I ask is because some people might have perception. Okay, this may only work for certain cases maybe internet based software, right? But actually you can apply it almost in anything, especially in technology.
So let's go back to the discussion about implementing a sorry in your organization. So let's say you have done it. The teams have already implemented, maybe from your view. How do you actually measure that this asari transformation is successful? Are there any indicators that you probably check periodically to say that? Okay, this is sorry. Bill Addition is actually progressing really well? Maybe some tips here. I know that you have some dips
in the books as well. Maybe you can give some summary. What kind of measurement that you do in order to find out that your transformation is successful? The so, especially the beginning. It's a good idea to just see how many services the teams are onboarding on to the infrastructure. This is non outcome-based course because you still don't know whether it's a group reliability. But at least this output based indicators in the beginning there. Suddenly good.
Because if teams are not willing to put their services onto the Surrey infrastructure, then there's no use in the infrastructure that's one day once you see, okay? So the number of services is growing on the infrastructure, so that's good means teams have finding the infrastructure useful. Then the next thing is okay. So do the team's Define this ellos in the number of its ellos actually growing. Then the next thing, okay? Now that the slos and there but are they being fulfilled if
there is a load? But the majority of them are broken, then that means the teams just forget about them, they basically don't make sense, then DC? Okay, so, what is the percentage of his fellows that have been fulfilled? Then the next step is coming back to those customized collations that we were talking about earlier. It can easily happen that all
your solos and perfect green. So that's all cool but because the must keep calling, these are the measures that you can take in order to monitor the process of the sorry introduction. I like the way you describe it, your error budget can be all green but the customers keep complaining. So I think that is also like a big sign that actually, you might have missed the most important slos of all, right?
Because the true measure of implementing a, sorry correctly or consumers, should be happy and then your error budget is representation of that in their book. You also mention a couple of other things, like, for example, perception from the partners that you work with, not just the users, I think that also matters because sometimes if you are like access provider, right? So some Partners connecting to you also matters whether they come, Not so thanks for all this
measurement. Dr. Pratt is been a great conversation so far but unfortunately due to time we have to wrap up. But before I let you go, I have one last question that I always love to ask my guess which is to share your version of three technical leadership system. Is there something that you want to share with us here dr. Blood let me think of something that would be in the context of this Rich information but still kind of applicable.
See General technical work. One thing that's important is pure lag in context. I think that's important because there are very many Technologies, methodologies and so on, but it's the context that actually decides whether this particular thing that you will have heard about everything about have thought about would actually fit. So, I think this kind of acting in context is an important, I'd
say evade principal to follow. Another important aspect, I would say is to practice servant leadership. That means that on the one hand, trying to Show the way to the organization to teams but on the other hand also being very well aware of their context and therefore kind of inviting them to go on a certain Journey instead of coercing them to adopt a particular thing on one end league. But on the other hand, do this in a servant. Wait, I think that's important. They start thing would be.
I'd say all these things that we've talked about they're kind of not easy. For the developers. It's not easy to just jump on cold, right? It's a totally different game. If you've never done this world for the operations people to start suddenly act as a real software developers providing Frameworks to the other teams and so on it's a totally different world and for the product managers to suddenly be involved in operations.
It's also you know out of their world so far so I think I do acknowledging that this is not easy empathizing with the people. Specifically is a third Point, kind of looking for ways to motivate people. I think this is really important to make some change thick especially on this Region's formation. During there are so many ways where you can point out very small wins, I think a really pointing out those small wins that now is a team, we've done this, that really motivates.
The people are taking this serious from the asari those for you to do and really seeking things that people can be thanked for. For them by driving the motivation. I think it's really really people. Well, I love the last one. So first you have to acknowledge people situation empathize with them and find motivations and celebrate the small wins. I think that's always important for any kind of data dr.
Flat. If people want to learn from your journey or so maybe they want to establish the same as sorry transformation in their organization and they want to learn from you. Is there a place where they can find you online connect with you or ask questions? Yeah, definitely. Can be easily followed on LinkedIn now with LinkedIn. There are several conversation threads about the book. I think that would be also the best way to just join those threads or start new ones where
we can discuss those things. And also thank our watch all the conversation on the questions. I think that brought out early, good aspects, and was really fun to talk about. And while we were doing this, I was thinking that there is formulate another Myriad of things that we could talk about because that topic stove us. Yeah, when I saw your book is very thick and lots of good information there. I even finished reading it definitely, but people do check
it out. So I think it's really like transformation based kind of learnings, right? It provides you like, the rationale of how you implement necessary, the step-by-step Journey, including to the end measuring the transformation, making sure that you are successful in your journey. So thanks to dr. Vlad for this conversation, I really love it. So good luck with their future Improvement for your, a sorry transformations in your company. Thank you very much.
Thanks a lot for having me and thanks to everyone who took the time to Listen to this episode. Thank you very much everything, like to improve operations together. Thank you for listening to this episode and for staying, right until the end if you highly enjoyed it. I would appreciate if you share it with your friends and colleagues who you think would also benefit from listening to this episode. And if you're new to the podcast, make sure to subscribe and leave me your valuable
review and feedback. It helps me a lot. In order to grow this podcast better. You can also find the full show notes of this conversation on the episode page, at Tech Legion o.f website, including the full transcript, With interesting quotes and links to the resources mentioned, from the conversation. And lastly, make sure to subscribe to the shows mailing list on technology. No dot f to get notified for any future episodes. Stay tuned for the next technology. No episode.
And until then goodbye.