Should we try to build the perfect platform? I mean sure if you don't want it to be used, you want a platform that fits everyone's purposes as best it can and it won't be perfect, it will not be perfect. Welcome to DevOps Sauna season 4, the podcast where technology meets culture and security is the bridge that connects them. Welcome back to the Sauna. Hi Darren, how
are you today? Hey, welcome, I'm doing okay, bit busy, housings on your side. Fantastic, but there's something that I'm seeing a lot in history and in my current customer engagements that I wanted to have a conversation with you about and that is the resistance to change. Yeah, I get a lot and I think this is one of the few topics we can talk about that's not even specific to technology.
This is a topic that touches basically everywhere change happens, which is usually everywhere, so it's going to be a bit of a bigger topic, maybe a bit more wide-reaching than I usual, very technical, or very process oriented things. Yeah, there's one way that I can look at this that helps to kind of spread the spectrum, which is, you know, modern command and control hierarchies, everything from
project and program management, PMO project management offices. A lot of this came from the, you know, the way that governments worked and the way that big projects worked and things like Manhattan project kind of set modern project management, but a lot of this stuff still comes back to things that were command and control hierarchies for labor. Yeah, and the modern world has developed in a way
that doesn't really support such architecture anymore. It's kind of, well, I don't know, maybe there are some managers out there who think it does support such architecture, but we have like spread out workforces, we have remote work, so all this kind of spreading of the workforce and people taking more responsibility for their own tasks. It's kind of maybe a bit of a tangent, but it's sort of interesting how a lot of the way we look at these things still reaches about, you know, 70 years.
It should us, and this concept of the knowledge worker is one of the paradigms that, you know, modern management needs to be aware of and have a good grasp on because what we want to do is we want to move the conversation to how do we get the best out of our knowledge workers who have domain specific expertise. They know how to add and create value within your organization, and we
want to give them interesting problems to solve because that's their job. And when those people are upset that they aren't able to do this effectively or that you might be, we have this pets and cattle thing that we talk about in software a lot, so it may be that somebody has their pet tool that they really, really love, but a pet requires taking care of it individually when you think of something like a herd mentality, like a whole suite of tools that are used across a broad organization.
It's more like cattle. We need to make sure that there is enough resources or food and water
for the cattle. We need to make sure that they have places to run, places to graze. So in the tool environment, it's about making sure that we are able to have scalable tools that can support many different types of workflows that are standardized enough that you are able to have security compliance and the ability to, you know, really do the work that is at hand, which is taking this knowledge worker domain specific knowledge and using that to create value for the enterprise.
Yeah, and I think that actually leads in kind of an interesting topic for discussion because one of the things about change resistance comes from that knowledge worker in my opinion, it comes from their tooling bias that they've built up over the time of becoming a knowledge worker of like learning these tools, learning this tool chain and then wanting to stick to it. So you raise the knowledge worker as a, like a kind of a driving force for this change, but it's also in
my opinion one of the largest sources of change resistance. It's a knowledge worker who's set in their ways who is very specifically looking at tooling as pets and this is their pet tool and they want to keep this tool and they know how to do everything in this tool and then again, we kind of circle of home to the unfortunate Jenkins in the room where it's like it's a tool that I don't want to be abusing constantly, but how much can we do this abusing Jenkins before we get someone
to send us a nasty comment? Yeah, I think someone's going to comment at some point, but we do have to outline that if you're picking a tool because you're familiar with it rather than it being the best tool, then you have made a poor decision. But at the same time, one of the best tools is the one
you're now had to use. So it's like a weird balancing act and I think walking this path is kind of the way through change resistance because you have your known variables, you have the tooling you want, you have the tooling that's good and being able to give the developers something they can
use and more importantly, being able to involve them in the process because there is this tendency in my opinion to have like in post versus them mentality of we're developing this platform to developers are using it and what we say goes because we're developing it and that's that like I just think if you don't involve the developers from the start and involve them in
the iteration process and this goes for every change management. If you don't involve the people who are required to change in building and taking ownership of that change, all that's going to happen is you're going to end up with a tool that's either resented or not used. It reminds me of a related story where you know we the directors of the company we went into the the boardroom and we were involved in creating the strategy of the company and we felt that we understood it quite
well and we were very committed to the the strategy of the company and then when we went to the auditorium in order to present it and we started to look for the execution of the strategy a few months later we realized that people didn't really seem to understand it and certainly weren't committed to it and it's like well were they involved in creating it so yeah I think that just if you cut people out of the the actual iterative process you could be surprised if they're
annoyed and resistant to using the end results. Yep absolutely and you know we have a couple of Nordic friends that have cultures of consensus and one of the things that I think is really important to illustrate is you need to have everyone's voice heard and I believe that you also need a leadership with a vision that says there's a vision that we have for what we want to do we want to be able to release five times a day or we want to be able to release any pull request because it
allows us to serve our customers like this. When you have that kind of vision it helps to answer a lot of questions as to the why but then when people also have their voices heard then they feel like okay I gave my voice I still wanted to do things this way but now at least I understand
that I was heard and it aligns with the vision the choices that have been made. Yes so you have like a golden path you've got the leadership there to have the direction and the feedback loop to improve the direction but I think it's important to iterate the fact that you when managing change
you will never be able to please everyone the idea of pleasing every one you have to deal with everyone on your platforms that's a pipe greed that's just that will never manifest so we can pull examples of that any tech company that allows the use of Linux for example what you end up
with is several hundred different people wanting to use several hundred different tool sets wanting to use their own distributions of Linux and a security team who kind of have to say okay this we can allow this be gone and then most of these requests we don't even have the capacity to acknowledge
because it comes back to the old joke about IRC and how it doesn't matter how communication technology changes there will always be one thing still using IRC to connect and talk to their co-workers and the burden of that IRC user can't be put on the developers of the platform it can't be put
on the people forcing the change because one person who cannot modify themselves to the working methods to the platform as it's changing is not indicative and that's I think an important thing to say by saying you need to leadership there you need to leadership to be able to discern when
something is valuable feedback and when something is just one person being particularly attached to a tool set particularly attached to the way they are working as you say cattle not pets I'll name drop here so Yarko Oykaurinen from Olo I had the pleasure to work with in in Nokia days the
inventor of IRC I assume Olobox is still out there somewhere but my hats off to you for having you know basically changed the way that the internet communicates it was nice for you to drop that in there and it actually raises another question though is there room for inflexibility because
I know in a lot of these change processes are related to technology we look at things like finance and finance has like IFRS the international financial reporting standard the US has the what is it generally approved accounting protocol procedure something along those lines dark yes and yeah
flexibility in those cases I understand is rare and you usually involves breaking the law and I think there's a kind of rigidity in that mindset that I think we're in danger of taking into technology because we're seeing like EU directives around security around the governance
and use of AI and as these come into play it's going to be interesting to see how it changes the landscape so I just wonder if you mock if what do you think about flexibility should that be room for the flexibility this is one of the things that I get up on a lot the first thing
is that we need to get good at change and we need to understand that if we're not changing you know the tools if we're not changing the processes then we're not growing and if you're not growing you're dying it's binary it's one or the other so then in terms of flexibility you know agile went wrong when the real flexibility given to developers was just what tools are you going to use and how are you going to run your team internally compared to the whole rest of the enterprise that
is choosing a different set of tools in a different way of of running their team and none of this can pass like an ISO 27001 audit you can't have everybody doing things differently and management 3.0 came out was it like nearly 15 years ago now and said you know the concept of having minimal constraints and I think in safe this is called guardrails I'm not really a safe guy there's a joke in there but what are the minimum things that people need to do in the same way in order to have a secure
compliant and efficient way of working and that's I think grown over the years in terms of these fragmentation that's been caused by everybody having their own instance of Jenkins except for those guys over there that have GitLab running under their desk on a server hit by the late cleaning lady
with the vacuum cleaner every day IT doesn't even know about it but there's a significant portion of very important software that is built on that open source product rather than using the paid enterprise version with all of the bells and whistles for security and whatnot so the thing about
this flexibility is I still want to come back to something I mentioned before as I want to move the conversation away from we think that we have these special needs and we need to have a special tool in order to achieve that versus I want interesting problems to solve for our business
and I want to know that there's a platform team that is making sure we have the state of the art tools always available because not only do I want to have the best possible developer experience but I also want to know that my skills are being based upon the latest technologies and things that
are actually going to be marketable one of the things here you know you have too much flexibility everybody does things differently you don't have internal portability within your company I can't go and work in another team and understand what's going on how do you onboard that you know
everybody has a different onboarding procedure a different development environment so for me it's kind of like the flexibility should be how do we solve problems creatively have an ability to experiment in production have an ability to have you know the psychological safety that if I make a
change it will have a limited attack surface or a limited what's the other way of looking at a attack surface like the ground zero type of thing or the blast radius right so if I make a change in production they will have a limited blast radius as to how many people that it might impact
so I have the safety to be able to do experiments in order to understand better the requirements that I'm supposed to be building for the business so you know I want to push this kind of conversation in that direction and say if you really need to have such flexibility that you have to run Jenkins
when the rest of the world has moved on to you know GitHub or GitLab and you know runners or you know actions then I think that you're having the wrong conversation in your organization yeah I think that it comes to the source of inflexibility so if the inflexibility is something imposed on you
from an outside influence like a governing body something like the ISO 27001 standard something like a reporting standard a security standard that's the place where inflexibility can't be avoided but if it's coming from inside your own team if it's the people there who are creating
inflexibility that's where your largest resistance to change is it's not inflexibility due to a requirement it's inflexibility due to an unwillingness to be part of the program to be part of the change and I think you touched on a couple of important things here and one is this
server that's under the desk that keeps getting hit by the cleaning woman you've mentioned this like three times now so clearly we need new cleaners sorry for that the second thing is that I think it's important to underline that ownership of the platform yes I think we touched on
it before but I don't think we quite went into it because the ownership of the platform is key here and you said it yourself at the start then move the conversation to interesting problems to solve and the loads of my work is just defined by which problems I find interesting to solve and I'm sure
it's the same for many other technical people I'm trying to avoid the word nerd out there but it's it's kind of the same that if I have an administrative task I don't want to do it and if I have an technical interesting task I want to focus on that and this is the key thing to like develop
ownership of the platform in such a way that the people using the platform the people making the change take ownership of it and find it an interesting problem to solve absolutely I will clarify my story that I had a customer a large enterprise customer wireless company in the
United States and the cleaning staff was not generally supposed to be cleaning the server room however one of the rotating cleaners who happened to be a nice lady her key worked on the server room so when she would go in to do her work she would try her key where it worked and she would do her
cleaning operation and there happened to be a power strip of machines that were outside of the the regular rack system as that she find in the server room and she would randomly unplug one of those and plug in her vacuum cleaner and do the work and then she would plug it back in and
then the customer was telling us about our unstable situation and that the machines were randomly rebooting you know like once every you know week or once every two weeks a different machine would be randomly booting and we didn't know why so I've transferred that story to the the real life
that you know many developers have to have a shadow IT in order to run the tool of their choice under their desk which is a real life problem that we see I breed and that's taking a lot of ownership but I want to take this ownership thing into a slightly different idea that you know
we look at research and you know if developers spend 40% of their time on maintaining their toolchains then this is one of the kind of things where I think the ownership is a little bit too great potentially and it's one of the opportunities where platform engineering teams who are helping
to create self-service environments for developers that they are able to make changes if they need to and one of the flexibilities here is for example to be able to try a different tool in a runner so if you are building your software and you are responsible for you build it you run it
in terms of you develop it you compile it and you execute it in production and you have responsibility for that you know having the flexibility that developers can still take ownership of a pipeline in terms of branching it or forking it off making a few changes trying some
different things and then potentially making those generally available through a platform engineering team might also be something that it has a lot of validity but it kind of comes down to as well many organizations from finance to manufacturing and AI and lots of different kinds
of companies can run with a very similar toolchain today and using one of the big platforms and then a handful of others so it's difficult for me to understand why the most critical area of flexibility would be in the tool selection today compared to doing things that actually create real business
value and that's kind of one of the things about the ownership as well I would be so happy as a developer if someone else were able to maintain all of that stuff and I can just come to my desk and start solving really interesting problems for the customers see those realized in production in
near real time get feedback on those through monitoring that I have access to and then be able to tailor my interpretation of my requirements in order to make sure that the customers are getting the best value and I think that that's one of the high arts of DevOps is being able to realize that and we're not going to realize that if everybody has a different way of working and is maintaining their own tools all the time. Hi it's Mark again the DevOps conference global will be live in London
and streamed across the world on March 14th 2024 don't miss it see you there. Yeah I think you're a hundred percent right there but what you're saying there is that the tool or the platform or the more generally the change the thing that they're changing into needs to be extremely good
it needs to be something that adds value it needs to be something that reduces the load of let's say this 40% that developers are spending on their tool chain that but they still also need to have a voice they need to have a say in it and they need to have ownership of it so would you
say it's then quite a difficult line to walk for me it's you know technically speaking I come from this from like a very technical point of view that sure we can outline all the best tools we can talk about the tools and implement them and then give it to people and it might be perfect
but it will be stellar and no one will use it because no developers have had feedback so what would you say towards that kind of making it so good it can't be ignored there's two at least areas here the first one is is there a management strategic company level thing that the change will enable
so for example being able to do changes that are effective in production on a daily basis versus having releases twice a year and I think many developers can understand that they want faster feedback and that as a part of that vision having reducing their cognitive load by having faster feedback
on their changes and the ability to see those running in production also brings you know a pride of craftsmanship you know the delivery lower the imposter syndrome things like this so the management needs to buy in and understand the investment and needs to say that this is going to change
and there needs to be some bravery in that decision to say that we're really going all in on this and we're going to make these changes and these are the reasons why and then on the other side it is how people are involved and how the ownership of those things are stepped up and taken care of
where if you have a platform team they can't throw things over the wall to developers they need to constantly be working with all of the development teams to make sure that they are enabling the things that are necessary but the the change leadership here is is so important as well
because it's so easy to fall back into the previous ways of working it's so easy to say okay you know we we've gotten into a new paradigm be it a new tool chain or onto a new platform or using just a couple of tools differently you know moving from azure dev ops to to get up you know it's
even something like that but the change leadership needs to understand what the reasons are and keep an eye on the ball there because otherwise you end up just replicating the previous pathologies in the new paradigm we don't really want that either but if I try to kind of put my spin on the
summary it's that I want to see it so that change is a constant improvement it's not just big bang all all in approaches but we need to get used to change because if we're not changing if we're not growing we're dying and then on the other side there needs to be some bravery and some mandate
that we're going to to be in this together and some people are not going to like it but they're either going to they're going to work through the situation and see the value of it or they can go try to find some other place to add value that either is growing or dying itself I mean that's
kind of a bleak summary in that case but I do agree you're going to end up in a situation where either enough people will change the others who have been kind of sticks in the mud are being dragged along or they are going to look elsewhere and in my opinion that's entirely okay like not
every company will work the same and but you do raise an interesting thing that I think it's mostly interesting that we kind of left it for quite late in this discussion but the buy-in from management being actually in my opinion the biggest and probably earliest thing you need
for change management because I think that I can talk about this from a security standpoint the whole pushing of security methodologies is always met with resistance because it always adds overhead and the best security tools are the ones that can do it invisibly without we're basically without
requiring anything from the developer and as a security person if you don't have the backing for management it's so easy just to be ignored and I'm pretty sure it's the same when it comes to managing change resistance in deploying platform tools in any kind of change resistance if you're
just one person trying to push a change you're easy to ignore the it actually weirdly comes back to the whole command and control infrastructure we're talking about before if you don't have to leverage from above to kind of push these issues I feel like you're not going to really get very far
with it so you can end up building a great platform that's no one's using because the buy-ins not that you know I've said in a meeting that had a senior vice president and a developer in it where the developer said that he would not support the change unless it had Jenkins and if you can imagine
what somebody who's responsible for a few billion euros worth of revenue thanks when an individual says that their whole the whole change program that's been planned can't go through because of a single choice of a single tool it's quite interesting but I like the on the positive side of things
I really like that you know we set a vision and we're going to make a change in order to stay ahead and you know keep everybody's skills current enable change in our organization and do this for the right reasons and I think that it answers a lot of those types of questions okay so since my
summary was so bleak my way or the highway evidently I'm not accused of that kind of thing very often I'll ask you for for some summary then Darren so is there room for inflexibility I would say yes as long as it's not coming from inside your teams all right who wants a platform everyone
if the developers aren't in on the process then it's just going to become something they resent all right should we try to build the perfect platform I mean sure if you don't want it to be used you want a platform that fits everyone's purposes as best it can and it won't be perfect it will
never be perfect and if you think it's perfect it will be out of date in three months so you have to iterate are there some known variables to look out for dumb use Jenkins but knows the I mean generally speaking I feel like you should avoid tooling bias you should also try to avoid as you say
that pets versus cattle syndrome these tools are the things that serve a purpose they shouldn't be decided on because you like them all right do we need buy-in from management to continue 100% and probably is one of the first things you do all right so thank you for that I'll add a couple
things which is you know I like what Darren mentioned a couple of times that you just can't please everyone and one of the things that I think is also really important that I think we've talked about a couple times Darren is let's even though we don't need to build the perfect platform up front
but when we have the mandate and the buy-in and we involve everyone let's hope that we're going to end up with something that's so good that it can't be ignored okay thank you Darren for this discussion I know that everybody that's out there change management you know you're growing
or you're dying and I hope that we gave some useful tips on and considerations for doing that within your organization thank you Darren thanks Bob all right we'll see you next time let's get back to hiking we'll now tell you a little bit about who we are hi I'm Mark Dylan lead consultant
into FAQ in the advisory and coaching team and I specialize in enterprise transformations hey I'm Darren Richardson security architect to FAQ and I would to ensure the security I manage services offerings if you like what you hear please like rate and subscribe on your favorite podcast platform it means the world to us