Welcome to Software Testing Unleashed, the podcast for testers developers and all software people who want to create best quality software. My name is Ritchie. I'm a software-quality coach author & keynote speaker And happy to bring you an episode live from The Hustaf Conference in Budapest. Hustef is one of the greatest conferences I've ever been to. Top quality talks, amazing people and that unique mix of friendly and intense atmosphere you only find at truly great events!
If you have every get a chance go there seriously You will love it. My guest today is Martin Hosens. Martin has over nineteen years experience in QA field and is also a very active part of the software test community, especially helping testers and QA people to get on stage in conferences and support him. To speak up! We talked about DFX developer experience or developer excellence? And how build quality into that.
based on the metrics by Dora & Space we looked at how we can support this process. now enjoy episode. Hello Martin great to have you here. thank you Richard. it's lovely to be here yeah so we had so much contact here in these days. i really like you because you are such a friendly, open-ness guy. So I want always get around with you at the conference but also the spirit of the Hustaf is amazing! It feels warm and lovely...it feels like coming home.
feedback from first-time speakers here, that they're experienced as a very friendly place but also like very warm cozy and everybody's really into it. So that's really cool! Yeah yeah... And good content the good spirit amazing. a lot of people. he is several hundred I think That's great! It was just... Yeah it's great and i came back to or started in Hustaf three years ago. They were starting up after pandemic.
about five hundred people in the Agvarium Club where we had the after social yesterday. And I saw them grow in those years to now. seven hundred people again and in this lovely venue being part of a PC as well, This year was new for me but also great experience. so i'm Also very proud what were able put together. But yeah As to content and speakers at level quality man. We just have track and couldn't choose. There are three talks across the board, so yeah.
Yeah it was a great program and you did that great selections. I wonder how can i... How do they get into the programme? That's anyway! I wasn't there but I was happy about this speech. You had also a speech here about Dora in space to bullshit. bingo words. we have to clarify what is behind those stuff. Yeah, exactly. So Dora, Space, DevX... I called my talk putting the queue because it's a testing conference for QA in DevX like DevSecOps and all that kind of mindset.
but since DevX has been around since two thousand twelve officially sort-of kinda. initially when i learned about this earlier in my career Like ten years ago i was thinking hey these are cool metrics. This is very closely related to what want to do and am doing as a tester, I was still working in teams at that time. So it got me thinking... And now i've had an opportunity to do a project last year In the role of a DevX coach.
But I'm not a devx coach or developer but with these metrics The focus plus challenge happening is very QA and quality focused. so I entered as a dev x coach leaning on sort of quality and testing, which was a challenge for them. But they tackled it from a DevX perspective. how as developers are you struggling to do your work? And how can quality testing and all the things that I talk about help you in your daily job?".
I thought this is very interesting because now as a tester we have different role and connection with our developer friends who were very close to us in teams and projects To talk to them about our goals and how are things that we achieve? And the tools that were used in the thing's there. We do to bring it all together In those metrics that we couldn't all get behind. business development and QA. Yeah, yeah so that was the inspiration for a talk. Yeah That sort of where I started from.
but before we put the queue into the defects part What what store and space? yes? Yes So good question. So first off developer experience. DevX was sort of coined as a term in two thousand ten, two thousand twelve. There was scientific paper that sort of posed the question hey we're talking about user experience a lot The end-user? Uh...the user of our applications apps whatever. How about the experience within Our teams building software?
and uh after that sort Of paper A startup started working on this And trying to put metrics towards This mindset and it boils sort of down to four metrics in Dora. And Dora stands for the DevOps Research Association, I think. The A might escape me? Yes! They were the trendsetters making these measurements into... like DevOps was really hot back then every team was moving towards DevOps and ownership in teams and teams are now capable deliver what? how Are you doing that? well capable?
Now, here are some metrics that you can measure but then also try to improve. Okay cool and that's where Dora started. Dora is officially eleven years old I think two thousand fourteen or two thousand sixteen. the startup was started And then in twenty eighteen Google acquired it. And when Google acquired it, of course got a lot of attention. So much attention that internally in Google they started using these metrics and starting reporting on their yearly state-of-devops report.
Now time passes people keep thinking about this metric are very oriented onto how? How we doing things or the right way. then came sort another movement from different perspective but some competitors Microsoft, a university I think the University of Maryland and another company came together. And they created.
space in Space doesn't stand for like A word but it stands for all The metrics that they bring which are very closely related because I can sort Of bringing up right now Because my talk really dives into Dora where? But space also looks at how are you feeling as a developer? Are you supported, do have enough mandate and directive on your own agency tries to identify that measure. And then you can try help fix it.
but there's the PIF in Space stands for performance performance of how you do your work as a developer, and that's closely related to all the topics in Dora. Because if you're doing your job well in performance space then... You have fast mean time-to-recovery, low failure rate All those kind of stuff. very tangible stuff Dora. they enter into space. I didn't talk about Space again because it is more really close to the developer space.
I hope to do this talk at developer conferences in a longer format where i can bring those two together. I did a whole like diagram drawing those out and then highlighting where the quality is, all of these aspects... ...I dropped that for this talk because.. ..i'm only talking about the tangible Dora metrics we as testers have very strong impact on. Does it make sense? Yeah does makes sense! And what are these metrics when you put our quality power?
So the Dora metrics are meantime to recovery, that change lead time. Change failure rate. and did I mention mean time-to-recovery? Yeah. Yeah. Mean Time-to Recovery, change failure rate, all end deployment frequency. Ah okay yes so deployment frequency how fast can we deliver changes? Which is super interesting because you know, whether you look at it from a process point of view or are very into tooling like CI CD and pipelines in quality gates.
There's always things that can tweak to make the pipeline faster smarter better. For us, it means that we need to do parallel testing have a right coverage in the test automation pyramid. If you only rely on very slow UI tests You can't do Parallel Testing all-you want but It's never going be as fast without a test environment spinning up then All the unit tests that she could've written and run In at same time Right? So looking at those kind of metrics Deployment frequency is important.
change Change throughput Forget the official name, but like if somebody comes in with an idea which is everything we do and software delivery How fast can you go from? An idea to working out their requirements a step that we sometimes forget Which bites us in the butt. And then yeah have a longer You know change implementation time. Yeah To understanding this maybe pair programming so you don't you can skip the peer reviews Do unit testing in pair, you know continuous testing So you TDD.
all those kinds of things help Shorten the time that we come from an idea and then including the deployment frequency You can you know turn around in a week. Yeah, if that's something that your company wants right? You can go play this. You can Go ham and say well he couldn't deploy any five minutes. so if you have an idea now Benjamin in the back He's our star developer push it to two production in ten minutes. do you want that? yeah is your? Is your setup safe enough?
Do you have the right quality gates and coverage for the risk that your company deserves, and desires. I don't know. Yeah And then change failure rate i think is most closely related to what we do as QA Because We find those failures And if we do a good job, we find all the bugs. Typically not because you don't test everything and there were a lot of great talks at Hustler this year about value-based testing and devalu offtesting. We need to think about what is the most conscious way?
What does add value? beacuse the highest risk isnot necessarily the value that your company sees. So theres a tradeoff where we have to take into account. Because they change failure rate You know...what doesn´t really mean? it's a metric, right? So we need to... Metrics are just metrics. It is insight like if I find the bug Is an important bug. well We need to discuss this Right! Its finding its information that we now have because i was looking.
We Now Have these metrics Because were measuring But it doesn't really say anything. There Was A Question From Somebody In The Audience Saying Yeah but If We Measure These Things Don't They Just You Know dissipate or magically improve because people are doing small, tweaky, hacky things. I'm like yeah you know. and then we talked about the Cobra in India right? So you need to understand value behind these metrics for your company. And what is an acceptable failure rate?
What's an acceptable deployment frequency? And you did this as a coach to get into with your quality perspective, so more focus on the quality parts in these metrics. How do team and people react when they are doing that? Yeah, so in this role I joined sort of the support team of DevX coaches. In an organization that had roughly twelve-thirteen development teams they were sort of spread around and divided into like streams or areas front end back end logistics all that kind stuff.
And then we were assigned to teams crossing quarters With the teams, like we see these metrics. We have some metrics. what do you think of? The metrics. well You know we have a high failure rate. We struggle to uh...you Know put fix those things in production. so Our meantime-to-recovery is very low and our Change lead time Is very low because we're so occupied with fixing these issues And then the pipelines are slow. So oh boy! So with these insights We came to a lot of like, huh?
I think i know what's going on. Yeah A lack of control quality coverage understanding What Quality actually means To your product to Your team what ownership is where you're liabilities are Like all the conversations that we've had in All our career. yeah like all The testers will pick out things like oh i've done That i'm doing this. i am having these discussions now In my work and same in This project right it. It just helped that we had these metrics.
And in my role as DevEx coach, I could support the team but eventually they needed to sort of do it on their own. As a coach you know You don't want to be in there or be the enabler. Yeah! You need to be able to step out. So through that process We identified...we came up with some experiments as we call them Because if they fail its fine. You now were gonna try some changes In your processes and tooling Whatever skills maybe?
After, I think we usually took four through six weeks depending on like the implementation and experience that we picked up. And then sort of saw which one they liked if some of the metrics improved because we were checking on them. Yes! Then at the end We did a retro and handover to the team... ...and then we would check in afterwards. hey how is it going? It's been three months. Are you still doing the things that we said were going to do? Not always.
What are typical things that change then, because when I see this matrix or think of these metrics... ...I find it very hard finding an impact on them and changing in existing environments with a team for something new. That's interesting! In one team was the web-based team responsible for the web portal And they basically were struggling with a test environment. It was as simple as this, we have new features but we don't have any test environments.
We have an acceptance environment that everybody's using to tests. So if you break it then everyone hurts and anybody else breaks. so one thing is just add a personal integrated test environment for yourself so you can speed up your testing and go to acceptance in confidence without being troubled with that area.
That's bad at the team, it gave the team more agency to look at quality on their own in sort of isolation as she should And helped that very specifically With all this speed like well not deployment frequency but the lead change time meantime recovery All those kind things improved because we help them with one of those issues. When you talk about experiments, how many experiments did you take? How many do they resist after your leave or the project ends?
when you say their metrics are good Do they only get implemented or just revert to a former state? Yeah, so this is human nature right. if we do something Or If We need To change Something There's always resistance. The benefit of This Is that We come in like As A blank Sheet and With An open Hand to Help out Where First They're Observing And Sort Of Like Asking Around. Hey What's Troubling you what's Hurting You? Why are you not doing the things that you want to do?
And then people automatically say, oh especially in Dutch culture ask somebody to complain. Do we have half an hour? and by understanding them being there to offer support a lot of people already sort opened up. well if it doesn't work at least someone tried help me because they were here for my manager typically has other objectives, wants me to go faster. And they already had like the support agreements with the managers and that kind of stuff.
so ultimately when we workshop ideas The cognitive load off a team is something that you need to take into account. So if whole team is complaining about all the things You need to drill down on but what change would make the most impact for you now? We will be back next year! But for now, what's the first one or two things that we're going to change? So keep the barrier low. The entry level and help them engage.
make sure that the friction of that change is low enough That people can get behind it. still see the carrot at the end of the stick right Because you introduce it as change. You know you are gonna Try something new. It's an experiment Right We're doing something exciting And you know, with a positive mindset but also helping mindset. I think that brought a lot of like goodwill to anything we've got. and then when you leave as soon people go back into their sprints.
one will say yeah didn't agree more time than what were gonna do in this. maybe there it'll always sort of sink away and come back saying hey how is he going? Yeah well forgot about the stuff... That's okay was an experiment But how are you doing? Yeah, not so good according to the metrics. I'm like well maybe we can try again right and then sort of it picks up And they see that helps. It's sort of sustained. This is also important.
You have small chunks Not everything at the same time because Then you say what worked Maybe this or That. So with that you could also Very specifically pinpoint which of the changes helped and if we did too, typically would pick two in different areas or for two different metrics. So that way they try to change things and actually try to uphold those. when left as a coach you could track those matrix. is it actually benefited them? Or just slow down.
then you can also say well this experiment failed because even though you didn't see that you're improving. That's fine, we'll try another experiment! When now someone hears this episode is like okay I want to do this stuff too. how can we start? so for our own... Is there a blog post? You can rank them in the community or book or YouTube channel something Like that way. well How Can We Start To Get Into The Topic?
Yeah, I think that's super interesting because all the material that i found in my research into this topic coming and looking for a QA perspective on DevEx. I couldn't find much. one of the speakers here actually pointed out he published an article about this very specific problem at his company. it was Tiago shoutout to you Tiago! About these last month. so that is super interesting right? If you want to find something, I think it's important to understand where comes from.
Yeah go look at the state of DevOps report that Google publishes every year. Look up what those metrics really mean aside for my explanation. And uh i'm also about to sort of Publish an article with all the homework That I didn't put in my slides. okay Where are qualities? To be found?
yeah and like a very big overview With all the activities in testing that You can apply to any one Of Those Metrics talk I highlight, for example that test automation is very closely related to all four of the metrics in a sort-of different way. In the twenty five minutes i had today at Hustaf ,I couldn't really explore and extrapolate on it. but this All the things I had on my slide, it might actually become a white paper.
But yeah there's a lot of cross-referencing to do that we as experienced QAs can already do. but maybe you know more meteors or junior QAs may need bit more information and guidelines... ...but i think this slide is good starting point for cross reference like the basics of Dora with QA practices. find those mappings in your company. You're doing stuff, how can you start measuring this? that developers or your product owner? Or maybe like the CEO thinks hey it's really cool.
if I have an idea and you increase change lead time then i will be a happier person uh... we'll be better companies. indeed We talked about value at this company. If we learn to talk value as QA I think it's super important for us to remain relevant, visible because that is typically an issue with being in QA. So this one of those ways you can make your work more tangible and measurable You come to people who speak a different language In a language which has already been popularized by Google.
Okay. Makes sense? Yeah, that makes sense! Cool. So we also have to follow Martijn on LinkedIn and so that we get the white paper if it's finished. Yes yes yes We'll do a little bit of pressure for you now. Thank you very much for being part of this show. Of course thank you That you shared all this knowledge here in the community as well And I appreciate your work Here. You're doing so much community networking.
How will see soon at another conference or maybe next year At The Hustaf the conference. Thank you, it's been a pleasure Richard! Yeah for me too. thank you very much.