Welcome to Software Testing Unleashed, the podcast for testers developers and all software people who want to create best quality software. My name is Ritchie. I'm a software-quality coach author & keynote speaker And happy to bring you an episode live from The Hustaf Conference in Budapest. Hustev is one of the greatest conferences I've ever been to. Top quality talks, amazing people and that unique mix of friendly and intense atmosphere you only find at truly great events!
If you have ever get a chance go there seriously You will love it. My guest today is Vitaly Sharovatov. He's real-quality enthusiast which are here now also in this episode. He loves everything around software quality, and today we talked about the equinopings in software testing. How can we sell testing and quality to our managers? To our C-Levels... And now enjoy this episode! Thank you so much.
Yeah, I love that we met at Hostev or Hustev whatever pronounced properly and now We're here online. So yeah life is interesting. yes Yes And it's my fault that we meet again because I had some technical issues and the whole recording from our episode was not usable for the public. So, i'm very happy you rejoined here and take your time to do a recording in this community as it's an interesting topic. It's related to your topic of from here.
for me who step talk it's about the economics and testing. And there I knew a lot of testers knows who now say oh my god Economics, don't go away with this stuff. so But you want too give us a new mindset. There isn't that? Yeah, I have a little background story behind this. Why am so passionate about bringing the basic economic knowledge and language to testers MQA professionals?
I've worked in IT for twenty-four plus years now And... ...I've worked at many companies as a contractor and as a properly hired person.. ..and i saw most of the companies that managers and testers and developers almost spoke a different language, even though they both spoke Russian or English or Ukrainian whatever like... They could understand technically each other. but things about work. Like should we test more? Should we test less? Should release now? Or should just postpone the release?
in most cases discussions that were biased and opinionated. For instance, if a tester has huge authority in the company he or she could say no I'm not releasing this. In some cases managers forced releases and some case managers failed to consider certain risks which testers were highlighting. so it seems that we lack common understanding of why testing is done for the company, because at the end of day managers are incentivized with economical KPIs or GRs.
With economics essentially and it is very hard to be a manager And I was a team leader and I wasn't an engineering manager in It Is Very Hard For A Manager To Always Understand The Technicalities Of Every Single Decision. So From one side, managers are incentivized to deploy now because they know that their hypothesis, product manager or whoever's hypothesis is. if we deploy this feature now users will love it. We'll benefit from it and bring us money!
And then the tester says no. so the manager starts perceiving those testers quite often as gatekeepers... ...and good managers try to understand the rationale behind certain decisions made by testers. But I would say that most of the managers are far from this state, a far from being really keen on understanding the technical nuances of the decision and i've seen testers fighting and quitting and being fired for misunderstandings with the managers.
Continued misunderstandings like what is he doing this test or why? Is his stopping us? Or when manager? I've seen cases where managers force releases and then bad things happen. for instance, uh i think everyone remembers Theranos And Everyone remember it's a bad product. It wasn't even tested. and everyone remembers Titan submersible which collapsed under the sea and people died.
There was no testing at all there, so some managers make strange decisions And this pain that I see in industry brought me to be more curious about finding a common language between testers developers and managers. Since managers are always incentivized financial terms I think that we as testers and developers should at least try to understand how to speak this language.
Because if we frame our decisions in the economical terms, chances are either managers will listen better or in worst case you can request them to sign off the risks. Essentially you can say, here's all the risks! Here is what I think our company will pay for this bad release and it's upto UDM manager when it's done in writing... You as a tester cannot be fired from this. so its even adding more job security if you know how to speak economical terms. but also yeah we go ahead.
Yeah, is this feasible for us as testers? Because the whole software development process and all that stuff we are doing in test cases. Test plans and their risk mitigation... Can you bring it to numbers so they can tell about the risks of financial parts or something like that? I think its a very complex situation For Us. It Is Indeed Because there is an interesting paradox, I would say that if QA it's done right.
There are no incidents on production or very little incidence and so Essentially we have paying at with efforts in time for things which don't occur Which is very very strange to do write. But then again This is what every single manager, and generally everyone on earth knows what insurance is for. With health insurance we are paying every month a significant sum of money to cover us in the case of an incident To reduce negative impact.
So if I lived in US with no insurance And break my leg What's gonna happen? I'm gonna pay a huge bill, right? i'm going to cover a huge deal myself. So every single person on earth and every single manager knows what insurance is... ...and knows what investment is. so the invest- investment. it's very easy to define. we paying now.. ..to get some better future outcomes essentially!
We as testers cannot guarantee that there are no bugs, but we can speak about this model of reducing the harm or reducing impact. And interestingly in Europe there is a lot of regulations, right? I mean testers can start with speaking about regulations. The first risk would be what happens if we are closed by the government officials. well...the company goes bust! Right?! I mean whole companies stalls. it dies essentially in most cases.
so these risks are well understood my managers because they know that Essentially no money flow will be there if we fail to comply with the government regulations. This is a good start! With marketing, we can speak about churn and marketability of our product How much it costs us to attract new customers And how many customers quit using our products? about the reasons why customers quit using our product.
And if we see that, for instance in twenty-twenty five We lost one million dollars on custom because customers stopped using a product they churned It means that we could have done something better. and it's not always just features Its...it might be UX..It might be bugs It might be customer support tickets hanging there for like months and months.
For instance, we at Case have a significant deal of customers coming to us... ...and complaining that with the previous product they requested some bug to be fixed And that request was hanging for a year. So see that we can't speak about this. We can find comrades, we can unionize let's say with marketing sales faults. they have a good understanding what people need and They also have good understanding. What stops people from buying our product using our products?
We can ask them for the statistics. if you are winning thirty percent of deals, our deal's dear sales. If you're winning thirty percent with the clients what stops from winning fifty percent? Maybe that twenty percent increase could have been done if we improved quality. so you can gather or for instance talk to customer success and see how much time they are spending on dealing with customer requests which result in bugs...in bug reports. Again, this is pure time. It's a direct cost right?
I mean our specialists are spending on fixing and processing requests which result in bug reports. So even these times can be used for calculating what we should do better. And also whats really interesting as well here Is that all the companies In every single company i joined process of software development, software release. So there is development then there's testing.
if people are doing things right There is testing before development but anyway there was a developments testing and Deploy And sometimes and quite often this happens. features come back from testing to development and If you speak to any product manager they will be interested in time-to-market. They will be Interested in how fast? How quickly the feature becomes from an idea becomes a feature usable by clients.
So if you gather your Gira stats or whatever, use and see that for every hundred features we have fifty percent rate of the future being brought back to development. from testing You can then calculate how much time it stays How much money? It costs us To redo The Feature so We can get. there's so Much Data we Can Get There's so much Statistics so Much Financial Information.
We can ask those departments to bring us this financial information, which could prove our managers that we need to spend more or maybe...which I've never seen. We should spend less, I've never seen this. That's so true! There are two very interesting points in there. The first one you mentioned that especially asked the customer why do they not buy? Why have their problem with software which is such a good feedback and often companies don't take it.
They don't ask what customers doing or measure them And i think its crucial for success. And the second part is what I often try to help testers get into more overview looking and also on processes, quality of all our stuff. Then they ask me why should i do that? The tester has the good skills for doing that because of critical thinking, quality finding bugs and stuff. Nobody else is doing this.
so I think it's crucial we as testers also stand up to speak if there are problems in development or with incidents from clients. Testers, by our nature we have this analytical approach. We are given a test object A feature or pull request whatever...we're giving something to analyze and having worked with the product for quite awhile we know the users better than I would say like.
in most cases i've seen testers knowing users better then even product managers because product managers Quite often, not always but quite often they leave in this imaginary world of fairy tales where every single feature is beautiful. Every single feature so needed by clients! But testers see bug reports everyday... Direct feedback from clients. customer success sees direct feed back from clients and testers see direct feedback from client.
And also when testers work on a product they have to do regression testing a lot. They have two explorator testing all so they are much closer To the real users. So, they are essentially imitating real users quite often? They have to get into their skins. We have to step in to that shoes. The half don't understand what it is like to be our user What it is. light too. follow the whole process of subscribing, purchasing using cancelling whatever. The whole process is very essential.
so testers do have to care about the whole product and I would agree with you here completely. they. we know how to operate this product in its entirety And This makes us good candidates To just Have an overview what is happening with the product.
I've seen testers who were only specializing on just following testing requirements, like doing Test Plans and Test Cases... ...I have seen them becoming quite unhappy because if you are just testing and testing and don't have this overview of whats going on in a product You might feel like a cog in machine. I do not believe that there are many people who like being soulless cogs in the machine, just clicking and clicking. Most of us at least strive to be proud. our work.
If i call my mom we released such a great feature. Many clients told me they liked this feature. My impact on it was five percent, three percent two percent. But I tell her we released it! This is important. this pride makes us happier. and then again if you don't have this overview your feedback will not be perceived well. If you're saying you burned out. what's the common answer to this? Well manager work better Don't burnout right?! And...this This is our fate.
unless we take action ourselves and unless we start understanding. The incentivization and the motivation behind for managers and developers, let's restart understanding users in customer support sales marketing. We can't really improve things and redoing all work over again is so tiresome. Yeah, yeah... So when we see a tester he or she hears now this episode and says okay that's so true what Vitaly says but I don't know where to start. my schedule is full nobody hears me.
how can i start in the practical way? What was first step I could do now? There are very interesting. There are actually two approaches to this. A fundamental one would be, it's a little bit idealistic I'd say... Would come to work and invite marketing, sales, PR or whoever managers for meetings where you discuss what fears they have in each department. how can we Quantify those fears into numbers and finance financial numbers. And then are we testing enough to reduce the risks?
This is idealistic approach, I've seen it work! The tester was heard... ...and they added more QA measures to reduce risk accumulated together. But in many companies this will not happen. Managers would say Vitaly Go work. What do you want from us? Just go test your stuff, right?". And so here we should take like little steps small steps. what I suggest to do first is to analyze your own work for repetitive wasteful work. So for instance this works pretty much always Like ninety-nine percent.
if You see that From a developer Ivan you get features often in the state that are not testable or working, so unsatisfactory states. You can pair with this developer and ask them how you could help build it right? They hate for bringing their feature back to development pretty much always! It's your both interests to synchronize on. they should improve their work, how you can help them improve that work to reduce this waste.
If we do it with one two three developers... You see the reduction of repetitive unnecessary testing efforts and redeveloping efforts! Then gather these statistics show your QA lead or anyone who think will listen and get their support to scale those efforts, To find waste in the form of continuous code reviews like two stages or three stages of Code Reviews Or retesting it again And Again.
Or redeveloping The feature because the developer didn't understand requirements etc. If you find good stats if You prove these stats are true? Even with man hours! Not time-to-market cost of delay, not anything. Just men hours! You can present to your manager that this quarter you saved yourself so many man-hours Your colleagues... So Many Man Hours. Can we my dear manager scale those these efforts? Can We See How Much Direct Waste Direct Cost Waste We Can Reduce Right Now?
Simply Synchronizing Better Maybe Introducing Shift Left. If We See That Developers Misunderstand Product Manager Soften Maybe introduce something else, maybe pair programming or a test for the developer so that automated tests are built at the same time while developers working. So you reduce your time to market. Start with direct costs which is one hundred percent waste... ...which you know are wasted and feel are wasted. Then check Kanbanu's Q&A theory to see how to optimize the process.
Time to market for features. It is not hard as well. There are measures, quite obvious measures, pairing or mobbing Or software teaming which reduce time-to-market. Yes they increase costs but they reduced time to market. This your next step. As soon you gain some authority and credibility with the manager Reducing direct wasteful cost You can then propose more risky actions.
Risky in perception of a manager Because if you just come to a manager and say, instead of separating work into people let's mob the managers. So we'll see I'm not paying seven times for one feature sometimes more. i'm Not going To be doing that. And your arguments That cost-of delay will make smaller or time to market Will Make Smaller Will Become Smaller. They Won't Work because Manager is Afraid That His Process Is At Risk the process that he's used to.
So yeah, gain some authority gains on credibility gained from experience on reducing wasteful costs. first then introduce more. look where you can introduce better measures. and The next step would be To ask questions two customer support friends how much time again same man hours? How much time are you spending on checking That this request maps to an existing bug report. How many cases like that do you have in this quarter?
They can tell you twenty-man hours, then he can ask them dear manager would you support me In my suggestions to improve the situation? how essentially...how Do you value those twenty man hours? maybe your understaffed so significantly That we have churn because you don't have enough time To cover real cases not repetitive cases but real case and then you start gaining this power. Some people say it's politics, I have mixed feelings about that.
i think we should first strive to be proud of our work. as a consequence at least how works for me. when I see inefficiencies I tremble. No, this shouldn't be happening! We should improve it. we should work better. That's like continuous improvement mindset but these term is so over marketed now. But again if you see some bad stuff You can work towards reducing It maybe not know Maybe next quarter?
Maybe next year when you gain power When you become more influential in the company by showing how much money you save and only when you have proven that you Can Save Money Only then new initiatives on investing money in college assurance. I think you have very, very useful tips for the people. to start one. And you had...I think a very implicit insight bonus tip there too what i heard when you go to the developer and as you said that developers don't like if they throw back their features.
so When you collaborate with them They also are happy with your new approach and will support in management team meetings Some people around you who always support these activities then. Very true, I do not believe that by near authoritative requests we can improve anything because We value our work. so i would even say That we have a tendency to Have some cost bias when we invest some effort into doing something, like building a feature.
I mean every developer had this mistake When they worked on the future for two months. It is a mistake right? i mean after two months if you pass that feature to testing it's not gonna work! Like one percent chance at will work from first step. So Every single developer who has experience knows he needs to passed testing quite more often. so We have this sunk cost By so fallacy. when we invest money invest our effort into building something and it's enormous. It's like two months worth of work.
And then the tester brings the feature Be just demolished, like Twenty items twenty items in a bullet list. This doesn't work. these don't work. this doesn't or this is this? Well I'm. Why are you doing this to us? So if you're just Doing that as a tester Yes, it works but... It would be much better if you then sat with this developer and found a win-win scenario like in the game theory. Win-win always works!
If we find something which is to developers satisfaction and to our satisfaction chances are that initiatives will work. And the whole reason why I jumped into testing economics is, well at least i couldn't find any other way for us to significantly improve our life. when working in a company you can quit of course! to do good things. They might be really good!
You can come up with good initiatives, you're not accepted... ...you're tired and you are not smiling.... ...you worked on an initiative for one month.. ..you propose it , you get rejection. And why they so stupid alquid? It's not good for anyone. Yeah, that's so true what you say.
And I think it is a much healthier way for all the quality stuff in your company than just dealing with regulatory and we have to do... ...and then some companies are doing their tests because they're doing them as ever! And so there is no thinking about more, what can be better? What can be done in another way. What can we automate it? which processes are not that good? and uh... That's not a very healthy way to do QA.
I like the approach for more improvement and i think economics can't be the leverage there too To gain that. yeah. And also speaking of win-win When managers report their results, they always report numbers and there is so much of misuse of metrics. And yet you could reframe this whole discussion. You could help the manager Report good numbers to the upper management and he would gain Good like perception from his manager.
manageable will value you more because you helped them become more valuable in the company. They will always do that, they won't say it was Vitaly who saved this project! They'll say I saved the project and Vitaly helped. but even then is so good because they know that Vitaliy helped. So its a win-win scenario. And you're right. there's another problem with testing Is sometimes people just follow best practices or they follow their way, the way that they used to test everything.
Like even from company-to-company. if you hire a person for start up of big companies and where is your design system? Your frontend isn't testable. well no my front end is testable. it's being tested now without tests running all time. why do we need design systems? just an example because bad tester was using design system with components testing and everything worked for him in that company. But if we think about economics, We have to think about our company!
We can't just say...we're using this approach because Amazon uses it? No I mean..We can't!! We have calculate our own money Our risks Our scenario Our people. Like For instance i've seen cases when developers were joining the team And they are saying let's rewrite everything too let's say react and everything was an angular. A seasoned developer will say, well where is the economical justification for that? What are you trying to achieve?". And yet in testing I see this quite often!
Let's rewrite everything from one framework to another framework. but again can we propose these in economic terms? because if we speak about economics even things like does my colleague know how? Program in least new framework or will we have to have some onboarding and training for this person? If we speak about economics, We have to think about people a lot. And also what I hear as a counter argument quite often that if you think about economics everything becomes numbers.
well even UX Bad UX can be evaluated. not one hundred percent of course Not. no there will always qualitative evaluation, but even bad UX can be tied to a number of people who are churning for instance. Every marketer knows that every single marketer you speak they will tell you that Bad UX equals high churn. so if you think as a tester this feature has bad UX.
If have some authority and good connection with marketing or gain the soul Use marketing expertise for your qualitative decision making, saying this UX will likely increase churn. Go talk to Marketing Dear Manager if you're signing this off! So true, Vitaly that's so many tips you gave here. I think everyone has now at least one step. go on in the journey too. Thank you very, very much that you joined here this episode.
I knew it was worth to come again and we talk about because It's so valuable information for the community. So thank-you very much That You were Here And i hope We will see in The Future! Last, we will mention in the show notes also that people can join you too there. That's a very great initiative to take here! So thank-you and have a good time!