Welcome to Software Testing Unleashed, the podcast for testers, developers and software makers who live quality as an attitude. Get fresh ideas and sharp insights to grow your mindset, to learn new methods and to drive real change in how we build software. software and better teams for a better world. Hi, I'm Richie, software quality coach, keynote speaker and author. My guest today is Daniel Knott.
Daniel has nearly 20 years of experience in IT from hands-on software testing for web, desktop and mobile to leading teams as head of engineering. He's the author of books, for example, about mobile app testing. He also runs a popular blog, speaks at international conferences, and his YouTube channel, you don't want to miss out on it, it's about software testing, has a huge content area and about 150,000 subscribers now. In this episode, we talked about the pains and types of software testing.
Why are testers still struggling to sell their value to their craft? Why developers and managers think they can cut quality first? What happens when we trust AI and low-code tools to generate apps without real architecture behind them? And could prompt reviews become the new shift-left testing practice for the AI-driven future? We will explore that.
We talked about the risks of ignoring non-functional requirements, the need of communicating testing in business terms and why domain knowledge might matter more than deep technical skills in the years ahead. Daniel also shared his take on new AI-driven tools and why testers should try them out rather than trust the marketing. And now, enjoy the episode. Hi Daniel, great to have you on the show. Hi Richie, thanks for having me today.
It's a pleasure to be here. Yeah, it's a great pleasure for me that you are here in the show because you're a very well-known guy of the software testing community and you have a lot of insights in the community, a big YouTube channel and now you're here in the podcast. So it's great to have you here and also a little bit unusual for me because we both speak German as another language and now we're talking English. Yeah, I totally understand. I had
these situations as well. But you know, since the software testing community is so widespread across the globe, it's perfect to share everything in English. That's also my main reason why I'm doing my YouTube channel, also all my content contribution in English, because I want to reach more people. I mean, there's a great testing community in the German-speaking market though, but yeah, it's limited, you know, the world is not enough. That's true, yeah.
So when you are so deep in the community and you know a lot of guys meeting a lot of people at conferences and talk to them, so I think it would be great if we look at two topics and the first one is what do you think, what was the greatest pain up now for testers and quality engineers and all the stuff doing dealing with quality? That's a good question. I mean, I'm in the industries for, I don't know, 16, 17, 18 years now. And I mean, now AI is everywhere, right?
But I don't see AI as the, I mean, it's a trending topic for sure. And it's also a pressing topic for everyone in the tech world right now.
But looking back the, like, let's say 20, 17, 18 years in, in the tech world, and especially in testing, it's, it's usually the acceptance of software testing that we have to struggle and deal with, you know, I mean, we have seen the hypes like, Hey, when was it like 2010 or something or 11 when we had the hype of Selenium and other like testing frameworks Becoming like a hip a hip topic a trending topic back then they are they all said, okay, let's automate all the things
Let's automate all the testing. We don't need testers and now with AI it's it's a similar way that we are going and what I have learned over the past years is that We are really good in communicating problems and issues that we are seeing in our products that we're testing however, we are not so good in Communicating and say like to be like in sales advocate basically in in our like in our craft like what testing is really
It's really doing and what quality measures are really there for for businesses to help them grow and to bring out more products or better products, so to say. So really talking about our craft and also dealing with the business side effects of that topic. Because as we have seen now, also if you open social media, there's like layoff here, layoff there. The people, really great people, have their open to work badge on social media networks.
And it's always like for me a shocking point. Why is this person now looking for a job, but he's such a great person or she. And, and that, that gives me the exactly again, that feeling that we don't do like too much in terms of advertising our craft.
Yeah. Yeah. I think another problem now is, is, uh, also when, when the economy is not really growing and it's more starving or go decreasing, uh, uh, the companies tend to it and managers that they cut off the quality and testing stuff because developer can create the code and UX can draw their forms and so on. But the testers, yeah, we can get rid of them because we have to build our product anyway and testing is, yeah. Yeah, it's an easy way out, right?
I mean, of course developers, they have to test their code, right? But I also have seen companies when I was like in a consultancy site that like a lot of companies put a lot of stuff on the shoulders of developers. Like not only like to do the coding part, like to do the testing part, which is for me given that they have to do it, but also, I don't know, maintaining pipelines, running the product on production and all this kind of stuff.
And I think if companies decide to cut on quality these days and replace testers, also developers like junior developers, for example, with any AI that is available on the market, it might work in the short term, but in the long run, I just talked to a person last Friday on a conference that I think, I truly believe that we have like in five to 10 years, we see a huge
demand in people who are able to understand system architectures. It's not not directly connected to testing but well testing plays a big role on that side too but I think that we have like a huge demand where people really understand what has been generated or what what systems are doing because it's so easy right now to build your own app in a couple of minutes, your web product,
whatever you would like to publish. It's great for startups and also big corporations going that way and which is not bad but going that that way too fast can lead to really bad situations in the long run and looking forward to it. Let's see let's see what's happening maybe we see a huge demand on quality advocates, testers, test architects whatever is available on the market but it can be painful for companies so they should really think twice in getting rid of the people right now.
Yeah, I think you mentioned a very important thing, because when we look at now at all the web coding stuff and creating apps and websites and programs, and there is no real architecture behind that.
We already knew that from the non-functional testing, if performance testing and security testing, if we there have issues, they are mainly addressed in the architecture, in a bad architecture or a gap there and something that And it's so expensive to change the architecture and after it is released and so on. So I think there is a huge demand on looking on that part two when we go now to AI-driven development. Yeah, I mean, yeah, I totally agree 100%.
I mean, right now, the tools are in a very, very early point in development phase, like really not that mature. But who knows what's going to happen in five years. Maybe AI is so brilliant. They are really so good that they also oversee architectural patterns and are able to suggest, I don't know, a rewrite of the architecture or also the things that you just mentioned in regards of non-functional requirements,
right? So this is something. And also one interesting side effect on the, it triggered me when you mentioned non-functional requirement. I lately talked to some product managers and it was also like, "Hey, when was the last time you guys were thinking about non-functional requirements. And it was like, I don't know, actually, what do you mean? And
they really started something on their side, right? Because as you just said, I mean, with accessibility coming around the corner, this is something that triggers people to at least think of accessibility stuff. But not too many think about load and performance tests, for example. Some have security testing on their plate, but luckily they get help from external experts but it's a huge topic definitely to also not to forget about.
Yeah, you mentioned that we have to learn to communicate more the value of testing and not so much focusing on communicating the problems. So how can we do that? What are the points we can bring to our management, to our leads to give them some points why we are so important for business and for the product. Yeah, I mean it's not, there is no silver bullet to and how to communicate that topic. No, actually it's not. It really depends on the people that you have in front of you, right?
I mean, and with people I mean like really the personality of that person. Like whenever I talk to people that have not too many insights into testing or they don't understand the value of testing, I try to get some more information from their history, from their back, like what have they done before, where they're coming from, maybe some, I don't know, different technical angles or they were like complete coming from,
I don't know, from economics, marketing stuff. So what is their background basically? And then I try to catch them where they are, right? If they're coming more from the business angle, I try to communicate more in the business value. So in case you develop a feature and there is a certain revenue connected to that feature
outcome, so I go with that. Okay look, our KPI is to, I don't know, reach a 1 million in revenue in the next month and if we don't do like proper testing and from a holistic point of view and we are not covering this and that, it's likely that we're not going to reach that target KPI, for example. So I really try to get them where they are. If you talk to tech people, then of course, you need to dig into architecture.
Maybe you have a system architecture map open, see, OK, you know already this part's automated, this is our weak spots. And then I try to sell activities in that direction. I just talked about non-functional requirements, or if we do some more API testing, contract testing, whatever is in the toolbox of that person. So that's always an important part.
And with what I've seen in the past is talking more business-related KPIs in terms of revenues or also user satisfaction helps a lot in terms of getting accepted by the things. And it's not working out if you go there and say, OK, we have 20% code coverage. We need to raise the bar to 80%. It's like, yeah, it's a number for them, right? What does it mean actually code coverage? Sounds too complicated. And that's my approach basically. - Yeah, yeah. Yeah, I think I like the idea.
What comes to my mind is that the gap from our real daily work to this very high level KPIs like user satisfaction. And so there's a huge, huge gap. And I see the manager in front of me and asking, what are you doing to get into this KPI? And what are the steps there? So I think, how can we overcome this gap? How can we address this? That's a tough one. How can we overcome that gap? No real answer to that one right now.
No, it's really-- it's so many angles that you have to cover from what's the business that you are working in. What are the competitors doing in terms of business value and adding value to products? Or what are they doing? Also, what are the customers requesting in terms of customer feedback? So there's so many data points that you have to basically connect in order to make the right next steps. Yeah, OK, maybe that's also a question for our audience.
And they can comment what they think about the topic. Yeah, definitely, for sure. But really, I think there is no right way on that side, right? There are so many-- because we're working in so many different industries and so many different situations right now. I mean, I talked also to people. They don't have any issues with AI coming around the corner right now, because this is such a sustained business. They're not affected to that. Data is safe and not somewhere in the cloud.
But yeah, still, it's interesting times right now. When I talk to more business-like testers, they often say to me, yeah, I like my testing stuff. I have my foundation level. I know my test methods, doing my test cases. But all the technical stuff is nothing for me. So I don't want to learn it. Do you think this is a mindset or an attitude that can survive in the future? Can we just rely on this manual business tester?
But do we have to change this role and to get more open when you look at all the AI stuff which is coming? And so what do you think? - It's an interesting point in time, right? Because I mean, years ago I would have said like, okay, they need to get technical. They need to learn at least one programming language, need to know into architecture patterns, clean code and stuff. Not saying that this is not important anymore, right?
But right now, with all the AI tools coming around the corner, and before we had this huge wave of no-code, low-code applications where normal people could write test automation, so to say, I would say with AI coming around, and AI will not go away in the next years, decades.
So I think it's not too bad to be not too technical, I would say, because I would say it's more important to be more like in the business expert in the long run, especially with all the vibe coders and stuff and tools that we have generating code and generating features that we as testers or quality advocates need to judge the outcome of that LLM or that generated tool. So that's why it's important to learn more about the business domain that you are in. That's really helpful.
And then, of course, having-- or like to know the foundation in software testing. This is definitely a good thing, I would say.
However, I mean, having a little bit, like a little fundamental knowledge about how a system works, like what's a response, what's a request, like really bloody basic stuff, like how a client-server communication works, how modern tech stacks look like, this is definitely helpful also to generate new testing ideas and also asking the right questions in terms of where is, if we stick to AI, where code has been generated, in what area, and what areas are still human,
like human developers in the lead. So this also helps again to oversee the whole feature development.
- Yeah, and maybe even more to think about out of the box, what can be good test methods or good test cases for our product and not just thinking in our business box, but looking outside and on the edge cases, which is, I think, more important in the future than it's up now, because the AI will generate the code and it's maybe it's done, but to look at the edges and outside of that, it would be very interesting, I think.
- Not only that, that is definitely a good point to start on, but also, I think this is also something that I haven't seen too many people talking about is to actually do testing or to do quality checks on the prompts that have been used in order to generate that code. So this is also an idea that is just in my head for weeks now, but I haven't had the time yet. So it's the first time I'm talking about it.
But so maybe if you heard someone talking in the community, I would love to chat and exchange on that, Because I think that's also a big entry point of potential issues. If you have developer and she is doing prompting with an AI, we all know that context matters when we use AI and prompting and how the prompt was written. And sometimes you just change a little tiny bit of the sentence and the outcome or the output of the prompt is completely different.
So I think that doing these prompt reviews is also something that can become huge in the future, right? Especially still people writing the prompts on their own. I mean, if they're using LLM or something else to generate the prompt, something different. But this is, for me, a new way of shift left testing, you know, to do like review the prompting that go into the system. Yeah. Yeah. Yeah. OK. need in the future as a tester too.
And another part is what I wanted to have your opinion of is about the tooling. You're testing a lot of tools on your channel too. And in the last two years, a lot of new and old test automation tools write an AI as an appendix to their name. And now they have AI. But do you have any examples what they really benefit now from AI and what is really, really working for daily business now then beside the marketing stuff?
I mean, I'm always saying that on my channel too, is like the tools, when I'm even before AI hype, the tools that we use for testing, they should be easy, you know, it should be easy to install. It should be user friendly because it's it's our extension to to our work. So we need to speed up with our testing time. So that's why I don't want a tool that I need to configure for days and weeks and to do some dependency checks and downloading gigabytes of libraries just to get it up and running.
So this is something that I don't want to see. And this is something that vendors should really keep in mind. And from an AI perspective, I have seen a lot of tools doing really great stuff from test case generation into your ticketing system. You install the plugin, for example, And then they scan your acceptance, not the acceptance criteria, but the requirements, the designs. And based on that, they derive basically the test cases, which is really cool.
I also have seen tools that are completely, let's say, they're hidden. You'll install it with a library. And they do some magic in the background. So they basically track the customer, track in terms of they see what customers are clicking, like similar like analytics tracking. And then based on that data, they completely tell you, OK, that's the user journey through your product. And here are these cases that you should automate. And this is something that I would like to see more often.
And I mean, that's really cool. Or you have, again, similar to all the white coding tools that we have in the market, similar for testing. You have a prompt interface. You prompt something. And then you get your test automation up and running, or your configurations up and running. There are a lot of tools on the market available for that kind of stuff already. So this is something that I would love to see.
And then in the end, also, it's up to the tester to decide and to pick the tool that fits best into the tech stack of that company, of the current situation. And what I also always tell on my journal is to try out as many tools as you can. if you have time and you can, or do it in your spare time, as a hobby, like I do with my channels, I love to do that, right?
I mean, I get trial versions, I install them, I try them out, I just would like to see what's working, what's not working, just to see where our community or the industry is heading to. And with that, you get a feeling of what really is in for you and what really matters to you in your current situation. As I said, I cannot say, okay, use that tool Like we had, I don't know, 10, 15 years ago, everyone would like to use Selenium. Yeah, we use Selenium, let's go for it.
And now on the mobile space, back then it was like Appium, let's use Appium. There's other tools on the market that you could use, but they can be the right tool for you, but not for me. So that's why it's always important to do your homework. - I think it's so important to try out these tools because the marketing and white papers, they are telling a lot of stuff, what could be in there. - Yeah, I mean, I would do the same. if I would be on the other side, right? - Yeah, sure.
- You have to play the game with like the others, right? And I mean, Zio is now going down the, you know, going, it's not the best sustainable business right now with all the GPTs and clothes and stuff like that because less people are using really like search engines anymore, they go to those interfaces. So this is definitely something. And then of course you need to use other channels as a tool vendor right now, like social media. So definitely a huge topic.
Conferences still, where you can really present to the best possible audience your tools. And so yeah, they have to be creative too. I had a very nice situation this year in April. I was at the Swiss Testing Day, and they had a test automation lab where the tool vendors had prepared some workstations. And you can try out the tools and sit there and use it, not at the booth in the exhibition area, but in our own test automation lab. It was very interesting format for something like that.
Yeah, I like that idea. I've never seen it before because going to a booth is always like you have to jump that hurdle. Like, OK, I just want to read it. And then there's some salesperson grabbing you in. There are a lot of great vendors out there doing a really cool job on that side. But trying it out firsthand is even better. Yeah, yeah. Daniel, when you think of the future now, and what is your one tip to give for a tester? What can I do now to prepare for this future? What should I do?
- Yeah, I mean, read and get into AI. I mean, I think I cannot tell it even more. I cannot hear it anymore, actually. Like, get going there. I mean, I still remember when I started in mobile testing in 2010, Appstore were just around the corner and every company jumped on that thing, like, "Hey, we need to have a mobile app." And this was a huge, let's say revolution in terms of industry technology-wise, and still companies out there that don't have a mobile app yet or like a really
crappy one. So it took them like more than 15 years. But now AI, as we have just seen in the last years, that the speed is, I don't know, 10, 20 times faster than the mobile revolution that we had 15 years ago. So that's why learning as much as you can about AI is helpful, like prompting how LLMs are working, also like internally, how they're doing this stuff with all the vector databases and stuff and the different principles.
I mean, it's not really connected to testing, But it's definitely something that will help you to shape your testing skills. So this is definitely the one thing. And this is not something that-- it's not a tip. It's more for me, or more like a wish for the community. So we should not trust our capabilities and our knowledge that we just gained over the past years. So don't listen to the people saying, oh, we don't need any testing. Your skills are irrelevant. That's not true.
the fundamental testing skills that we learned from the bottom up and since years now, since 10, 15, 20 more years and stuff, this is still valid and still helpful for AI as well. So but it's definitely tough times at the moment. Yeah, yeah, yeah. Okay, thank you very much for this insight and for the discussion about this topic. The beginning I talked about, we will talk about two topics.
The first one was pains and the second one was was hypes, but I think we've emerged them together and talked about the pains and the hypes. Yeah, so I want to mention here your YouTube channel, we will also link it in the in the show notes so that everyone can move on from this channel also to your channel and to look at your videos.
And yeah, thank you very much that you were part of the show now and I'm sure that was not the last time you were here because I think I now have a backlog with a lot of topics I want to discuss with you. Yeah, perfect, let's move on. I mean if the audience also would like to hear me more often here I'm more than happy to share. Yeah, that's great. That's part of this podcast. I like it so much. So, Daniel,
thank you very much. Have a great time and see you soon. Yeah, thanks for having me again. See you soon. Bye-bye. (orchestral music)