Welcome to Software Testing Unleashed, the podcast for testers developers and all software people who want to create best quality software. My name is Ritchie. I'm a software-quality coach author and keynote speaker And happy to bring you an episode live from The Hustaf Conference in Budapest. Hustef is one of the greatest conferences I've ever been to. Top quality talks, amazing people and that unique mix of friendly and intense atmosphere you only find at truly great events!
If you have ever get a chance go there seriously – You will love it! My guest today is Peter Földhasey. Peter is in the Quality field about twenty years now…. …and for more than ten year's in the Field-of-Test automation. He has also authored the ICTQP Syllable Test Automation Engineer and Test Automations Strategy. And so we talked about test automation, how to deal with that in our projects? How can be think of test automation on different levels?
looking at our test pyramid and clarified some keywords around a test automation area! Now enjoy this episode! Thank you very much. How are you doing? Fine, thank you! I like the HUSTEF so much. it's such a great conference here... We're really familiar with each other. It is my first but i'm feeling as if i was there ever. Yeah..it has a good vibe. You know that it has evolved from small conferences all the way to such an international one And still looks intense.
There were about seven hundred people But you have direct contact with everyone. It's a nice place. Yeah, and you can easily connect to anyone like during the breaks or even doing some sessions if you just feel like connecting yeah? That is great! Peter your are an expert for test automation And I write in your abstract The story that i always see at companies.
They're buying a test automation tool because it's fancy its new good salesman whatever And then they have no strategy and no idea what to do, how to deal with it. I like that! It's good for you here. so get rid of this strategy... ...and take a better strategy on implementing test automation. Yeah the thing is.. ..I'm glad you highlighted this because For me i am too lagnostic. I don't care about tools.
Of course I do care certain projects test automation tools, or other tools that fit better there. Like an Angular one. probably I would use Cypress but for something where like... There's a team that has been using Sony forever? Would i go with the Cypress or Playwright to anything else? Carla they already know Selenium so we will just pick that! But ultimately what matters is What Is The Problem That We Are Solving And what is the architecture?
Usually I like to go with some lightweight test automation architecture, but we still need to know where we integrate into whole ecosystem. Yeah! With that type of testing and stages... That's usually an answer you have to address at beginning. Then tooling can come afterwards. Oh yeah! To highlight this question, why are we doing that? It's very important because test automation is often said.
Yeah sure We're doing test automation because everyone is doing tests automation but it cannot be the reason. Yes I think Like. if you think about like more than ten years ago and more then ten years people were saying that manual testing Is dying everything going to be automated. Is it true now? Not really. Of course, there's more tests with automation and companies who picked it up that previously only did manual testing. so thats great. but again the question is what do you want to address?
And where do you have test automation as the solution? What is the value that's brought by it. Maybe, Test Automation is a solution for what we are doing but if your not doing in right way or with just too many people maybe you're spending more money on compared to this return-on investment. Yeah yeah. and then what our typical goals I want companies wants to achieve with Test Automations? well what other problems they wanted solve? That's quite too many, so let me try to think something.
I can give some relatively more recent example. There was a gaming company that I assessed their current situation. So where did we start? We figured out what is going on And it's not just about their coding practices and existing test automation solutions but overall like the defects come from It turned out. during eight weeks they found almost five thousand defects. So what do you want to address? That, like okay.
and it also turned out that we are able to resolve only thirty percentage of the defect. so what happens every couple months... They have two or three sprints on just defect fixing so their velocity goes down zero! If I had over a better QA practices and I am able to bring in test automation with these challenges, that's the value you can bring. Maybe resolution is going. That's something that is a reason for test automation. Yeah, yeah I understand.
so really to take care about what are you solving? What is the problem? if want solve with test automation and when our team then has this goal set... ...what is it good way to create a test strategy for test automations because we can do so many things! And what did we do in? Maybe because the podcast episode will then be published. So what is... ...the way to get started with this strategy? So, say if it's twenty-twenty six and I'm pretty sure AI is going into being in the equation!
But there are things that were true already like twenty or thirty years ago but they're still true today even within an AI of the equation. What's the scope of testing and what is your approach to testing? Hence, a test strategy really going to be custom tailored for all different projects. If it's enterprise-test strategy then you can have some generic approaches that you can custom tailor afterwards.
I personally like first discovered doing an assessment show the test automation pyramid that hey, these are different types of testing we identified and this is amount of tests automation you're doing on all those levels. And it's great because You don't have to be technical to understand the kind situation. So even if I'm talking to a C-suite who isn't technical but has money i can convince them.
yes You have a lot of Focus on UI level, but you're still finding defects later off and spending a lot money on testing. Well maybe you could do shift life testing. I start doing that earlier But i can only explain it once. I'm showing it visually And then I can show that okay? Maybe six months later This is where we can get not the ideal pyramid shape which by the way Is usually not D act real. But mostly I like to go with, what can we do within six months?
Yeah. This is a very nice approach because it takes automation on another level Because usually when companies say they want to test automation They understand UI testing Or the other understands unit testing. And third one understands API testing. To look at the pyramid you see okay We have tested automation of each level and can think about how to deal together. People forget about it because they don't even know what an API contract is? Yeah, that's a really good approach!
What are the steps when we have this picture of the testing pyramid where we have automation on each level... ...to get away from the tool? generate an architecture or something like that. Okay, so let me try to be brief here because well known for its in my mind. So first of all yes the architecture...so you need like okay Let's start with the pyramid. You see the current state and a future state. See have the delta.
That gives you the scope of work For next six months If for example your scope Of Work is To Have More Functional API Testing And To Introduce Contract testing. That's what is going to be in your strategy, that you are gonna focus on these two types of testing and automating them. Then You can see like do already have API testing? And if yes Is it already automated? so Do I even build the framework or not?
Or If i do Have To Spend Weeks On It Maybe Just Go With Something That Already A Pretty Well Built Framework Like Rest assured.net. you don't really have to create additional libraries. So what happens is that when we start thinking about all these aspects, You start writing it done make your plan and then actually try it out like let's Make a POC.
Is this tool Actually going To provide me What I'm looking for once i Have basically as the tools Evaluation period Once I understand which Tool is Going to be The best fit For our project That's what I'm going to be leveraging for expanding later on. I also need to understand whether the test cases that i need, if you're creating a lot of API tests do they still require some UI testing? So again thinking about it but this is the Delta that I mentioned before.
But when we come through these different levels in the test permit Often I cannot decide, is this maybe already done in unit testing or should we that better do at UI testing? So what's coming to my mind? where you have to work together over the different levels with a developer, with the UI tester. Or whatever you have there You don't have to but it.
so first of all yes It's very important to understand What's already existing on the different level But its also important To know That what you covered in Unit Level is not exactly the same way of testing it as on, let's say a service level or an UI level. So you might still test something on UI level that is already unit tested. but the goal of the unit testing UI level you want to understand whether at the end everything works as intended once Everything is integrated.
You are seeing it visually and you're able to go through The user journeys that the users go through. they don't go through the code. Yeah, so its a different approach. There is for all this test automation thinking, we have in the ICTUB. This Test Automation Engineer and the Test Automations Strategist Sylla B. you wrote that too! You are one of the authors who built it at Syllabus.
And when i often talk to people there are different layers mentioned In their ICT UB like a generic test automation architecture and test automation, architecture, test automation framework and solution and often practice that the people cannot do not understand. what does this mean, What are these four? I have my selenium. My cypress. So can you give us a brief explanation about those four layers in which ice tea could be meant with that?".
Okay so i will try to be as simple in describing them as possible! Think of it like we go top-to-bottom or bottom-top doesn't matter but GTA Generic, so it's all about like a generic understanding of what the framework will need eventually. Like I'm going to start building a framework from scratch. What do i need for it? To work and be able to integrate into whole solution in general And these are the generic capabilities architecture. That's what the GTA tells you.
now that TAA, The Test Automation Architecture is about a concrete test automation architecture. You are working for someone so your taa Is going to be very different than mine because I'm working For someone else. yeah So and even like within a company that could be multiple TAAs. So, the TAA is an actual concrete design of a test automation architecture and GTA is not a concrete design. Next step is uh...the Test Automation solution.
The Test Automations Solution Is the Implementation Of The TAA. so the TA basically gives you a blueprint on how are going to building it in future And then actually implementation test. Okay, once you have the tests within that test You have to tough The the test automation framework. So it's a smaller part of the whole system. is this test automation solution isn't just the framework It's also the integration two pipelines the integration to different management systems.
if we think about Jenny I how? You have MCP. you can leverage that for For bringing in AI agents cursor or windsurf and bring in test cases. And now you can, you can wipe code your test automation with relevant data. so that's what's happening. uh That's also part of the solution. So which is not the part of The framework? The framework are three layers them Test scripts a business logic layer and then the core libraries. Okay, now we have clarification about this keyword.
I'm very happy to have it here in the podcast so everybody can listen and all people know it! So when you take a look at Test Automation, you mentioned already AI and there's also the key word about agentics agents who are dealing with test automation as what? What do you think is the real part that AI brings now in, into test automation? In a practical way. Not just that the tool has an A.I. name on it!
Yeah yeah... It's good to mention this because I think there was a key difference between having AI mentioned and AI capabilities versus actual real agent value provided by that tool or platform. One thing is of course prompting but Of course, it's not the past like simple chats but we are moving more to the agent world that you highlighted.
So having agents which means I have a bunch of different tasks and some of them take so much time or i'm not happy about doing this But im very happy that can ask an AI agent for me So I can focus on something that i'm more excited about. so, I can be more efficient this way That's Something. That's already changing but I think next year it's gonna Be even More of a thing. mcp? We already had and have regs so we Think the ability to bring in enterprise data Already existed Even before mcPi.
think The key difference is that I think for more people And more teams It Is more accessible with mcp and inside your id. that's just amazing like. i think that's what is a huge difference because if I want to keep working in the ID,and don't wanna open additional tabs for whatever reason then that's fine.I can work from my cursor or winster for whatever... And That's What I'm More Comfortable With and I can still have all the generation from those agents.
And will AI take over the test automation creation too? Or is this a way which is now feasible? That's that question, i cannot fully answer yet...I have some doubts about like what's gonna happen What's clear When I generate something with with chat GPT or cloud, whatever Then i still need to make changes. So first of all that means That i still Need To learn more about like how i can configure my agents better and How i Can bring in More relevant data? And ensuring that the level Data is used.
That's one thing. second i Still believe that The human in the loop Is important But the difference is that Knights more focused on coding because if the test cases are generated by an agent and a tester can cover that for you, what happens is now a year from now or half-year and we will see it better, but I think that's kind of the way we are going. And i don't want to scare anybody hopefully! But yeah what can you create in the next years?
What he told us is that we shouldn't be preparing for what the current trend and what are the best practices. We should try to figure out, what's going to be next thing? That mentality... ...is a huge mindset shift! When somebody hears this episode and sits in team where just testing it manually He said, okay but we have to do test automation. and I get my team together. What should we think first?
what can we go...what are the first steps that we can do in practical to start on with Test Automation in our project? My first question would be is there anyone already in your team who has relevant experience in Test Automations? because if you don't have anybody Developer with a coding background, of course they can start working on it. But we still need to learn. that's the learning curve That you have to calculate okay?
Or You Can bring in a test automation engineer or some Some services To help With the kickstarted. so that's The first thing I would think about. like how can i even Like Have this staffing? because if I don't have the relevant expert there I will probably fail or It Will be too costly. Yeah Yeah, okay. And then getting into problem thinking what do we want to solve with that? What are the challenges and benefits of addressing those challenges? Peter, thank you very much for all these insights.
I think it's a good wrap up of the test automation how we can deal with today and not to focus so much on the tool but more strategy or architecture. Think about what would do want to solve was there? With that has done my Asian end to thinking the pyramid style. i think That is a good way of doing dealing with debt. Thank You that you shared this knowledge with us here in the podcast And I wish your nice rest of the conference here and hope to see you again soon.
And that we can chat around in another episode maybe! Thank you very much for inviting me, let's hope we meet each other either here in Europe or in the US? Maybe yeah thank you very.