As a developer it's your responsibility to make sure that what you do works in. The only way to do this is by writing tests that prove to you and to your team that your code works and automated testing is just as such an easy and cheap way of doing it. And that's why I say in the book, that's effective developer is someone that effectively tests what they do, Hey everyone. My name is Henry Surya with Robin.
And you're listening to the technology, you know, podcast the show where I'll be bringing you the greatest technical leaders practitioners and thought leaders in the industry to discuss about their Journey ideas and practices that we all can learn and apply to build a highly performing technical team and to make an impact in your personal work. So let's dive into our Journal. Hi, everyone.
Welcome to the Chocolate, you know, podcast the podcast where you can learn about technical leadership and Excellence from my conversations, with great thought leaders in the tech industry. If you haven't, please follow the show on your podcast app and social media on LinkedIn, Twitter and Instagram. And for video contents package. You know, is also available on YouTube and Tick-Tock. And if you are willing to support my work, please buy me a coffee at technology. Know that deaths.
Last tip or subscribe as a patron at technology. Arnold Dev slash Patron. My guess what? Today's episode is Mauricio a niche, Mauricio is the author of effective software testing in this episode. He explained how to become a more effective software developer by using effective and systematic software, testing,
approaches. We discussed several such testing techniques such as testing pyramid, specification based testing boundary testing structural testing or code coverage, mutation testing and property testing Mauricio also shared his interesting. You about test-driven development or also known as TD, which you may find quite surprising and towards the end, he suggested the one area we can do to improve our test maintainability.
I hope you enjoy listening to this episode and learning a lot of things about effective software, testing and several different systematic testing techniques. It will be really great if you share this with your colleagues, your friends and communities, and leave a five-star rating and review on a podcast and Spotify, your small help will help me a lot in getting more people discover and listen to the
podcast. So let's go to my conversation with Mauricio after a few words from our sponsor. Are you looking for a new cool swag pack, leader, No, no office. You some swags that you can purchase online. These wax are printed on demand based on your preference and will be delivered safely to you all over the world where shipping is available. Check out all the cool tracks available by visiting technology, you know, the death / shop and don't forget to break
yourself. Once you receive any of those tracks, hey everyone, welcome to another new show of the technician our podcast today. I have with me Mauricio an inch He's the author of a book titled effective software testing as the title says, we'll be talking a lot about software testing today. How to do it effectively and how to do it properly. So our Niche is the technique at the end a company in Netherlands, I believe.
And I think, what is really cool is a niche, is also a lecturer and he has won a computer science teacher of the year in 2021. I think that sounds really exciting. Maybe you can share a little bit more about that. So thank you so much for this opportunity. I'm really looking forward to
discuss about software testing. Today, thanks Herring party, invites Marisha. I always like to ask my guests to share, maybe a little bit more about yourself telling us, your highlights or any turning points in your career. De interesting for us to learn from. Yeah, cool. Yeah. So, my name is Mauricio, I work as a software developer for
almost 20 years. I had a little break when I went fully to Academia. So, I spent five or six years as an assistant professor in software, engineering doing research in software engineering. So I wasn't coding professionally for the period. And my career was always about, or my passion, in my career was always about software, design, and software testing.
So, how to model your code in a way that's easy to maintain and how to write tests for it. And at the very beginning of my book, I tell this story that one of my first projects in my career, as a team leads, we coded some software that was supposed to run on the hardware. And then we spent six months coding version. Number one, we flew to another country, we installed it, it didn't work for 24 hours. There was Super big bug and that was super sad that my first
software didn't work for a day. And that was sort of the cake for me to start focusing on testing. So since then, and that was in 2006, so it's been a while ago. I've been trying to write tests for everything that I do, and I think the book was just a consequence of me putting those ideas on paper. Well, it always amazed me like, your first experience in your career can actually leaves you a lot of impact to your career, right?
So you mentioned that your first project maybe it didn't work as much as you want. Is a team and that actually let you two have more passion in testing and I think let you do also writing this book. So maybe tell us a little bit more about what if you can reflect back on that experience? What actually probably some of the root causes of why the project failed and it didn't last for a day, even?
Yeah, lack of testing was the main reason we were doing a lot of testing for sure but manually, we had beautiful Excel, spreadsheets full of things that we Test. We even create a simulator because in that software, we would talk to an external party via serial Port. So, we even create a simulator, so that we could test a little bit more like the real world, but those tests are never
automated. And what happens is as a developer, you change something, you test the happy path of your change, and that's it. And yeah, then we had a bug and a bug that was totally preventable by very basic automated testing. So there was a click for me, as I said. Yeah. And also you set you go back to Academia, right? So you spend five six years and I think that also lets you to win this award. Maybe tell us a little bit more about it. What made you?
Leave a lasting impression to the students I guess? Yeah. So it's very hard for me to decide if I like working in Industry where I can write software and deliver value right away to people or if I like Academia where I can just sit and reflect about how to make software Engineers better at what they do. So my life was always a little bit like this and it was working as a developer. I did my masters and then I said, oh, that looks cool. I did it.
HD. I finish my PhD. I said, well, maybe I want to try this a little bit more. I did a postdoc. So, I took a postdoc position and they're like, yeah, it's amazing to the research, right? And then let me try being a professor full-time.
And what usually assistant professors do is they do research and they also do teaching and I started to teach software testing here at the Delta University of Technology, a very important Technical University in the Netherlands and first time I gave this course was 2017 and I've been doing this course until today day and I think teaching testing has been also an amazing experience for me because it made me just understand way more about everything that I was
doing. I had to formalize my thoughts so that I could pass it to people. And I think, throughout the years, I've been doing lots of changes, may be a big one was during Corona, because when Corona came and then we suddenly switch to online. I didn't want my students to be in front of the camera as a compulsory activity, you know, because people had their own challenges. So I decided to open my lecture notes and I remember I just got it.
Clean assistant here and then I said, those are my lecture notes, just make it a little bit more. Beautiful publish it in a website and that's how we're going to do. The course, people can read my lecture notes and to my surprise and then they started to get emails from people from other universities. Hey I'm reading your lecture notes, this is very cool. Do you have exercises that I can use? We have slides and then same thing in your number 2 and then
this is started to grow. And at some point said, well maybe I need to turn this into a book and I contacted many, I submitted a proposal. They accepted it. My book went through. They are very thorough review. So the book just got way better and I think in the end this teaching award that I got. So Delta has a ward every year that they give to teachers that are doing something impactful, and this is based on the surveys from students and Etc.
I think a lot of it was because of this first switch from, instead of me lecturing, I just gave them a book. That is easy to read. That is very practical, and in, creating the book itself, I think students, like very much the content of the course. So my course is far from Those theoretical courses usually see about software testing that you never see source code, it's more practical. So all of these combined gave me this award that I'm actually
super proud of and even today. So I'm giving this course right now, actually in this quarter students are reading my book. And again, the feedback has been super nice. One of the students came to me this year and said, you know what, this is the first book in my bachelor so far that I read cover to cover or back to cover. So yeah, that's the story. Well, thank you for sharing, such a beautiful story and I think, yeah, it always gives you a very proud moment to hear such
impact, right? Especially when someone reads your book and do and especially it's a technical book. I must admit myself. I rarely finish technical books and to end because there are hundreds of pages, sometimes it's dry. But I think, looking at your book, I think I can see. There are practical things. They are sample code, they are things that guides people to actually understand your thought process, which is a good segue for us to start talking about your book effective software
testing, I think one key. Key sentence that I pick when I read your book is that you mentioned to Be an Effective developer. You must become an effective software tester, but in Practical World, sometimes developer, and tester, there are two different roles. So tell us more about this. Why do you say that to become an effective software developer, you need to become an effective software tester. Thanks for the question.
I think it's, because as a developer, it's your responsibility to make sure that what you do works in. The only way to do this is by writing tests that prove to you and To your team that your code Works show indeed before in a even worked in environments like this where I would just focus on coding and then I would send my software to another company.
This company will do the testing for me, they would send me a report I would fix the bug so on and so forth, and that's just not productive back and forth between these two things is too big and I think again, it's our responsibility to make sure that things work. And automated testing is just as such easy and cheap way of doing it. And that's why I say in the book that Effective. Developer is someone that effectively test is what they
do, right? And I think, I mean, I don't know, I was a developer last time, sometimes developers can be very confident type of person, right? So we rewrite the code, we test a little bit during the local development. We think it works. We always think it works. Many developers probably do not enjoy writing test, for some reasons. May be apart from you neither sometimes because it's very
close to their workflow. So, how would you actually invite most of the developers to actually have more? Passion or energy to start writing that if they have the perception that I'm super confident, my code. I don't want to spend more time to write automated tests and things like that. Yeah, that's a very interesting question. So the first part was you mentioned about developers being confident, right?
And what I always say is you never know, you don't know how many times a day you make a mistake. So you program something wrong and you only know this once you have tests because if you don't have tests everything that your code looks perfect, right? To just believe it works. As soon as you have test then you see, you know, oh my God, I'm ready. He might test 50 times a day. And then you quickly realize how much you need them because we
break stuff all the time. Now, how to convince developers to write text. That's a very deep question. I think there are many angles to this one is to show to them and to have them practice so that they don't see writing tests as something that is a burden. They just get used to it, they get proficient with writing tests, not only with the testing Frameworks, but on writing code, that is easy to be tested on. You look at a program and the test case is emerging your head
and all you need to do this too. Them in your programming language. So there's this aspect of just practicing and getting better at test itself. The second one if you are in a very large and complex Software System, things are too complex, right? And then for you to have this pleasure back in writing tests, you need to make sure it's easy to write tests. And this means as a company, as a team, you have to invest in a small infrastructure. That facilitates the act of
writing tests. I was discussing this yesterday, actually not here at the company, but in another University, It was yesterday and I was saying if you're working on very complex Software System to test something, maybe you need to know to instantiate 10 entities or have data in 30 different tables in your database. And that's just because your business is complex and you have to have some something in there that makes this easy for you that in a bunch of couple lines of code.
You can set up the scenario you want to test and then you can do the test so those are the two perspectives. I see, one is again getting good at testing itself and the second one is having the proper plan. Form so that you can write it, this totally makes sense for me. So, from my point of view also sometimes we also need to maybe introduce back like, what you mentioned? The beginning of the pride, right? So how can you tell that your code works now and also in the future?
There's no other way to do it other than having some automated tests that can actually prove that your work is always passed. Write that in terms of test cases, and I think many developers also think, probably writing, more code means like doubling their effort, right? So I think this must be changed as well. In terms of perspective. Active so that you don't finish writing the code. If you don't have the test, maybe just some perspective from me. Let's move to the thing that you
mentioned in the book, right? So you mention about effective and there's another key word that actually you mentioned in the book, which is systematic. So, I find these two things are very interesting, the way you explained in the book. Maybe, if you can explain here as well, what would be your advice for people to be a more effective software tester and also to be systematic in coming up with those tests?
Yeah, indeed. So they make a very strong point that Maybe you can be a little bit more systematic. When we approach testing, we wrote a paper a couple of years ago. And in this paper we asked developers to write tests. So we gave them a program and write tests and we observe them and they were thinking aloud, so that we could see how they were coming up with test cases, and we observed a couple of things. Maybe the most important one for this conversation is that developers were never
systematic. They were always, you know, just following their hearts. They would look at the code is this shows like the next thing I want to Test. They would go and they would write the test and then they would repeat, what is the next thing I want to test? So, they would just follow their hearts and follow. Clear experience, and feelings are very important when it comes to testing. But it also opens up space for mistakes, maybe you're not in a good day.
And then you forget something second. If you're always using all your brain power to come up with test cases, even the basic ones, you're not saving energy to think of complex cases. So when you're more systematic in terms of testing, that means No, just follow some sort of cake recipe. That helps you to come up with a bunch of test cases that are quite easy to see.
You can put them on a checklist, basically, and then it is your brain power to focus on test cases that really requires human Mark, people thinking about it. And the fun part is that are lots of techniques like this. If you look at academic books on testing, even the books from the 80s, they are go back. Then was to come up with a recipe that teaches you how to write perfect tests and what are lots of good ideas there. And you can think of, I'm not going to give it 30 hero.
Of course, but you can think of basic things like, for example, if your method receives a list as an input, there are a bunch of interesting cases that always make sense. So, for example, what happens if the list is empty, what happens if the list is new? But if there is no linear programming language, what happens if the list has just one element, right? Because sometimes your algorithm changes if there are multiple elements were just 14, there are
a couple of days. It's stuff that you don't even have to think and you can write those tests and they are usually good enough to get you to a very high coverage already and then leave Space for you to then focus on Boundary testing, you know, Corner cases in this type of stuff. And being systematic, is something that you seen other
engineering Fields, right? And I think I said in my book, there's this very famous book called The checklist, Manifesto, and there's a story they're backed up by research. That shows that medical doctors, that use a checklist before some surgery, they make less mistakes that medical doctors that don't and you don't have to be ashamed of following a checklist. It's goods and I think that's the point of my When I say we can be a little bit more systematic.
Of course, we don't have to be systematic all the time, it's just too expensive. Maybe too complex to be systematic all the time, but identifying, like I'm testing a complex method here, let me be a little bit more systematic. So that's the whole idea of being systematic when it comes to testing. Yeah, I think your point is Right, sometimes we have typical use cases, just like list or maybe you'll have an API failure cases, success cases. So, all this can be made as a checklist.
I've heard about checklist, Manifesto as well. I think the author named are together. Day and I think becoming systematic in solving. This kind of a testing problem is really crucial, because in your book, you mentioned that, if you are systematic, you can assign any developers to the same problem. They will most likely come up with the same test Suite, right?
How cool is that? Because sometimes we feel that we only need a certain test, has to come up with a comprehensive amount of number of tests, but I think the systematic way is actually to have any developers working on the same problem, but they can come up with the same test Suite that I think is really, really powerful concept and I think it also becoming effective. Means that you write the right test because I think it never ends, right?
You can come up with any number of tests as you long as you can produce the test inputs, right? And I think to come up with the certain right. Amount of test is very important, which brings us to some of the techniques later on one of the techniques that is commonly. I don't know discuss in the software world is about test pyramid. Maybe let's start from their first. Do you think this is a systematic way to think about producing test and what is your
view about this testing pyramid? I think the testing pyramids. Helps you, too. Pragmatic when writing those tests because one thing is to come up with a test case, it's right. So those are the inputs. I want to give to my program and this is the expected output. The other one is writing this in code and making sure that this works nicely in your, let's say, development process in your CI and Etc. So for example, if you just write end to end tests, maybe your test Suite will cost you
too much to execute. And then at some point this becomes a bottleneck, right? So I think the testing pyramids gives this pragmatic point of view on, hey we're going to have to automate the Stats and we're going to have to maintain those tensor and we're going to run them the whole day and I like the idea of the testing pyramid very much on the idea is that the bottom you have unit tests, right? And why is it at the bottom? Because they are usually cheaper
to rights. They are cheaper to run. They tend to be more robust. They did not do really fail. They are not flaking General and then you go up the pyramid. You have integration. And then to intestine, you still have to do them, right? You have to, but you do them a little bit less, maybe focus a little bit more thin than you. That's maybe it's okay to maybe have some duplicated test series for ensure. Are you covering the same thing or not? I think it's okay to make this
sort of mistakes in the unit. That's what it integration and end-to-end tests. You want to prioritize a little bit more. And I like this idea of the pyramids. Although, you know, in the software engineering at Google book, I think from 2018, I think they bring A New Perspective that I also appreciate very much. That is forget about unit, testing and integration testing, right? Just separate your test Suite between is this one fast enough
that I can. Actually run it during a pre merge together with your merge request, or pull request. Or is it super slow? That I will have to run in a separate machine and etcetera. So separating, the death sweet infesting slow, makes a lot of sense to me because if you remember back in the 2000's, our big discussions were what is the unit test? If I'm testing two classes together is this to unit test or not right.
Then, I think in 2023, we know that this doesn't matter at all as long as it runs fast and it gives you proper feedback. That's good. So, I like this new way of seeing things and it brings way. This is space for this type of discussions, right? Because if you no one can argue that a slow test is better than a fastest, you can argue that it may be like integration test more than unit tests, but no one can say a slow test is better than a fastest, right? So I like this new way of phrasing.
This yeah, let's go to the point, you mentioned just now because there are some discussions and debates about people preferring more about integration tests rather than unit as something that doing unit test because you probably most of the things they feel is less valuable. So what is your view about this? I know it's probably a little bit more hot topics to discuss, but what is your view on this? I think I have a pretty clear opinion on this, right?
And I think walking can be dangerous for sure because if you can walk too much then at the end you're not testing anything so you want to test as much as possible real Behavior or behavior that will look like in production. But at the same time you don't want to be slowed down by thanks that you don't control so much. For example, let's say you're writing a piece of code that makes a call to web service that
is developed by another thing. And suddenly You to really run this test, you need that web service available to you, right? And this becomes very quickly, a pain. So in this case is smoking, makes a lot of sense and the web service you don't control so much. That's given an example of something, you control a database. Should I mock my database or not? To be honest.
I believe in my opinion is that you should not walk your database because today you can make tests with database so fast that you can actually run them during your pre merge time and you get feedback. They're very fast. So why would you walk something that is not really preventing you from, right? In the test, in an easy way into running too fast. So I think that's the trade-off that needs to happen. I think it's not about writing integration tests or unit tests, but it's about writing.
Lots of tests are fasting the end and that you can have full control over and those are the things you should mock, right? Things that you don't have control over and then of course if go back to the testing pyramid, let's say you're mocking this web service so you can write lots of unit tests you know Superfast tests that just mark this web service but you can still have one or two or three integration tests that are a bit more expensive that make real calls.
To this web service. So you see the things work, when you put all the components together. But you know, something that I always say is you should not write the integration test that could have been a unit test, right? You don't need an integration test exercise. An if statement in your code, just write a unit test for it, leaving integration tests for what it really pays off. That is to find the integration
works. I really love your pragmatic approach to answering this question so I fully agree as well in terms of control, right? So first, think about the you control that and especially is not just the external service or infrastructure related time. Is also, probably one thing that you want to consider, right time is something you cannot control by itself, but you can actually create like a fixed system or Market sometimes. So I really love the way you
explained about this. So let's move on to maybe some of the other techniques, right? We have talked about unit testing. As integration tests. And to end tests in your book, you actually mentioned a couple of testing techniques. So let's start with the first one. Which is called the specification based testing and it's actually very interesting the way you start by expanding this. I would assume like some people would start with unit tests but you start with specification based testing.
So maybe tell us more what is specification, based testing why you prioritize that as the first who? Yeah. So what is testing, right? So best thing is about trying to find bugs and how do you do this? Because you compare what Expected program to do with what the program does, right? And for you to do this, you need to know what the program should do. Where is this information in the requirements? Then I'm not saying like a Word document that contains the requirements or uml, it doesn't
matter. The requirements can be in your mind's at some pointers is notion of what the program should do. And specification based techniques are the ones that help you look at the requirements and identify interesting test cases. This is not new again this dates from the 80s and we became very good at coming up. With techniques. And basically what I want to show you, my book is to look at the requirements, it's quite easy to see, you know, those are
the inputs of the program. This is sort of what the program is to do in the output. Those are the different paths that the program may take, so on and so forth, and you can look to all of this and then get inspiration to write your tests. And the one technique of showing my book is actually a very basic one. That basically says, look at the inputs of your program for the message or testing or the component, whatever, or the inputs separate them, one by
one. Your function receives three inputs and integer is string that represents whatever and the list. Look at them separately and explore their domain. So let's say this is string. What are the possible strings that can come here? Are there any special strings that would make the program to do something different? You do this / inputs, then when you do the separately because it's just much easier for our brains to process small things, right?
So you do each one of them separately, then you try to look at all of them together you look for other possible Corner cases and so that might be explicit on the documentation or the requirements and then and only then you come up with test cases and that is sort of the idea of specification based testing that you start your tests from what the program should do, and not from the implementation. And I actually like this very much, because as a developer, right, the person writing the
codes that I'm about to test. That gives me the opportunity to disconnect from implementation. Really focus. Hey, this is the input that I'm going to pass the program. This is the expected output. So this is my suggestion. If you really want to be my little bit more systematic when it comes to testing, His first step is start, dragging tests, or creating test cases from the specs from the requirements.
So when we talk about specification based testing, I think people always, I mean, not always often associate with bdd right Behavior driven design and sometimes it's like gherkin type of language and things like that. What's your view is of this, do you always have to come up with bdd type of specification test? Or is there some other types of tests that we can also use? Yeah very good question. And maybe that's a difference between my book and No other
books. Usually when people talk about specification based testing, they are focusing on a very coarse grain feature from the point of view of user. Why should I test in my book? I show that you can do this at method level, for example or a class level because as a developer, that sort of the unit's, you're always handling
with, right? And I think, to me, really makes more sense when you're looking at the big picture and looking at the whole functionality from really, the point of view of the final user and the specification based techniques you can apply in both now, Should you write tests in a bdd style? I think that's really a matter of taste. I am personally not a bdd fan, so I don't like tests using vdd style. I don't use cucumber. That's just not my thing. But I see where this comes from,
right. And I don't think, you know, in practice, this is two different. It's really telling you to focus on the specs, on the behavior of the program, right? And coming up with the best cases tooling is a matter of personal taste in the end, right? As long as you're looking at the behavior of the program, what you expect from it? I think this is a great Step towards a good testing. Right. You also mentioned just now in your explanation about Corner
cases, right? Sometimes when you have requirements, most often, it's just the normal case, right? I expect the program to work like this. So first of all like who will come up with this corner cases and how would you actually invite people to come up with more creative Corner cases? And I know you mentioned about boundary testing. Is it also one way to actually generate Corner cases. So tell us more about this. Yes indeed. So Corner cases are the cool.
Sprite and Empirical research actually shows that bugs, love boundaries because we are very good at implementing happy paths, the bugs, then they start to cluster on things that were not so good at that. Is to handle Corner cases, right? And coming up with Corner cases, is very hard because we develop complex software systems, but one way to get started is to observe your program. You look at the inputs and how this inputs change the outputs and Boundary testing is a perfect technique.
For this. What this boundary mean? So imagine you have a very simple program, you know that has an if if x greater than 5 do something else. So this if divides the execution of the program so if the input is 4 4 is not greater than 5. So the program will do something. Then next 155 is not greater than 5, so the program will do the same thing. Now 66 is greater than 5 another program, there's something else. There was a small change in the input from 5 to 6, right?
So there is more changing the integer value but the program responded to something. Completely different. So this is a boundary and this is precisely where you can write a test because we love to put bugs there, right? And as a developer that makes sense because it's very easy to confuse, you know, a greater than with a greater than or equals to. And the ideal of boundary testing is look at your program and look at the inputs and how
the inputs affect the outputs. And look for those moments where a small change in the input changes the output. There's a paper from 1994, the reciting my book that explains this in a more mathematical way. So this was really written. Buy a computer scientist and that shows that if you write tests like this, you are more likely to reveal a bug. And I think this is something very easy to change in the behavior of the developer, right?
And it's like a very quick win. That will give you better test, because as a developer, you look in, if you write a test for the true branch and for the false Branch. But then we usually pick random numbers that exercise. The true branch in that exercise, the false Branch instead of picking random numbers, pick the numbers that are close to the boundary. These two tests will be way stronger than the other two tests. That's Of exercise the programming the same way but
very far from the boundary. It's a very easy change and a funny enough. I get emails from people reading my book and this part of the book is the ones that people like the most. And people say, oh my God, this is so simple and that really made my test so much better because it's also easy. So think of boundary testing for
sure. And of course in this example is very easy to find the boundaries but more complex programs you're going to have to spend some time looking for the boundaries, identifying the boundaries, but that's just part of the exercise that we have to do. Now thanks for sharing this. This technique this tip. So I really loved the way you phrase it right box, love the boundaries. So before people here, maybe that's also a lesson for all of
us, right? So when you think of test cases, now start to think in terms of boundary first then you can pick maybe some random inputs later on after you have covered the boundaries. So let's move on to the next technique, which you categorize, as structural testing. So maybe tell us more. What is structural, testing, how can we use it to become an effective software tester? So structural Things sort of the chemical name for using Code
coverage in practice. So the idea of structural testing is let's use the structure of the program as a source of inspiration to write tests. So before we were looking at the requirements with specification based testing, right? Maybe texts. Now we're looking at the implementation and trying to come up with ideas and how do you do this while we look at them, if statements and then you think, well, I want to write a test that actually exercises that if statement, and maybe you
can do this line by line. A line, right? And then once you cover all the lines, you're done with your structural testing because you tested the structure in the way you want and your coverage tools, give you this
information, right? You can see line coverage for example, or you can be a little bit more thorough and say, well I want to cover all the branches and whenever there's an if I want to make sure that there's a test for the true branch in for the false Branch, or you can go deeper an if statement can have multiple conditions and I want to exercise every condition as
true and as false, right? So you can keep going Crazy about this, but this is sort of structural testing and I think the main point of my book is that there's this hate in Industry about called coverage, right there are many reasons for this, you know, one, you can cheat the coverage number, right? It's very easy to write a test that covers a lot of stuff, but the test is not so good if you're using coverage as a target number. So let's say your company says 90%.
Otherwise, we don't get from merge request. Maybe that forces you to write useless test, just so you get to 90%. Also, if you get to a hundred percent coverage, Each that also doesn't mean your code is perfect, right? And it's quite expensive to get to 100% College. This is why people usually hate it. But I think people usually hate cold coverage because they are focusing more on the number rather than on the output the information that coverage brings
to you, right? And in my book, I showed it coverage should compliment specification testing because, you know, you write the test based on the specification and then you do, maybe boundary testing. And then the question is, am I done? Well, you can triangulate this and you can look at coverage, right? And then you see, Did I cover everything and maybe you forgot to cover something. And then the question is, why did I forget was it?
Because I forgot was it, because there was a mismatch between the requirements and the implementation you reflect about it. You either write a test or you don't try to test and it's also fine, right? But it helped you to reflect a multi, my done writing tests. And if you use coverage with this purpose, notice that we are not talking about numbers anymore. Furthermore, talking like the test that I wrote, are they good
or not? And coverage is giving me insights about it. And I think that story You should use coverage but then the 1 million dollar question then is also, okay. Okay, coverage is cool. I use structural coverage that will maybe help me increase my coverage because I'm going to look at an obstruction that I didn't test. I'm going to test it now, is there a correlation between High coverage numbers and the effectiveness of the test Suite? So that's sweet.
That has a very high coverage. Is this one more likely to find bugs? If I have a bug in my code? And this is a perfect question for academics to answer, right? And then you see lots of Papers written in my Besides papers from the 1990s to 2010 or something. And a lot of those papers show correlation between coverage and effectiveness of the test suite and that makes sense. Right? Because the more code your test Suite covers, the more likely it will find a bug.
If you introduce a bug, of course, different paper, show different levels of correlation, some of them are strong, correlations some others anymore, weak correlation, but the correlation does exist. And I think the lesson that I get from these papers are. 100 coverage, may not mean a lot. Because once you're there, you're desert very good. But that doesn't mean it's perfect. And then when you're at 100% coverage coverage doesn't give you more useful information because it's all covered.
So if you're there that stream, then coverage is not so good anymore for you. And doesn't tell you much with each other on the other side. That is you have very low coverage. That's a 10% then that means a lot, right? That means your test Suite is still, may be poor. Maybe there's something that you can improve their so, to summarize it in coverage can be used as a way to complement your tests to help. You see if your test Suite is good enough for now.
In coverage, helps you to identify poorly tested areas of the code base and hey, this is not covered. So maybe I should write a bunch of tests for. They don't have to go to a hundred percent coverage, but I have to add some text for it. Well, I really love another pragmatic advice on this, right? Because people always debate about code coverage. I know some companies also who put code coverage as the so-called the build gate, right?
So if you don't pass a certain threshold, will fail your build. So I think you remind us that code coverage should be a complimentary thing. It should not be the main. Main thing, right? And I really loved the way you mentioned that 100% doesn't mean that we will not find any bugs,
right? Although there's a correlation of course, but a low coverage actually means that there are room for improvement and I also like one particular statement that you mentioned, how should we use a criteria in terms of covering more line coverage you say in your book that all code should be covered until proven otherwise, right? So I think that's also another good guideline I find sometimes, we can leave some coverage behind but actually you should have a good. Listen for that, right?
So I think that's a very good thing. Another technique that sometimes is used for, this is actually called mutation testing. So this is also in the structural test kind of territory. So tell us more for people who may not have heard about this term mutation testing what it is and how does it help to provide effective software testing as well? Sure. So, how do you know if your test Suite is good or not? You can look at coverage for
sure, right? But maybe you can get to a lot of coverage, but maybe your assertion. As our poor, right, and then there's a bug but your assertions are not catching that bug but you're covering the code. So your coverage is very high. Mutation testing helps you to identify this Gap and what is the idea, the idealist I'm going to create mutants of my codes. So imagine like, I just get your production code and I own purpose insert, the bug. So I change a greater than to a
smaller them, right? So let's say I do this, if I run the tests, your tests, you must have a test. That is failing because it just introduced a bug. If that happens, that means Okay, your test can actually q that mutants. So your tests are doing some good stuff but if I can mutate your code so they introduced a bug on purpose in your tests are still green. Hey, I just found the case a possible bug that someone can introduce that, you'll never know with your test Suite.
So that is the idea of mutation testing. And what mutation testing tools do for you is to automate this process. So they change your code, they run your tests and they see for test. Catch the bug and they repeat this in a structured way and they give a beautiful report in the the end solely for in the Java world. For example, like I am, you have Pi tests which is an open source tool that does that for you. And I love the idea of mutation testing.
It just makes a lot of sense to understand if your tests are good or not, I think in practice a lot of companies are not there yet to benefit from mutation testing. You start to benefit from mutation testing when you have a lot of tests and have very good test. Suites if your test Suite is very poor. Well it doesn't get it doesn't kill any Mutant. So why would you run mutants in first place? Right. Comfort is good enough for you
at that moment. So I should like a lot of companies are not there yet in terms of materiality to use mutant but if you are so if you have a beautiful day sweetened put mutants in your pipeline tools are evolving. So by test is a very nice tool that are ways for you to reduce the time it takes because it's an expensive process, right? We have to mutate your code and run. All your test suite and we do this a million times or the tool does this a million times. So those tools are getting
better and better you know. So that you just run the tests that are really relevant for the mutant and blah blah blah. There's a beautiful paper that I cite in my book called, I think it's mutation testing at Google. It was published in an academic conference but in the industry track and that paper sort of describes the experience of Google in trying to put mutation testing on scale mutation testing. By the way, is also it dates from the 70s, right?
So the ideal is super old but I think now that we have very good Hardware to make it work and the tools are getting better and better. I think now that industry is adopting it more, which is really cool. Loved your analogy using mutants to explain how this mutation testing works. So, I think it's true that. I think if you have a very low coverage or kind of like the poor test Suite, you will not be able to kill the mutant so to
speak, right? Because your test will always pass because you don't cover enough. Another technique that you mention in your book is called property testing. How does this differ in what is property testing? Yeah, so property based testing is the way of writing a test that is different from the way
we write normal tests, right? So usually when we write, At a normal test, which I call example based test in the book, you know, in our minds, what sort of Branch you an exercise in the cold or test case when I create based on requirements and what you do is you think of a concrete inputs that room run the program will exercise the program in their way you want. So in the example that I gave X greater than 5, it can be Any number greater than 5 5 6 7 8, right?
And you pick just one because you're pregnant again because you cannot do more than that, right? So maybe we can do two or three or four then you cannot do 50 by. Right hand, it's just too expensive. So this is example based testing because you're testing, by example, you pick one example in property, based testing what you do. Is you try to describe a property of the program and you let the to come up with the inputs for you.
So, let's say in this program, if for any number that is greater than 5, you know, that the output of the program, should always be a positive number. So, inputs, X greater than 5, then the program always prints positive numbers. You can write the property and then you say, create any number that is greater than 5. And just assert, it's that the number that comes out of it is positive. And then the two creates a lot
of inputs for you. So, for example, in the Java world, you have J quick, when you run the test, Jake will think of 1000 inputs for your function and it will send 1,000 input for your function. So in a way, you're exploring way more, the domain of that input right? Because you're not trying one example, you're trying multiple.
So the ideal is very cool. It is, of course, much harder to write a test because you have to stop thinking of specific functionality when a test but more like what are the properties of my program? That I want to exercise. But once you can do this, then it's very powerful because you're just testing a lot. So maybe another example so that you can see the usefulness of it imaginary just implementing some data structure. So let's say you're implementing
a set yourself, right? So you cannot have repeated elements on the set and you can write a property based testing that just transfer to insert random elements in the set and you make sure that you're having repeated elements and it doesn't matter. The order that you insert the elements in the end, the property is The set must only have unique elements in there and erectus like this.
And then suddenly you're testing dozens and hundreds of combinations of inserts of repeated stuff, new stuff repeated stuff. Something that you would never be able to do by hens. So that's how powerful those things can be. Now we need Distributing Information Systems, which is what a lot of us do property. Based testing is I think you have less opportunities to write this to use property based testing.
So I still feel like example based testing is something you should do as the to Approach but considered property based testing especially for situations like this where sometimes you just feel unsure. This one example is enough and it more and then property based testing can help. Thanks for that. Toro, explanation about this property, based testing. I personally haven't done, mutation testing and property testing myself. So I think it will be cool to introduce that.
Sometimes in my test Suites I guess. So for people who also have an experienced it, I think you can check some of the papers that Mauricio has mentioned. Also, some more materials from his Book. Make sure to check out as well talking about testing. I think it will be a Miss if we don't discuss about test-driven development or tdd. What's your view about this workflow? Are you a proponent or do you always do tdd? So maybe tell us more about
that. That's a tough question hearing because usually you know you have some very passionate people about disturbing developments, right? And sometimes I make fun with my friends that on my Twitter bubble, sometimes I just post something that says that he's not that perfect. And then yeah a lot of backlash in, right? And I'm a big fan of tdd.
I even wrote a book about TD until 2012, it's in Brazilian Portuguese. So maybe not accessible for everyone that's listening here and that was because I was doing a lot of DVD myself and I really felt I was just being a better developer doing tdd and this was 2012 and when I wrote the book so I was doing tdd for a couple of years now, 2023.
I think I do 3D wave less and I feel that just because I found a way to get the benefits of tdd without having to do tdd and what are these Benefits. Right. What am I talking about? If you like the big benefit for me, about TD is the ability is that we work on very small steps, right? And we make very small steady progress towards the bigger feature before. Knowing TD, someone would give me a feature to implement and I would be all over the place, trying to write the full
algorithm in one goal. And that would just complicate my life. I would code for an entire day. A lot of frustration because you know you bump into barriers, you break stuff that was already working. And so on and so forth. And then after, you know, a few days of work or whatever, then writing test is, you know, I just want to get this feature done. I don't want to do this anymore, right? And we tdd it was like, oh, and I cannot do tdd if I work on
such a big chunk of code. So it forced me to see programming as an act of writing small things, and combining these small things. One by one to give bigger Behavior. I think that's sort of the big difference between TV and known tdd. And if you can incorporate these three development practice I think it's okay if you don't start with the test. So today, for example, in a lot of cases, I actually start with the production code but my coding sessions are very small.
I write a little bit of production code and then a little bit of testing the experiments sometimes I delete the code and I start again and do it small tests. I think if you find your way to work on small and steady steps, I think you got the benefit from DVD and this is actually what's more recent research shows about test-driven development. So if you look at the whole body of knowledge on DVD, you see papers and What have you built of these papers in a nutshell
qualitatively. If they s developers that are doing tdd, do you like to delete? The answer will be. Yes, I am. Actually one of the authors of these papers, I wrote a paper that did qualitative research on this and that was sort of the feedback we like tdd quantitatively. If you compare the quality of the test Suites in comparison to people, not doing tdd. The differences are negligible very small. So 3D by itself doesn't do
magic. So more recent research actually shows that the benefits are on those small steps. And not because you're writing the test before that means side, do I recommend you to do tdd definitely? Yes. Especially if you've never done it because if you never done it, odds are you're not used to work on doing small things. We're just used to work on big things. So TD is the best teacher, you can have when it comes to explain to you how to program in small bits. So, do tdd, do a lot of it.
Once you internalize the ideas, then it's okay. Then you don't have to do it anymore. Wow. It's like a safety will write so too. Figure out you start using it and once you kind of like Master it and like a bicycle, right? You can take it off and you can choose when to use it and when not to use it, and I like that, you mention about this Empirical research, when I read it in your book, I was intrigued. This research doesn't actually show that TD produces much way
better test versus the non TDS. So I think that's really cool to know about this and I think I'll make sure to put it in the show notes as well, for people who are interested and also have passion about Didi. It's not saying that tdd is not. Good, right? But I think it's the mindset of working with small things. That is very important. So we have a few minutes left before we go to the technical
leadership. Wisdom, I have one question, maybe tips from you for people to be able to become a better tester in terms of their quality or maintainability. Is there any few tips that you can just give us here so that we can improve? That's a very good question because, you know, if you really write lots of tests you're going to have lots of source code in, they're going to have to maintain this as well. Not only Production codes, right? So I think all the love that you
put in your production code. You also have to putting a passcode and to me in the test code if you have to focus on one thing because in my book I mentioned lots of ideas you know good practices that you can have to make a cool beautiful and blah blah blah. But if I can just talk about one that is you have to make sure that the part of your test where you create the data, right? That you create the inputs that you're going to pass it to the method, or to the class, you
want to test. That part has to be crystal clear. Because if you're working in a very complex information, Mmm, odds are that the complexity of the test will be in that part. So that part needs to be very easy to read. It has to be very easy to evolve because your entities in your domain are evolving all the
time. So it has to be easy to make an evolution in the production code, without breaking the test code, and it has to be easy for you to come up with complex data inputs without having to spend 50 100 lines of codes. So you have to have lots of utility methods or whatever you want to call it that help you build data, right? Test data, Builders the famous. For patterns like this, you need to invest on them. So, that is my tip number one, and that's what you need to focus.
Input data has to be Crystal, Clear? Yeah, I also mentioned reading about this in the I think the growing's object-oriented software Guided by test. Right, I think this is also one key technique that is mentioned in the book. So I really love it because I have seen in my experience before seeing, you know, lines of test code which are probably too complicated to understand and especially the part where you produce this test data,
right? Initially, We will see a lot of gibberish, probably like, why are these logic here? But it's actually to produce test data and I think it's a really, really important for us here to improve. And if I can tell a personal story Henry, if I can tell a personal story, you mentioned this growing object-oriented
Guided by test, right? Altered by Steve Freeman, Annette pranks and I have a very funny story about it because in 2010 there was this Workshop in Terris about test-driven development, the one and only on the series and I had a paper there and Treatment. The author of the book was giving a keynote and so he gave the keynote and it was then giving a presenting my paper and you, my paper will siphon him a lot, so that was like, Freeman and price.
Say this in the yearbook at some point, Steve looked at me, you know, for like, because it was a tenth time. I said his name on my presentation and then I said, I'm sorry. I just like her book, very much and we became friends, right? And when I moved to Europe, then we were closer to each other and then Steve started to teach one guest lecture every year in my course in delft.
So he always comes to death every year from my I course and that book influenced me a lot and he's, in fact, one of the forward altars of my book, right? So one of the forward there was written by Steve, so he influenced me a lot in this book is an amazing book if you have the chance to read it as well. Well thanks for sharing about this personal story. I find it really, really
interesting. So I think that's also One Good Reason. How do you get connected to a more famous gas of famous authors. So I think thanks for sharing that. So, as I mentioned, I have one last question that normally, I ask for all. Guess which is called the tree technical leadership, is them. You can think of it just like advice that you want to give to the listeners here, maybe from your experience, from the expertise. What would be some of your wisdom that you can share?
Here, Maurice, you three. Let's do it. So number one focused on these episodes, right? So you should Master testing and measuring testing means not only mastering the tools like ju and it and mockito and whatever to you use. But also mastering creating good test cases. And once It's you really become proficient with it. It just becomes way easier and way cheaper. So you start having less reasons not to do it because it just more natural. How do you get there by practicing?
Right, it's going to hurt at the beginning but the more you do it, the easier it will get. So practice, testing advice number two, for software engineers in general is you should never stop learning, right? I think it's very easy for us, you know, a lot of us, go through education and then we just join a company and you start working as a developer and then send it. We forget that we need to keep updating ourselves.
I once heard from a developer, you know, hey, I'm here working for two years in a row and I started to read this book. I recommend the book to this person, and he said, I started to read this book in a few. So nice to learn something new, and I don't know why I wasn't reading books before. I guess I was just too busy working, and that's life. That's life, we're all adults, we have responsibilities, it's very hard. So, I think you have to find
time during her work hours. To be honest, your employee needs to be on this to upskill yourself. So read books. Are you interested in? I don't know, microservices with the book so just keep studying because there's so much new stuff going on and so many best practices and something that I feel among developers is that a lot of them when they go, you know, in a big company and you're going to have this big discussions to decide what architecture to take, should we use practice a versus B?
Whatever people have intuition, but they have very little evidence to explain to people and if a reading books you have clear ways to explain why you want to go for this or not. That's so there are so many benefits in studying and putting studying as part of your daily job. And so work on this, make sure you study a lot in advice. Number three will be. You just learned a lot from my advice number two, because you're studying. Now, it's time to share your knowledge.
So make sure you also write about what you learn or give talks at conferences, right? And this is very good for you because it exposes you to other people, it forces you to formalize what's in your head. And you have to put in words and I find that putting in words is the best way for you to really understand how much understand about something. And maybe that's because, you know, I have this academic background. So writing really helps me to see how much I understand about something.
It's good for you, right? Because you're just getting better as a person as a developer. And it's also good for others because I'm pretty sure you have cool stuff to share and that will be others that are willing to listen to you. So please share knowledge. It's as important as learning new knowledge. So there you go, the tree. Is Henry. Well, really beautiful. Thanks for sharing that after I hear about our discussion about testing.
Now, I think sharing is like a form of testing itself, right? That you actually learn from what you study, right from reading the books and things like that, may be the best test case that you write for those learning is actually to share it either, you know, writing a blog or whatever YouTube video and things like that. So I'm having this testing mindset these days after hearing what you explained earlier. So thank you so much for covering this testing.
I think it's really part of Out of the skill set that developers should Master more, right? Like, what you mentioned in your first wisdom, developers need to practice more and be good at testing because I think that really is the key to produce a quality software. So for people who love to connect with you, maybe to talk more about testing, is there a place where they can reach you on mine? I'm always on Twitter. So it's at now, this your
nation. So my name and my surname, I also have a free newsletter related to my book so if you go to the website effective Dash software Dash testing, You can subscribe to the newsletter. Those are the best ways to get in contact with me for sure. LinkedIn, you can also add me on LinkedIn now. Recent issue find me, feel free to drop me a message there as well. Cool, so I'll make sure to put the newsletter as well, so that people can learn maybe every week, right?
How do you improve your testing experience. So, thank you so much for your time and ratio. I hope people enjoy learning about testing and I wish that your book is also going to be teaching people a lot more about how to produce the best quality code. Thank you so much, Henry Ford invite and I hope you like it everyone. Bye bye. Thank you for listening to this episode and for staying, right until the end if you highly enjoyed it.
I would appreciate if you share it with your friends and colleagues who you think would also benefit from listening to this episode. And if you are new to the podcast, make sure to subscribe and leave me your valuable review and feedback. It helps me a lot in order to grow this podcast better. You can also find the full show notes of this conversation on the episode page, at Tech Legion o.f website, including the full transcript interesting.
Quotes and links to the resources mention from the conversation. And lastly, make sure to subscribe to the shows mailing list on package. You know, dot f to get notified for any future episodes. Stay tuned for the next technology. No episode. And until then goodbye.