Hello, everybody, Welcome to another thrilling episode of JavaScript Jabber. I am Steve Edwards, the host with the face for radio and the voice for being a mine. But I'm still your host with me on the panel today, all the way from Tel Aviv, Israel's mister Dan Shapier.
How are you doing, Dan?
I'm doing well. How are you, Steve?
Doing good? Doing good?
It's Wednesday morning, first day back at school for kids here, so.
That's always good.
I'm still hot and toasty there in Tel Aviv as always.
Yeah, when is it not? Yeah?
Yeah, it's a little cooler here, starting to cool down a little bit, which is always a bummer, but.
It is what it is.
And our guest today coming all the way also from Israel, mister Joannie Goldberg.
How are you doing, Yanni?
Hey, hey, great, great Tom. I'm really happy to be here today. It's one degree cooler here comparing with Al Aviv. Very hot, but the good news that after two months as of being at home, kids are finally back at school yesterday or before, so.
Yeah, back to work a little yes.
And for those of you that might see Yohannis drink, he is not drinking and podcasting that is tea, not beer. I clarified that before the episode, so so not that it's not okay to drink and podcasts, but just clarifying.
I literally drank beer on stage while giving certain talks. My contention is that it only made the talks better.
Started listening up a little bit. Is that it Yeah, if you probably.
Know, is it also legit in the States? I know it's you in Europe, it's okay. But if I drink at like it's the States during the talk will be considered PC.
I've never heard anything against it. I guess it depends on how well you handle the the alcohol and well you stay on topic and have somewhat of a sense of humor.
Then you're saying that the first two okay.
In Germany, it's perfectly legit to have been at lunch during work days and legita required.
Yes, there is something to that, and it's really good beer.
Uh.
When I worked at Wix, let's just say. Let's just say that we worked hard and played hard and sometimes played hard during working hours.
Alrighty, So with that we will transition into the topic juju early topic of the day, which is JavaScript testing and what's going on in testing in the JavaScript world, so you on need to take it away.
Where do we want to start?
I'm sure you wanted to start by sharing a few words about myself and.
My Now we don't care about that. We just want to know what you have to say.
Now.
I just yeah, seriously, sorry, I'm a little off track. Tell us who you are wire famous and where people can give you money or will say that for the end, just tell us who are true things? Surprise, surprise.
I'm a developer, freelancer and a consultant doing full stack a lot of JavaScript, but more than anything else, really my love, my passion. The things that I'm trying to specialize in is that all developer workflow and specifically testing. I think that I've worked with more than forty organizations. Some of them are big ney you're probably recognize on improving their testing. So I literally said, over the shoulders of like hundreds of developers, as we try to craft
tested are both efficient and yet simple. Done a lot of mistakes, learned, a lot, tried a lot of new frameworks, and I hope to share some of these lessons here today.
Isn't the process to your basically cursor right tests?
For me.
Exactly exactly this actually we're about to cover. I would I would also cover some kind of testing AI trends, mostly the useful parts of AI. So yeah, okay, just to just to.
Finish that particular point. The scary thing is afterwards, when you make changes and then the tests break, you tell it, Cursor fix the test for me, and you're not sure whether it's fixing your test to hide the bugs or fixing your code to fix the tests, or doing both or whatever it's It can be really interesting.
That's this way, definitely. I mean, I think that one of the first things one should tune in Cursor or cloud code or whatever you're working is that you can let it work in yellow mode. But I think there is one exception any a human A human confirmation should be made anytime a test is changed. You may change my production code, I may inspect each and every nine or not, I don't know, but a test changed should supervise that that's.
My That's a good take. I think.
I can't think of any time I'm just going to say here writing some code and then committed or do whatever without looking over it and making sure it works.
Well.
Yeah, that's easy to say, and everybody starts with good intentions, but at a certain point in time, you know, especially when you're under the gun and certain things and things seem to work and you build a certain level of trust. I've known people to almost even engineers, almost get to the point of almost vibe coding their way through projects.
And I think you Only's statement that if you've got good test coverage, then ensuring that the tests are properly maintained and reviewed is a good way to add a significant layer of trust and security around your vibe coding.
Yeah, I agree with both. I think that people have various degrees of how much they supervise AI generated God, I'm just saying that the least conservative one can be is at least guard you're testing. You might do this also for the production cod but I think that letting AI also change tests has to change God. This is the highest amount of frizk one may take.
Yes, Okay, then, so what are the latest and greatest trends when it comes to tests, aside from having AI do all the coding for us?
Yeah, so I'll kind of mention some kind of super trends and are very impactful changes, and then we can dig dive deeper into each one of them, at least the ones that we found interesting. So I think there is always in testing one kind of everlasting progress in which we always we try to get our tests to be more production like without paying the price of So in the past two years we keep seeing this progress and in this in front and the front and world.
For the first time, I think like we ad React testing library ruling this world for a lot of time tests that runs in a jazz things that's for the first time it's safe to say that tests don't run in the browser, that are not visual are almost out of scope. There are many new ways and frameworks to run your tests in the browser and get the full visual experience and the confidence, so it's more like production.
But some of them run really fast, so you're not paying the price of like full full blown end to end framework on the back inside it means that you having a micro service, you can more easily now run it on your computer with all the layers and almost light production infrastructure, and again without paying a tremendous price. I think these are bold and the changes that we're seeing, and we I guess we will dig deeper into the specific tooling.
And just to clarify, there are obviously layers to tests, so there I don't know if you should refer to them as higher or lower layers. It's just different positions in the testing cycle. So you've got obviously stuff like unit tests, which should be very relatively small, relatively straightforward, and certainly very very fast, and they usually don't need special wiring, although you know a certain degree or a certain amount of mocking is often involved. Then you've got
integration tests, and you've got into end tests. It seemed to me that when you were talking about the improvements in our ability to do effective, efficient and performance testing, you were mostly talking about integration tests and end to end tests and less about unit tests.
Correct yes and a little no. I mean, I think at first there is a trend. I'm not advocating, by the way, on anything. I'm just trying to cover what like trends that I observed. And I think another trend is seeing less unit tests and more kind of component page integration tests. But so called the testing diamond. It's a kind of an emerging concept where you should write more integration test but with the new tooling, even if you write unit tests like take a framework that we
will cover with that. It's called VITAS browser mode. Even your unit tests run in the browser. Now think about it. Not all the unit tests in the world are pure. You might test the models that actually doing some stuff with the web APIs. So instead of running it in no JS with polyphils and trust and trust the similarity, it runs your test in a real browser and Chromium for example. So even your unit test gains more kind of production like experience. Does this make sense?
Yes, it does, although it's I need to.
Think about what I think about it. It's kind of a recursive statement because really, when I'm thinking about unit tests, I'm really trying in most cases to be as stateless as possible and consequently not be dependent on having access to a browser dome, which is kind of like persistent by the definition.
And so I'm.
Not sure that this is where I want my unit tests to go necessarily, but I certainly appreciate the fact that we don't need to mock browser a p I S domin p I S, which is always let's say, less than ideal. But why do you think that is what has changed that has made this What has changed that has made this possible?
Now? That was not available before.
Yeah, first forced margining that I fully agree. The real value is in is in integration tests and about page testing or component tests, also in end to end. In unit test, it's indeed less significant. So it's mostly if if we if we double double click into the front end word for a start. I think now there is a key up from decisions the team should make. There is kind of a triangle decision between three different frameworks for testing components and pages, and one of them is
relatively One of them is is new. It's for the past two years, and it makes running visual test in the browser almost as fast as unit test. So there was a lot of people trying to avoid that kind of visual experience because of the hassle involved. But with some of the new frameworks like vitest browser mode and the new storybook model, it becomes easier faster, so you have less reasons not to just hey, let's test it
like where the user lives. And maybe it's worth at some point explaining what is vitest browser mode, how does it stack with playwright go for it?
Now?
Is that the same as v test UI this browser mode?
Ah, not exactly, because you can you can also have vites I with unit tests and browser mode is a new sub frameworks sort VI test and to zoom out for a second, I think that we have now three measure and good options in the front and word that
one must make a decision up front. The first one is Playwright, the good all known Playwright, very reputable, very reputable frameworks that is being very popular, which allows you to test your system as is like you can spin up a whole page and you get a real page in a browser, but you get the all the you get the entire you get d their application with the router and the state and a lot of things beyond the page level or the component what you want to test.
So the performance is like a little of concerning Playwright even when you mock the back.
Correct me if I'm wrong. Playwright is essentially just a headless Chrome with api is to automate various tasks. That's more less it right.
Yes, and maybe a little more because it can run your test in other browsers as well, like run the test once and get it running in Chrome or Safari or anything.
Yeah, because they basically later on added support for the same APIs as I.
Recall exactly they imported kind of a CDP protocol for other browsers exactly. But on top of this, they also have a very nice and stable API to make the experience of interacting with pages less flaky. So for example, if you're trying to assert on something, they will take care to retry in wait and until that condition is met on the page. So the overall testing experience is really great, and.
It replaced all the frameworks like Cyprus and Selenium.
Right exactly exactly. It is.
Okay, So how is v test browser mode different from Playwright.
Yeah, so first worldt mentioning mentioning is that the viittest browser mode is using Playwright under the hood, so you get a battle tested engine that you can trust. But what they try to do Playwright is a little bit disruptive to front and testing. So if you take traditional front and testing with people used to work with things like React Testing Library, the syntax of React Testing library, the concept of React Testing library supporting from there to
Playwright was a little disruptive. Vitus Browser mode allows you to take your existing test or learn nothing new, use the same syntax, same concept, only instead of running it in nogs like React Testing Library, you run it in real browser, which obviously allows you to see. You probably know that that really nasty experience when you with React testing library and the test failed. What you see on the screen is three hundred lines of XML of HTML in a CLI in which you try to understand in
this my component, how does it look like? What's wrong here? I mean nobody wants to travel shoot test in HTML over CLI. With this browser mode, you can see them in a browser and benefits from all of the Chrome Developer tool bar perks. So that's that's the premise. That's the obvious upside. They also make it like you know, vitt stands for fast in French, with the word wit is fast in French, and they they stand up to their reputation like I can I already use it in production.
It's amazing. I can like test then entire page, some interactions, sometimes specific scenario in two hundred milliseconds. It's like blazing fast. You click on a button and you don't have enough chance to see what happened on the screen. So the performance is really awesome. With that, there are some cons
that should be mentioned as well. It's very new, it's still tagged as experimental for like two years or so, and so's it's it didn't yet stand the battle of time like Playwright, And there are many other other decisions that were made that are that might appeal to some teams and not to others. But it's definitely a bold opportunity I think now in twenty twenty five for testing and I could not finish that. I'm sorry. This is
kind of a mini mini TED talk. But there is also Storybook, and Storybook now change their model and now they're using Vita's browser mode under the hood, which use Playwright. So you have three options. You know that Russian doll which is called I believe Matroyshka Babushka or matroys Career.
It's kind of a nested doll. So what we have now the in the testing space, it's a troche car, so you have to play right or VI test brother mos that you just play right, or storybooks that use you got idea and you have to make you have to make the choice.
So if you're starting a new front end project today, and let's say that you're using one of the two leading options, I don't know how familiar you are with them. Either React, which is approximately fifty percent of the market or view which is approximately twenty to thirty percent of the market, and you need to do and you want to do testing because we started with testing for the front end.
What would you use?
Yeah, that's that's first. Very I think that the choice is really hard today. I had to choose for a customer like two weeks ago, and the decision is really hard. So from one hand, Eileen Tower being conservative and say, hey, let's use play right. It works. It's like the most simple, it's like the last layer. It's simple, it's proven, let's just use it again.
Playwright in this case is for mostly for the end to end tests.
Right, or you can use it also for components, for for component or page testing. It it has great tooling to allow you to mock your bacond and I think I think that for the majority of the testing front and team want to mock the beacon. But yeah, you can also run in.
Well, let's put it this way, if you're not mocking your back end, then you're by definition dependent. It's almost it kind of almost means that by definition your tests are going to be flaky at least some of the time. If you do involve if you do involve the back so so yes, if you want to go for a operations of concerns, you want to mock external services reduce the amount of dependencies that you have on external services and that includes your own back end.
Totally agree, it's slower, it's more flaky. But my highest concern with end to end is actually that it's really hard to simulate things beyond heavy passes. So think about it. You want to test now that the user is disabled, it's a premium user. You get twenty records, all of the things that are beyond the simplest response. With real back end, it might be very hard to tune the
backet into each and every scenarios that you want. However, when you mock it, you can simply just say, hey, I want this response, I want that response.
Yeah.
On the other hand, when you're mocking it, you're basically inventing the responses, so it's not necessarily how the system behaves. It's not really end to end if it's mocking.
I mean when you normally like spin up like a maybe this is what you're talking about with mocking, but basically create a test database or a copy of your database, spin one upset it. You know, you got your data in it and then run your tests. Are actually using a database. It's not your Pride database obviously, but at least you have a clone and a real database that you're interacting with, because I know, speaking personal experience, a lot of times that's that's valuable testing.
Yeah, yes, but I think that first before answering that I'd like to repeat to a dance. And I think it's like a key point here. If you're doing naive mocking, Okay, I'm just walking some Jason. Yeah, you are now at the risk of doing things that are not exactly the same in production, and this is why you also need contract testing. And I think this is definitely a topic we want to double click into later because there are a new interesting tooling in the JavaScript space related exactly
with this problem. How do you create smart and aligned mogs? Now to this one. Yeah, in a simple case, it's just crawd applications, So you set up something in the database and you display them. But in many cases, creating the server response is much harder than this. Let's give you an example. One of my customers, is there a user entity and it might get disabled? How does it get disabled, there is some chron job running every minute
and concluding that the user is disabled. So how would you create a disabled user from a test and make the back end run a chron job and tag it. It becomes much so much harder than just returning one line. Hey, just give me a disabled user, Jason, am I being clear on this point.
No, it's It's obviously true like getting an application to do what you exactly what you want and can be challenging, and configuring a certain backend application to replicate certain scenarios is a problem. Just think about, you know, the scenario where a customer tells you that they're experiencing a certain problem, and maybe you even have traces that kind of show the problem the way that the customer is experiencing it, and now you're trying to replicate that scenario in the
lab in order to debug it. We all know how challenging that can be. So I totally agree that replicating weird behaviors or edge certain edge behaviors that are legitimate and you want to test for in an actual environment that simulates production can be challenging, and it's often much easier to just play recordings of back end responses and you know, effectively mocking the back end it can be much much easier. Likewise, also mocking the front end.
You know you want to.
Test the back end, says system, then you might use some sort of I don't know, just post things out of Postman or whatever or playwright rather than actually having an actual front end that you know you create a process that clicks through. It can be much more challenging and more complicated than length fear.
So is there a point, you know, Joanni, with that example you gave with the Crown job, is there a point where you have to say, Okay, we can't test everything in an automated test and maybe we have to do something manually or do you always work under the assumption that everything in Proude can be handled with an automated test.
Yeah, so I'd say, based on my experience, as long as you stick to gray box testing. When I say gray box testing, it's a test that you don't test the entire system. You test one component. You test like the user like you just put things in and expect some outcomes. But you also run on the same process like that application. You can simulate anything I didn't encounter anything that is not simulated that it's hard to simulate as long as you are your test and the col
under testing within the same process. In terms of ROI yeah, I would definitely prioritize and focus on first on the things that the user does and the things that are likely to happen and where the bugs are more severe. But in terms of the technical capability of simulating stuff, I think that today's in the jobs critical system, I rarely encounter the situation that something was not reproducible.
And I'll add to that that manual testing is kind of like by definition anecdotal, and I hate to depend on anecdotal evidence of correctness. And so even if if if you do a certain amount of manual testing, then record that testing and then turn it into an automated script, it doesn't need to stay manual forever. So a certain amount of manual testing, especially for I don't know complex systems, might be required, but again try to automate it going down the line.
Definitely agree. I also have if someone is interested in articles, that's called testing the dark scenario of your back end, in which I've shown in an OGS application how each in seven lines tests no test there was more than
seven lines. You can test really critical scenarios. It usually teams don't test, like, for example, test that when your bootstrap, your your beacon bootstop phase fails you then you get the right monitoring or test that when your live liveliness route is not behaving correctly, you get the right thing. Or when you know, very common in JavaScript word, when there is a process uncut exception or one hundred that rejection is happening in your application, do you do the
right thing? This is a critical event. I mean your process might just crush. Do you test this? I showed out like in five lines of code you can also cover these advanced or endling passes. So yeah, I think that.
So again to summarize what we've talked about so far, So basically what you're saying is that you're kind of promoting more sophisticated tests, usually component level tests or integration tests, less unit tests, and that using modern tooling that you mentioned like Playwright, like v test, browser mode, and like the storybook testing, it's much easier to do than it used to be, is basically what you're saying. If I understand correctly.
Yeah, much easier than before. And i'd say, I mean I'm not against unit tests, but I think that for a start, your first step should be what the user is doing. And the user is not stretching functions or models, right,
The user is visiting a page. So I think that the first things that you should test is your page, how the user interact, and then, as you call it, if you realize some complex, highly complex logics that you want to isolate, testing isolation for better whatever performance, then yeah, go for unit tests. But I think this is kind of contextual and optional.
Well, to be fair, unit tests actually were kind of born out of TDD, So initially the concept of unit tests, at least for people who take that approach, is basically, we first write the tests, then we write the code. Now AI writes the tests and the code, so that's less that approach to be Also, to be fair, I've never been able to adopt the TDD model. Maybe I'm too much set in my ways. It just doesn't seem to be the way that I think. But I have
gotten value from unit tests. The main advantage of unit tests is they're very lightweight, so they're almost effectively free to run, and they run, and I have them run constantly because you know, I don't care and if something break, it's there. They test very specific things so you get an instantaneous indication of where that problem is, so you don't need to think about Like with and let's say, especially the end to end tests. You know the test fails, Okay,
why did it fail? It often requires pretty significant amount of investigation in order to understand why in end to end tests failed. And like we talked about before, it could also be related to certain things that to you know, flaky tests because you're dependent on external services. With unit tests, you're not dependent on everything and anything. You're just testing your own code. It runs very quickly. You're testing a very specific function. That's say, and.
You know I've introduced I made a change that function. That's why it breaks.
Couldn't agree more. You know, I observed once something very interesting that can it back. For those of you are not familiar with, Can is kind of the father of DDD, A person who should really appreciate unit test and he said in one of his blog posts, unit tests give up predicting suitability to production and inspiring confidence. So it's not about product confidence at all. It's about being writeable,
fast and specific. Now I think that most of the developers are looking in testing for confidence, and yes, it turns out it's not only about this.
Yeah, there's that famous picture about the difference between unit tests and integration tests, about you know, building urinal in such a position that when you open the bathroom stall door.
You can't actually get at it.
So everything is fine at the unit level, at the unit level, but when you put it all together, it doesn't work. The only we've seem to have lost your picture.
You're still with.
Us, Yeah, I'm working on fixing the camera. The good news is that I'm still online. Yeah, okay, And I think it worth just revisiting for one last minute that that crucial decision between the three front end things. So we've mentioned playwright. I think that storybook and Viness browser mode are also great options. And we didn't story books or storybook if you have a design you asked me,
what would I pick? So, if we're having a design system and rely a lot of small components that we tend to communicate and testing isolation, storybooks gives a very compelling package in which we get first a great workflow of visualizing our component and documenting them, but also a great new testing experience. So you write a test once
and you get both accessibility testing. If you have some accessibility issue, you are covered visual regression testing without writing any other test out of the box, if you're if you are out of the box, and if you sharing your credit card, visual testing and also also functional testing using playwright under the hood. So you get where.
Say visual testing just to verify that you understand your terminology. You basically mean that if I accidentally, I don't know, change the font or something, it will break my test because the content, the actual pixels on the screen are not the same as they're supposed to be.
Exactly. It should be smarter than this, and it should realize and detect significant changes from like minor improvement, but exactly this. Since that functional tests can tell you about. So your side menu is crewed, something some I don't know, something is totally broken on the screen, a big change in the layout.
You put a red background or something for testing purposes and then forgot to remove it.
That's that's that's a nice example. Yeah, so storybook gives you all this together interesting option. On the other end, it's the more sophisticated option. It's it's all the storybook narratives and its browser modan hand play, right, So you have a lot of moving pieces to handle.
Is it open source? Is it? Is it a paid product or service?
What? What is it?
It's open source, it's a free product, except I think this is how they make the revenues. Accept the visual regression test. What you just mentioned this one is is a paid service.
Okay, cool?
Hey, just to clarify terms. I know we've talked about different types of testing. We talked about unit testing, integration testing, and then testing visual testing.
Uh.
The one I always get hazy on is a difference between I guess what integration testing is how you find that versus unit tests versus and the s test? What excuse me, end to end testing? So what's what's your definition of integration testing? Yea, because without understanding that, it's hard to get Dan's toilet joke about uh integration testing.
So yeah, testing terminology is totally broken. I think that if when you say end to end testing, or or even unit testing and integrationion testing, different people will understand different things. And for this reason, maya suggestion in every company in your company, name a very specific name for your test which is not integration and to take for example, Uh, just one example. In one of my customers, we used
to test pages. We take a front and page we isolated and the test cope is every time one page. We call it page testing, and every everyone understands what page testing means. It's a test of a page. Integration tests just mean might mean three different things two different people. If we try somehow to clarify the integration test is. I think that the most common definition of it that we test the user journey in a thing. We test the wholes user journey in a single tier of the system,
not and then tire system. So it might be the user visiting some front end but without stretching the entire system. It might be a user visit a call, coding some backend API and expecting your response. This is our examples for integration tests.
So correct me if I'm wrong. It also includes, for example, testing components in isolation. So let's say I implemented a date picker component that the date range picking component, because you know you don't get that out of the box in the browser, and I want to verify that it's working correctly, and I can host it within like independently and basically simulate interactions with it. It's not end to end because it's not done in the context of the
entire application. So it's not for example, if you think about I don't know, let's say some sort of hotel booking application, it's not done in the context of actually looking stay at a hotel. It's done in isolation, but it's still done at the level of actual user interface and interactions, not just invoking functions and checking return values.
Exactly. You're in other words, you are testing the user experience with this in its almost real environment. There is a browser here, There is your code. There is like you have multiple layers here orchestrating together. You have that UI kid usually the UI kids that you're using, and then you have your code that trapping it, and you have us sometimes your state manager, and there is the browser. All the moving pieces that exist in production are playing
together in this test. In this regard, it's integration.
And if you're talking about non front and stuff, for example a node system, it might be testing a particular specific API call. Again not in the context of an actual complete user flow, and potentially even with certain other parts of the system being mocked like we said before, and not actually working with let's say you're working with
various external services. You're not actually working with these services, you're mocking them, and you're just testing effectively your particular let's say micro service or particular aspect of a microservice in isolation.
Yes, exactly what I observe is it works great for backing teams is testing your entire micro services is put the infrastructure on local doctor composed, test all the layer of the micro service. If you deploy it, you test it, your data access there, your business logic, your API, but only intercept calls to other services. Otherwise it becomes end to end with all the downsides that we have mentioned.
Yeah, I'm currently actually doing exactly that, and the portrait that I'm working on. Being able to use a doctor composed to effectively recreate the entire system on your own particular on your local developer machine and then be able to dynamically deploy changes to the specific micro service that you're working on is effectively a godsend. It makes life so much easier when developing in such complex environments.
Definitely, I mean's surprising how fast this test is. Like I think usually like a fifty proximately fifty test for micro service on a developer machine with real database last lesson twenty seconds. So yeah, it's a really efficient technique and maybe it's worth mentioning if we already discuss back end. There is now a build in test runners in o JS, mostly for back end, I think, and it's really gaining a lot of momentum.
Yeah, it's it's interesting that it seems that between node implementing its own testing built in testing capabilities on the one hand, and v test on the other end, they're kind of cornering the testing market in a sense. And I recall, for example, seeing this post on x I think from KENC. Dodds talking about how happy is that the React testing library is no longer needed because effectively all this functionality is now built into v test.
I think yes, And I think it's really interesting to see how people are contributing libraries and ideas together. So it reactesting Library was a library, but it was also
a philosophy of how you test. Now the library is probably fading away or at least losing some momentum, but the concept that the concepts there, the techniques are being totally used in Vitas browser mode and in Playwright, and so many new testing framework are taking inspiration and building on this that It's very very interesting to see this evolving process.
So in general, what would you say, is it still much more difficult to test the front end than the back end or are we achieving some sort of parity?
M m yeah, I tell that I'd easily said that testing a back end is much much easier than testing a front end. Yeah. I mean, first, you can also judge it by the tooling. I means there are except that new test runners of NOJ that is actually bring
less features. It's funny because the new test runners that a lot of back end teams are now using, it's us it brings much less testing features and what used to be like ingested people don't need even all the features, and there are very few like new tooling in the
back and accept maybe contract testing tooling, which we will mention. However, you see that in the front and words there is a bloom and always constant change and struggling to build new tooling that are can catch up with the front and complexity so well.
The fact that we actually need visual testing is an indication of how much more challenging front test and testing is. It's it's like almost like saying, let's let's test the bits flowing over some network connection and and basically try to analyze the system from that. If I'm using like silly analogy. So you were talking about you mentioned contract testing, Can you elaborate on that?
Oh, definitely, I think that this is now one of the most interesting fields in testing in overall. I would even say that if in twenty twenty five you're not doing your your coding against API without any explicit contract, it shouldn't be an option on the table. First, let me explain there is. So we are all consuming API, whether front and to back and back and to backan
and every system is distributed. Now, if as a consumer of an API, the payloads that I'm sending and receiving for me are just kind of unknown jason, some kind of Jason that I can control and predict, then imate risk of receiving a lot of breaking changes. So say for example, things work, but then my the APIs and un consuming change something. My test pass, but production will fail. And we are at this point we are really good
at testing our components. But but the ocean, that ocean between components, we have much less testing ways and techniques to ensure the two different parties are aligned on the on the API between the two and now there is a there is a bloom of new techniques and tooling to cover exactly this as API consumer, how do I know? How do I get a guarantee that the payloads are changing and I should be prepared for that.
So what are the solutions in this space?
Yeah, so there is a spectrum of tooling and options. Uh, the simplest one just in a moder repot, just creates OD schemas, share them between the back end and the front end, and you get a solid protection, right, I mean, whenever the back end changes the schemas, your front and tests or it can be also your back end will fail because you have now you can generate typing from this. From these schemas, you can also do runtime validation and you're protected. But it doesn't cover a lot of frisk.
On the other sound side of the spectrum, the more fancy tooling are mocking servers that will let you know that we'll provide you with a very sophisticated mechanism to synchronize the two parties. But there is the middle. I think that most of the solutions that are somewhere located in the middle between that are not too complex and not too simple and are not simplistic. Use the following.
The API provider generates open API, usually it comes for free, and it somehow shared that open API with the consumer. Now the consumer all the consumers have tooling to generate both types, but also run time validation out of open API. So now as a fronten for example, whenever my back end changes the open API, if my types I say that's a new field was added, my code will generate
types types based on this open API. And if I'm trying to access some fields it doesn't exist anymore or is the wrong type, I will get compile time or testing failure right away. So it's kind of me and back end are always aligned without spinning up a complex and fancy edit end environment.
Yeah, but like you said, it almost kind of necessitates in many cases working in a mono repo. Otherwise, First of all, I hesitate to call it tests. I mean, it's it's basically you know, one of the benefits of using static typing is that you get certain bugs become impossible because of the type system. But I don't really consider it to be tests. It's basically that's it might You could call it another step in your build that might fail, but I hesitate to call it tests.
Maybe it is. I don't know. That's an interesting philosophical question.
Some of the fancy solutions are going beyond static typing. So what they do is that if you have an open API, it's not all about typing. Some kind of the assertions there are can be asserted only on run time. Say, for example, you have a rejects some of the fields type is a rejics. You can't assert this in typing Typescript. Also, Typescript is now adding some kind of tragic support. So
what these tooling are doing during testing? Whenever you are sending to your mock some on payload, they are very fine whether the received testing payload is indeed conforming to the open API definition. So in that regard they can make the test fail if what you're using, the pedal you're using is not what the Beckend API defined. Of course for this you need like real testing. And maybe
it's worth mentioning one work very cool features. I guess you both know ms W. It's a very popular JavaScript library for mocking intercepting HTP requests.
No, sorry, don't.
Oh okay, So let me introduce it. Ms w H is a library that allows by the way both also know jes also back in. It allows you to kind of define in your in your environment, kind of in kind of intercept network requests and say, hey, whenever I'm approaching another API, say the beckend, intercept it in browser. It's using by the way service workers, so you will
see the request in the network tub and Chrome. But instead of going out outside your browser, MSW will return the response on behalf of your API, so you get
a much more a much faster, and more predictable working environment. Now, ms W added a new feature this year, and we're discussing recent changes that instead of you defining what will be the API responses and you might be wrong there right what you think that there might be threefills in this payload, the beck end in fact, in production will return four or two. Ms W generates the responses from
the beckend open API. So you provided, hey, this is what the API provided us the real world, the real scheme as it was generated, and it takes and generates responses based on this. So it's kind of you are you and the API that you consume are aligned, are always aligned? Does it make sense?
Yes, as it does.
Yeah, MSW stands from Mark Service Worker in case that wasn't clear already. So before we wrap up, is there anything else you wanted to talk about real quick that we have not yet covered in this world?
Let me see, I've prepared a table of content here on the side. We can discuss a little bit more about testing with it. I also have we have covered this nothing significant syd with We have touched the measure points and advancement in testing, all right.
So if before we I do have a final question though, in this context, So when you usually are when you're brought into a new project and and you want to before making any changes, I assume you want to make sure that you have good test coverage, because who wants to start making changes in a project before we have
good testing coverage. What's the main thing that you're looking for in order to say this project has good test coverage versus another project that you might say this project does not have good test coverage.
Yeah, that's that's the million dollar question. I'm even struggling to enter it. It's so it's so deep. But I think that I'm using two I'm using two aroistics for this. The first one is unfortunately not not in period. It's not measured. Uh, if people are moving, if people are really are having confidence to deploy fast, deploying deploying fast and just feeling confident to do it, you see it, I mean you you feel it, they trust are testing.
That's I think the ultimate eventually the ultimate test. On top of this, I'm.
It's literally just to just to pause you for a second, it's literally that subjective feeling that you test. You can ask people working on the project, how well do you trust your tests? If you if you make change and it makes its way all the way to to to being deployed to production, how secure do you feel about it working properly and not breaking any of the things or introducing any form of regression. It's it's essentially that that level of subjective feeling in the team.
Yeah, that's the bottom line. I'm also using test coverage. I'm using branch coverage. Yeah, I know that it's not a perfect metric, but it's very simple metric and for the majority of teams, they are not like they're not fooling themselves. If if if system has a very high branch coverage. It's means that the testing are indeed testing most of that ninety coverage, not all of them, but a high percentage of this at least it tells me that most of the system areas are are trying to
be tested. On top of this, I usually add some kind of mutation testing. So mutation testing means that it's a framework that plant bugs in your code, Like it can plant a bug in each and every line and then it runs your test and then true that they fail. And if yeah, now, because I mean it's it's an it's amazingly efficient technique to realize testing efficiency. But on
the other end, it's really slow and cumbersome. So what I usually do I run it like I run it occasionally just to get a sense of what is the ratio between the COD coverage and the mutations. So in other words, if I see that I have COD coverage of ninety percent in a file, say I find file as ninety percent coverage and the mutation score is very low, it tells me that this is a nonsense coverage. You
know that that's just visits amer coat off it. But it's not asserting one anything On the other hand, if I seek correlation between coverage and mutations in file, I know that this coverage is authentic. It's like trustworthy. So this is the empiric side of measuring that the test.
Cool, Okay, thank you for that.
Elucidation is that how you put elucidations.
That's the word.
I'll I'll google that word right after elucidate to explain something clearly is basically what it means.
So that is the word of the day, elucidate already.
So without move on the picks, picture that part of the show where we get to talk about anything we want to within reason of course, board, games, books, food, movie, tech, if you want to, I'll go first and my pick. I do have a pick before I get to the high point of the Dad jokes of the week, and
it has to do with dad jokes. So at the end of our podcast episode that Dan and I recorded with Ryan Carneato and Tanner Lindsley, I found out something that I wasn't aware of that now Crosby, who was the founder of a G Grid, had died and apparently it is some type of helicopter accident.
Well, it's to be fair, it's not new right.
I know it is the first I'd found about it, you know. So yeah, that just tells you how out there in the news I am every day, and that brings back memories for me. So four years ago we did a Jobascript Jobber episode with Nile, myself and Ag and just a j O'Neal and Chuck and the Reason
that I was. Episode five oh four came out in October of twenty twenty one, and this was one of my favorite episodes, as I mentioned before, because when it got to the end, Nile just jumped right in and throughout a whole string of geek dad jokes that I had to do with going to a no sequel bar and there's no table, So then he went to a sequel bar that had tables, but he couldn't join anybody, and it was awesome.
I was giving him a standing ovation.
So that's my pick of the for the week is you can find it on Top Endos episode five o four and if you want to look at the transcript or listen to it, it's it's one of my faves. So with that, we'll get to the dad jokes of the week. So question, how come nobody laugh at the joke about the fault DG eighteen.
It was poor execution.
That's good, all right.
My wife told me the other day, I'm really getting sick of you over using contractions, and I said, it's what it is. And then, uh, someone threw a can of soda at me today or i'd say pop on the West coast.
But I'm all right. It was a soft drink. And those are the dad jokes of.
The week, and those who are definitely dad jokes.
Yes they were.
I'm embarrassed to admit that I understood two of the three. I promised to to rewatch that section and catch up.
Okay, yeah, I can explain it. Yeah, it sort of kills a joke and you have to explain it.
That is.
Dan. What do you got for us?
Okay, I've got a couple of things. So first of all, I posted it on x but I also want to shout it out a big thanks to THEO also known as THEO three GG, who puts out a lot of
content online. I'm working on a project that involves Stripe integration for one of our services, and I've not worked with Stripe significantly, I think, ever, so I was kind of learning a lot about the stripe APIs and how it works, and they seem but Theo's video was essentially all the stripe gotchas that he encountered while working on various projects and how to overcome them, and it was very, very helpful. It came along at just the right time and undoubtedly saved me from a lot of pain down
the line. So thank you for that, CEOs, Thank you for sharing this information. This is what sharing on the web is all about. So that's one thing I wanted to mention. Another is that we did this kind of watch along party at work. It's something that we tried for the first time and got a lot of good responses. It's great when you can have somebody at work giving a talk on a particular topic, but it's not always possible. Obviously,
you may not have experts in how on everything. So what we actually did is we wanted to learn more about MCP and Jack Harrington, who we've had on the show a couple of times. He was also one of the hosts on the React podcast, and he created a series of videos about MCP that are just excellent. So we watched several of them in a row. They're actually fairly short each They're like ten to twenty minutes each
and he explains things so well. So we basically watched a video and then had a brief discussion about it, and then watched the next video and so forth, and it was really great and I highly recommend this if you want to educate people at work. It's a great approach, I think. And the final thing I want to shout
out is that I recently countered this video. This guy does videos about old hardware, and he released a video about a personal computer from the early eighties really called the Ti ninety nine four A. And the reason that this hit close to my heart is that that was my first personal computer when I was a kid. It
was a very weird and wacky machine. And he literally kind of opened the screws and showed the insight of how that machine worked, to explain why it was so weird and wacky, and why it worked the way that it did, and what was good about it, and a lot of things that weren't so good about it. And you know, it brought back a lot of memories. So I'll share that link to that video. Let's just say that it was really interesting to probe for a system that had all of four.
K of RAM.
You know, in these days of gigabytes, you know, this is all we had.
Yeah, that's it.
Four kfram And within that four kf RAM, I implemented a Donkey Kong like game that had two screens. And I was like thirteen at the time, so yeah, it.
Was which language did you use?
I'm curious, basic, because that's what it had. It was.
I joked that it simultaneously got me into programming and kind of also scarred me for life.
So my first.
Job in tech was doing tech support, and we were dealing with Win just three point one with our software, and I was constantly having to help users with memory issues and upper memory and lower memory and shifting things around so that things could run. And and then when I was ninety five came along and a lot of that went bye bye.
I have a whole story about that, but we don't have the time. We could probably do an entire episode on reminiscing. So those would be my picture today.
All right, Johanni, or to you, Yeah, I've recently found myself I'm interested in quality coffee, So I realized in mostly in the coffee story, So I really I always thought that like an expensive coffee is like expensive wine, you just pay for you know, for that brand or for that weird test. But then I when I started reading more and more and see videos about testing, I realized that like between the farm and your cap, they
are like ten mistakes, like like dozens of mistakes. Most of them are intentional, intentional to just to just cut coast and make the coffee horrible. Take for example, in any commodity farms, when they when they pick the cherry, when they when they collect the cherry picks, some of them are already just mature and some are unriped, some are red, some are green. Uh so, but you know they have to be efficient, so so just take the
entire tree down. You get coffee bins that are right unripe together, some are really sour, some are really really sweet. And and that's just one thing that the coffee coffee production is doing in order to bring your coffee with lower costs and more revenues for them, and really getting a great cup of coffee demands a lot of a lot of uh decisions and optimization all over the all over that process, and it's it's there's no point here,
just a very fascinating journey that involves biology, nature people. Yeah. Other than that, I realized that we are getting more and more dependent on lms, right, I mean we're about to use it for ten percent off coding or maybe ninety percent. I mean it's objective, and I felt like it's too much of a black box for me. I can't understand the mechanic them. So say, for example, I'm
providing it with fifteen instructions, it's respecting thirty. Why it fits inside the context within the Why did you ignore twenty of them? So I bought a book which is called I wanted to understand this box from inside. I bought a book that's called Building an LM from Scratch. It's just teach you build all the layers of an LM. Of course a very simplistic one, but you get idea
of all the moving pieces. I'm now reading it, coding it, and I think that I, after processing four chapters, I can surely recommend it.
Right, to go back to your first pick, I would.
All I will say is that the term quality copy is an oxymoron, so there is no such thing, and I'll leave bit at that. So all right, Well, thank you, Johanni for joining us today and talk to you about testing.
I appreciate it you coming here before we go.
If people want to get hold of you and give you money and and do all that kind of stuff, where is the best place to track your genius?
Thanks for mentioning this. Well, my gietime page has a contacts There is also goldberg Oni dot com.
And you said that you work as a consultant, so I assume that you give assistance training and help organizations improve their testing and test coverage and quality of tests and whatnot.
Yes, workshops ends on, ends on, and holding any any any kind of assistant and training.
And that's it.
Joannie Goldberg Goldberg Goldberg, Yohnnie excuse me? Alrighty and Yonie is y o n I just.
Yeah or gita dot com slash Goldberg Yonnie.
Okay, alrighty, Well that is it. Another thrilling episode is in the books. Thank you for joining us and we will talk to you next time on job script Jabber