How'd you like to listen to dot NetRocks with no ads? Easy? Become a patron For just five dollars a month you get access to a private RSS feed where all the shows have no ads. Twenty dollars a month will get you that and a special dot NetRocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, Carl and Richard here with your twenty twenty four NDC schedule.
Will be at as many NDC conferences as possible this year, and you should consider attending no matter what. The Copenhagen Developers Festival happens August twenty sixth through the thirtieth. Tickets at Cphdevfest dot com.
Ndcporto is happening October fourteenth through the eighteenth. The early bird discount ends June fourteenth. Tickets at Ndcporto dot com.
And we'll see you there, we hope.
Hey, guess what it's Dot need rocks again. I'm Carl Franklin, that Richard Campbell here for the nineteen hundredth and seventeenth time.
It's World War One. It's World War one. It it's officially started. Okay, we're well into it at this point.
Yeah, you're right, yeah, yeah, right, So yeah, what's up with you, mister Campbell. We just got back from Dev Intersection. Had a great time there. Yeah, I had a great time there. I was just writing up some of the pieces, the keynotes and things. I mean, it's a crazy time, right, Like the software is moving fast. Some of the demos were astonishing. You had a good workshop, You're real happy with it, very happy. Yeah, it's like thirty people in
it or something, and all about Blazer. Jeff Fritz did a sort of an intro to Blazer workshop, and I did a more more intermediate advanced workshop, and I also had a talk called Gotcha's in Blazer Web Apps and not at eight because there are some gotcha's. There are some things that Microsoft intended to work a certain way and then they didn't really work. So things that work around.
It's not that bleeding edge of technology anymore though. I mean you're kind of at V three. Things are sort of maturing out, like it's almost a grown up.
Yeah, And that's kind of why it was surprising that we had gotcha's in the first place. But I think they took everybody by surprise, not that there aren't workarounds for it. But you know, it's not a reason not to use Blazer, but you just need to know what they are.
I think you've also got a moving surface around you too. I've got the comment I've got will tie you into that pretty well. Yeah, because there's things on change in all the time.
And just to do a little plug, I'm also doing that Gotcha's talk with on Sean Walker's virtual event on October third. I don't have a time yet, but look for that. Okay, okay, So let's get into it with a little thing. We call it better no a framework.
Awesome, roll it all right, man?
What do you got?
Well, you know, this started out as me sort of pointing to little things in the dot net framework. It was even before dot net core, and quickly ran out of stuff to talk about. So it just turns into you know, hey, what caught my interest this week? I'd pointed.
It was not that quick. There was a period we were doing three of these a week, and you went through the whole flip and framework.
Yeah, pretty much. I mean the stuff that the stuff that was tricky.
Yeah, several years worth of content.
Yeah, you're right. Okay, So I found this event in the twenty through the twenty sixth of January coming up here. It's called Global Game Jam.
Oh, that sounds fun.
Globalgamejam dot org. And so just like when musicians get together and have jam sessions and they feed on each other and get creative together and good stuff comes out, that's the whole idea with this. It's it's there's sort of like four hundred and thirty nine thousand jammers. Wow, this is to date, eighty nine thousand games created, one
hundred and thirty two countries participated. So it's it's all online of course, and they have a discord and a Twitch stream, and they have videos from last year keynote. It's just it looks like a lot of fun. And the thing is too that it's not just for gamers who you know, who have experience like this is for people who might be just game curious, like they don't know how games are created at all.
And the tooling these days is so good, like, yeah, don't need a huge team, you can do it by yourself, although I've seen lots of teams of three be very successful and make it maybe a mobile game, make a web game, like there's a lot of choices right.
Yeah, so that looks really cool. I might pop in and check it out and maybe create some games. Might be fun.
Yeah, that's a good idea.
Yep.
So who's talking to us today?
Richard and Grab a comment of a show eighteen fifteen, So about one hundred shows ago with our friend Debbie O'Brien. Maybe you heard of her, Yeah, and we were talking about play Right. This is back from October in twenty two and I got a few good comments on the show, but this one was very specific. Admittedly it's over a year ago now. This is from Chris Martin. He said, just getting into play right and it was great listening to the episode of Dot and Rocks on it. I'm
currently getting this implemented for a Blazer app. What's the babe for authenticating with azure ad in our tests? Thanks? And Carl responded to this actually back then talking about azure ad B two c of through Blazer training include a link in the show notes for that particular show. Although it's changing, it's been a year.
Yeah, it's changed and stuff's changed.
Yeah.
We went through this period where we were using MSAL, the Microsoft Authentication Library right as a way to simplify accessing it, and now there's even more stuff that's like I haven't even i mean even kept up with it. That's how new it is.
Well, they call it an intra now, and so in some ways it's clearly better. I think definitely. My repertoire for the show's coming up is a whole conversation about managed identity. Yeah, specific And it's one thing to manage identity in production. It's another thing to talk about managed identity in tests and in the dev cycle. But you've heard on the news this whole Microsoft Future Security Initiative or Insecure Initiative, and a lot of that has to
do with undersecured dev resources, demos things like that. So Microsoft is pushing hard for their own people to do this better.
Yeah, across all teams.
Yeah yeah, and to be that's super exciting because they're going to knock the snot out of this process, like they're going to make it better for their own needs. That means it's going to be better for all of us.
I mean we're talking about people who write white papers, who publish examples, who you know, maybe do this stuff with three sixty the developer team obviously, Yeah, I know I know my friend Jeff Fritz is in that.
Yep, lots of folks that are buried in it right now. And and again it's like there's a there's a feedback loop here. Not only should they follow the practices, but when the practices are just too awkward to use, they know where those guys live, Like they'll lay in wait in the bushes outside of the office building.
They'll tackle too soon, too soon.
No, No, I'm just saying, like I watched what happened to WPF back in the day when the studio team started implementing WPF, WPF got dramatically better, and it was that back and forth.
Hey, if you want to go way back when sql server implemented the dot Net framework inside SQL servert got better well, without a doubt, because the sequel team kicked its butt.
Yeah, no doubt. So anyway, I'm hopeful. Yeah, and Chris, thank you so much for your commented. A copy of music Cobe is on its way to you. And if you'd like a copy of music code by I write a comment on the website at dot at rocks dot com or on Facebook. We publish every show there and if you comment there in a reason the show. We'll send you a copy music cope.
He's code by still going strong and people using it every day to be more productive when they code or sleep or relax or focus or whatever. All right, so let's bring in Wi O'Brien. Before I do that, though, I need to mention that you can follow us on Twitter, and of course we've been on ex Twitter for years. But the cool kids are hanging out at Masdon. I'm Carl Franklin at tech Hub dot Social.
And I'm Riche Campbell at Masidon dot Angele.
Okay, so Debbie has been on the show before, of course, but if you don't remember, Wi O'Brien is a Principal program manager at Microsoft. Wie has over fifteen years of experience in front end development. She's worked as a tech lead and consultant for many important and clients with various technologies and often with a strong focus on performance, static sites and testing. She has led teams both in house
and remotely, as well as giving workshops and training. Also has many years of experience as a mentor for online learning platforms, Treehouse and open classrooms. Debbie's also a Google developer expert in web technologies, cloud andary Media Developer Expert
and next Ambassador. Boy, there's a mouthful, as well as a GitHub Star alumni and a previous Microsoft Most Valuable Professional and Developer Technologies Debbie, I think you need to be a little more What should we say, you know, just try try a little harder.
I think I know, right, I'm trying to get the YouTube Award. That's my next one, you know that like big thing on one hundred thousand followers. I've got five thousand, so got a long way to go. But yeah, I need more awards.
Yeah.
I tell people the kind of the same way. I just don't know what I want to do when I grow up. I certainly haven't grown up yet.
Why start now?
Why start now? Yes, we're talking about Playwright, and Playwright is sort of the canonical web testing and automation actually platform that does snapshot testing, like we talked about with our friend Simon Crop, and it does all sorts of other things. Why don't you tell us what's on your mind in terms of Playwright?
Yeah? So, I mean everyone should already at this stage be testing their applications, because there's absolutely no excuse in the whole world why people are not testing right. Go back a couple of years, like ten years ago, no one was testing, and I completely understood that because testing was hard. Testing to time testing, the slow testing costs money. And now it's the opposite. If you're not testing your application, you're losing money as a company. You're dealing with bugs
that you should and even have to deal with. So yeah, I've been trying to get people to test their applications and I think still there's people not testing. I don't know how to fix Carl Richard, how do we fix this?
I think most people assume and I'm going to assume that they assume that, well, why should I test? Because I know this routine or this method or creates this UI result. Why do I need to prove it? Right now? I can see that it works. But the question is, well, how will you know when some other developer takes over your project and does something on the back end that changes the outcome? That's when testing becomes really valuable, right, yeah.
And just changing like little like think about forms, right, Forms are one of the hardest things to test manually. And I don't know if you've ever done this a manual test and you start manually and what I mean by manual is going to the computer's a user and filling it that form and then you find out the typical For me, it happened to me today. I was putting my name in. I have an apostrophe in my name. Well, nobody tested the web and I couldn't put a specific
type of apostrophe in. It wasn't accepting it, so it wouldn't accept my name, right, And nobody tested that, so I couldn't. I couldn't just couldn't get in. I had to like go to the actual computer. I couldn't do it for my phone, had to go to the computer where I had had more access. I was able to
do it. But when you're actually testing that out manually, you're filling in all those things and then you press submit and you get this error message and then something is broken, and then you go in and you fix it, and then you go and fill that it again, and you keep doing that process, a repetible, horrible process that I used to do, that you used to do back in the old days, when you know, that was the
only way we could do things. But now you have tools such as Playwright that basically is an extremely fast you right, it is you times one hundred and that can fill that form in one hundred times faster. So that basically is automated. Once you write that test, that form is filled in. And this is the cool thing that you basically to that test. You just manually fill in that form once and you do that by using
the code generator tool. You code gen the test by filling in that form, it's going to generate all that code for you, and then you just basically press play, press play and on your CI it just runs anytime the pull request is done in so if someone changes anything, it's going to run that and it saves so much time and it's so easy and quick to do it. Yeah.
So the thing that occurs to me about web testing tools like Playwright is that they have an advantage because they can inspect the HTML, which is just text. You know, they can inspect the output, whereas if you're trying to do that, say on a Windows forms application or graphical application, you have to do image you know, ocr for the for the text and image processing and all of that kind of stuff, and we just don't need to do that with you get. You can still do screenshot testing, right.
You can say well, the screen should look like this, but you have that the webliner allows you to be a lot more accurate.
It brings it closer to the user. And what we're doing when we're testing Blaser applications, right, we're testing what the user sees, what the user does. Simple thing is testing dark mode and lightmode. This is the cool thing right nowadays everyone has to have their website in two colors, right, how can we test that? Well, we can test that the CSS class is there and that can be done true playwright, and then you can just easily have a test. Yeah,
that's dark mode, light moode. That's a simple test.
Right.
You do you care? Well, maybe you should if that's important to you. What's more important is, you know, filling out the forms, making money, putting articles in a shopping car, actually purchasing things, and that's what the user does. What buttons does the user need to press to get to that payment? And if you can test that whole process from that start to finish, the money is going to be in your bank.
Yeah. Well, I mean lots of friction you can remove in that process. And turns out if it's easy to buy, people do buy more. So you know there's money to be made in creating good workflows that way, right. I just think these are the tools that help you detect where you're stumbling before you push it out there and start losing money on it.
Now. I use a tool called b unit from Egil Hanson for Blazer components. Does that work in concert with playwright? Is there something that does what b unit does in Playwright? What's the whole ecosystem look like with that?
So for component testing itself, we don't really support anything outside of the experimental, which is kind of you and like the component testing yourself. I'm not sure if you can put a Blazer thing inside of vat application that's something and push that up. That's something I've never tested out. But we're basically like, when it comes to a Blazer, you're testing what the user sees. That's already compiled, that's there, it's visual, it's on that front end. So you're coponents
are making up a web page. Your components are making a story, and that story is taking you. It's more about the integration between those components. Sure, how they integrate together and when on the page they do what they're meant to do. That button is a component. For example, that shopping cart is a component, and that all interacts together to be able to get to that stage of filling out the form, to fill in your delivery address to get the product delivered to your home, etc.
So you can still a b unit test to test the individual components if you think you need to. But what you're saying is playwright works the macro level. It just works at the whole website. You fill in this form, you expect this result. If you don't get it, then the test breaks.
Yeah, Like think about it. The user doesn't care about your code, doesn't see your code. So unit tests is not covering what the user sees and does. It covers your code and is your code going to do the job right? But end to end test integation tests, they're covering what the use physically does in your website. And it's like manual testing years ago when I used to do this on a Friday right in a job ten years ago. Everyone stop, stop we are doing and start
using the website as a user would use it. Go and start doing stuff right and then fill out issues. Oh this doesn't work, file boog, this doesn't work. File a bug. That's ridiculous doing that nowadays, and still people are doing that. Still people are paying manual people to press buttons, and that should not be happening. So Playwright is that manual person pressing the button. But that's much faster than that manual, physical person that would have done that ten years ago.
One of the things I run into when we start to build bigger websites is we have so many tests that it slows down the CICD pipeline. I've gone to great lengths to like break tests out across ponstible instances and things to try and shorten that time up. Is there tools in play right to make that easier.
Yeah, so's there's things you can do. Like basically, you've got to remember what's the slowest thing about testing when it comes to to Playwright, it's the brows right, It's it's you know, you're normally you should be testing as well your website across multiple browsers, you know, across the fire across Firefox, across Chromium. And once you're doing that, you're spinning up that headless browser. Right. There's headless means that there's no visual parts, there's no graphical interface, so
on CI you don't need to see something. The CI basically spins up the browser without the actual graphical interface, but it's still spinning up a browser instance for your tests. And tests are running isolation, right, so that means you know there's a new browser instance being spun up every time there's a new test, because otherwise, you know your cookies would jump from the same if it was the same browser, right, and then you can messs of your
test you have flaky tests, you'd go crazy. So when you have like a lot of tests, you're spinning up all these browser things. You can do a couple of things, right. You can go charting shot across multiple machines, which can be good and can be bad depending on how you set it up. Because if you shard too much, remember you've still got to download the browsers for that shard, right, So if you have eight shards, there's eight times you're going to do that. Are you saving time or are you?
So sometimes you have to decide how many shards do I need and you've got to play around with those numbers. Now, another cool thing, cool thing that we have is the Microsoft Playright service, which is not the open source part of Playwright but the Azure part. So if you're in Azure, you can basically use cloud based browsers. So now the browsers that you normally are like downloading are in the cloud.
And I know it's like we all talk about the cloud and we have to imagine this physical cloud and the browsers are up there, and you're basically sending your tests up to this cloud so it doesn't need to download anything. What that means is you're just running your tests and you're not doing the browser instance part, which saves a hell of a lot of time. Now, obviously this is an Azure feature, so it's it's costs money, of course, but you save my in the long run.
If you're talking about scaling at an enterprise level, you're talking about large scale. You want to run I don't know, ten thousand tests and your pipeline is being slowed down forty five minutes is taken. You want to take it down to three minutes. You know, any company would pay for.
That, right, you know, if I'm running tests in the cloud, I'm paying for that. So the fact that you've made me a dedicated service for this purpose, that sounds good to me. That's just going to make fewer mistakes.
Is that ratio like forty five minutes turns to three minutes typical.
No, that's just me making up a number from the top of my head, because obviously everyone's tests are different and you'd have to test it out. But the amount of time saving across all the examples that we've kind of run it on and then the clients that are using it in the feedback they've been given us is the amount of time. No one is the same, right,
but everyone has saved immense amount of time. And then also what's really important to a lot of people as well is being able to see, like, you know, to see those those tests in a kind of like a dashboard style, to be able to go back and forward and then like have a report, right and see a report of like this is what happened, this is where it failed, and have a trace of that test on your you know CEE I, on your on the on in the cloud, so that basically you don't have to
you know, go to local and locally spin up that that trace file and then debug it. You can basically almost debugget in the cloud.
Right.
But yeah, so scaleability was an issue, and we solve that issue because we also need to solve that issue because otherwise we're losing time.
That's great news.
So testing as a service is that where we're going.
I think we just need to fix testing in general. And I'll be very truthful, we do not have it all figured out in Microsoft. We're still learning because we are so many people, so many teams. We're like so many mini companies if you if you kind of think about it that way, and everyone is trying to like, you know, deliver products. Right, We've got a lot of products, right, how do we test all these and how do we
create a better solution across Microsoft? And then how do we let you all use that solution because if we just test it into in ourselves, we don't get the value from like, oh but what about this feature and we need this so, you know, making sure that Playwright is open source and then having those other services for the enterprise scale really helps us to make a better product. But yeah, we're not saying that we have everything sold
in Microsoft and we know everything. No, we're building things as we go along, and we require like the Playwright service is going to the reporting is in basically GA this week, Right, It's like literally, it's new, so we don't have it all figured out because someone might come along and say, oh, we want this feature, Well cool, tell us let us know. We'll build that feature for you, because we might also think that feature is necessary for
other people. But yeah, everything is being built around making lives easier, making testing easier. I think easier is our big word here. Easier and getting more people to test.
Yeah. Absolutely, And I could see it's still in preview too, So it's just a hint that you know, you're all learning by how people use it and what it's for. But I appreciate the idea that you know, this is the same angle as why would I build a bunch of virtual machines if I can use an app service? Why would I, you know, deploy my own caching if I could use a reddish cash service like you go as far up the stack as he can. That's cool, that's I'm excited to see that, just because yeah, I
don't want to build these things. I'd like you to do it for me please and charge me by the test exactly.
And I think if you have that many tests and you're you're paying for that means you know you've got some good coverage going on there that it's worth paying for, right, it was just made.
Too many pages, you know, Yeah?
Maybe maybe.
Well, I mean it's been two years as you were on more or less, so it's lead to see Playwright continuing do evolve Like that's cool.
Yeah, so you have a special love for JavaScript frameworks. How does Yeah, that's writing your bio. I'm not hating, Nana, I'm just reading here, reading your bio. So how does a JavaScript play into Playwright? If at all is any JavaScript? Are there entry points for JavaScript in Playwright? If you want to do some custom things? How does you know? That's the question, I guess.
I think what's important to understand about Playwright is that it can be used across four languages. Right, So if you were like myself, I'm a JavaScript developer or typescript developer now is what the cool kids say, then I can basically use that language because that's what I'm familiar with, That's what my website is built in. So I'm naturally going to write my tests using the language that I know.
But Carlin Richard, you're dot net people, right, and you you're c sharp probably better than JavaScript, and your website is probably built in Blazer, so you're gonna want to write your tests in c sharp and not in JavaScript. And I totally understand that. And that's why Playwright is so cool because you can just do that, because you
can use Playwright as a dot net developer. And I know this is dot net rocks, but let's talk about the Java people and the Pithon people because they can also write tests using Python and using Java and it's the same API, and you don't have to learn new tricks, and you know, you can have one company, which we do.
We're like in Microsoft, we have like you know, various and we're using different languages depending on the teams and depending on what we're testing and what people are comfortable with. So we use dot Net, use Pithon, we use Java, we use JavaScript. There are differences, of course, right, I'm not going to lie and say it's the same across the board. It can't be the same across the board
for various reasons. Because test runners MS test, you can't compare the MS test to the pie test for example, they have different stuff, right, And in JavaScript, we built the test runner and play right, so it's a little bit different. We've got a little bit more leeway on, you know, doing certain things.
And I just want to say for the record that some of our best friends, right, JavaScript, Python and Java. Okay, so isn't that what they say usually when somebody uses you of anti say that some of our best friends.
But either way, you are testing a web app, you're just able to write the code for the test in c sharp.
Yeah.
Yeah, because playwright just sees like what's out on the website, right, what's what's being produced? The what the you know you inspect code, right, and so you can see what the user can see that accessibility role, right. It just sees that that is a role of a button, and it's looking for what a screen reader would read out or what a user would see. You sees a button and it clicks that button, and it does not care if your button is written in blazer React you write heighten whatever.
Yeah, no, and right, you know you don't want to make writing the test the barrier, exactly like you wanted to be as easy as possible. So I appreciate that writing the language you want. I could write tests in Python. That's one way to slow myself way down.
It could be fun there. You might learn something, maybe.
It's interesting you think about writing in different languages for different tasks.
Yeah, okay, that's cool. So break time. Want to take a break.
Let's take a little break.
All right, We'll be right back after these very important messages. Did you know you can easily migrate asp net web apps to Windows containers on Aws. Use the app to Container tool to containerize your iis websites and deploy to AWS managed container services with or without Kubernetes. Find out more about app to Container at aws, dot Amazon dot com, Slash, dot Net, slash modernize, Attention dot netdeville. Looking for the
ultimate SDK to handle electronic document processing? Meet tx text Control. Txtext controls your go to solution for seamless PDF generation, secure electronic signatures, and efficient digital forms processing all within your dot net applications. Empower your products with robust document management capabilities, boost productivity, and deliver top notch solutions to your clients. Trusted by developers worldwide, including me and Richard,
txtext control is the SDK that makes a difference. Check out demos dot textcontrol dot com for live online demos and see it in action. And we're back dot NetRocks on. Carl Franklin. That's my buddy Richard Campbell and our friend w O'Brien. We're talking about testing with playwright and just a reminder that if you don't like those ads and you don't want them, you can pay five bucks a month and be a patron and you can get a
feed with ad free shows. All right, So we were talking about we're talking about all sorts of stuff and testing in different languages. But snapshot testing is something that I learned about from Simon Crop. And so the idea is that you have you have these following inputs and you expect the following output and I get is that primarily how playwright works or is it even simpler than that?
So it's just another part of something playwright can do. Right, So snapshots is like, you know, you're testing that something looks the same that for example, this is really good for Wednesday, like those banners up here at the top of your you know, the Black Friday banners that appear, and then what I like is there and you want it to be there, but you wanted to, I don't know, make sure it's not there after Black Friday eatin doesn't
make sense. So you could, like you know, write a test that then I don't know, shows the banner, and then make sure that it is exactly the same as a banner right on the website. But yeah, there's a lot of people using snapshot testing or visual regression testing
as a whole ballgame. It's not easy, right because because across browser as well, the slight little differences, so you've got to, like, you know, also make sure your pixel differences are kind of the same, because Safari will do something a little bit different to Firefox because that's just
how they do things. But definitely, I think if people are getting started with testing, I wouldn't say, go and make sure your website looks exactly the same as the snapshots like everything, because you'll end up possibly things will fail if you change that button. You've got to re recreate those snapshots each time. So I would always say, like, that's an extra, that's great and it's cool, but first make sure that your user can do what the user
should be doing. Whatever it is on that website is going to bring you money or make you click that patron button that you talked about, whatever it is, test that. Then do the accessibility testing, the regression testing, the API testing, there's a whole ballgame of things that you can.
Explore, sure, right, but I mean the snapshot test is really about a look and feel thing rather than a functionality thing.
Yeah, it's about that it looks the same basically that it hasn't changed that the designer didn't put a CSS hidden on a button and now they can't see the button for example. Right, But then if that was the case, player wouldn't be able to click the button, so that test would fail.
It would still break anyway, exactly.
But sometimes there are times when visual aggression testing is really important, especially when using I don't know when they say, like a lot of advertisements on your website and you want to remove those and just see what the website looks like out those You can mask those kind of things.
Well, no websites are so twitchy these days, right, Like thank goodness for pie hole. Otherwise there's stuff flying at you in all directions.
And you can also exclude things like you say, I want this page to look like such and such with the following data that it because we put in a certain input. But you know, little things like the date in the time, we don't want to We don't care about that that can change.
Yeah, I'm kind of staggered at how it would do those visual representations you just talked about sort of headless browsing where there is no visual component, but is do something different when it wants to create a visualization.
Well, it's just like taking a photo and then like it takes a photo kind I'm just doing like an example here, right, you take your phone out and you take a photograph of your screen of the website, and then you change your code, and then it takes another photograph and then it compares those two overlays one in front of the other, and it kind of sees like is there a pixel difference? Has the button moved over there?
What's it expecting and where is the button now? And I'm using button as an example, but it could be anything, right.
But it is pixel perfect it.
Yeah, but you can adjust that, right, You can say I want it to be like so many pixels away because I don't know it's an animation that's gonna like, you know, I don't do stuff, and you know you want it to be able to move a little bit or whatever.
So I wasn't really thinking of taking screenshots, screenshot testing, snapshot testing something I learned from Simon because he has a tool called Verify, and it's really just testing the HTML output of a page. So like a form, Right, you fill in a form, you press a button, it's going to give you a summary of that data. And that is what it tests that the output of this matches what we expected to match. So I think I think we might be talking about two different things.
We're probably talking about different Yeah.
But it is that in that case, it is literally a text comparison to the HTML. Yeah, Like if I was doing visual representation comparison, I could write the page different ways but have to present the same way.
Yep.
Not that I'm good enough to do that, but you know that would be interesting.
Yeah, you want to make sure that this did with this ID has this content right, which is different than you know, comparing uh screenshat, which is also valuable. I mean absolutely, There's just so many ways to go about testing.
Yeah, I'm just wondering about the brittleness of tests, right, Like most people dive into testing the first round of tests they build. The next time they do enough to do the app, every single test is broken and they right, why did I do this?
Yep?
And you Debbie jump in on this, like, how do you write non brittle tests?
I think you just have to constantly test. You've got to like make sure that I don't know, I think, I think use the tools that are there. I would say, use the tools that are the code generator, right, the code gen will generate the test view. I think a lot of people going back the previous days would have wrote their test using things like XPath, using things like CSS selectors, right, And that was the right thing to
do back then. But our front end, and I don't know about you guys in the Blazer world, but we CHANGEAVASCRIPT are known for We're changing things all the time, right, We're constantly like changing, so our htmil is changing. And if you're going to change that, the x path is going to change, which means your test is now going to be brittle. It's going to break just changing your CSS class. New developer comes along and he changes a
CS class. If your website and your tests break, well, that's not good, right, So you need to avoid things like that. And the way to do that is, you know, use the test generator tools to generate those locators. Locators is how you locate an element on the page. So how do I all locate that button on the page. You use a get by roll button, and that button
has a name called submit, and that is tested. It's also tested for accessibility, and it's easier than for you to go out and just do whatever you want with that button, change the CSS of it, put it in a different place. Play right's going to look for that button with a roll of button call submit, and it doesn't care where it does in your page because that's
not important. What's important is that the user can click it, and the user can get to that next section of the page or whatever it needs to do with it. So I think making sure you test as close to the user as possible will save a lot of problems in the long run when it comes to brittle tests.
M So instead of on Fridays you guys all get together and push buttons, now you're getting together and updating your tests or maybe you're doing that every time you check in something, right, yeah, and make sure your tests are working.
And what's really cool is is like the code Generator tool is very cool because you literally like it launches up a browser window, right, and you put in your website address and you start using a use as a user would, and you generate that code, and then you've got tools like get up copilot, right, and you can do really funky things at copilot because you can turn around and then say a test step for my tests, and it's going to like literally take the code that
you've just written, that you've just generated, actually you haven't written it, you've just generated, and it's going to put it into test steps for you, and it's going to do all these kind of other stuff that you want to do it or put it into a described block or you know, you can help get a copilot can also help you to improve the code. So as a developer, you're not sitting there just like writing the boring parts of testing. You're using the tools to your advantage.
Mm hmm.
Good.
Yeah, And I think definitely I remember in Strange Loop. I think the third go around of writing tests for Strange Loo, we've gotten better at writing tests that just we're going to be more tolerant, like you're still checking the value the way you want to, but you're not fussing over you know, unnecessary elements of that. That means
that the slightest change in the test fails. Yeah. I think it's really an art to write good tests and stuff that will give you check the stuff you need to check without being infrastructure where minor changes the things just derail all the tests.
I think that's the difference between unit testing, right, whereas unit testing really is so close to the code that it's like you change something in the code or your
test is going to break. With play right, really, you can change something in the code and your test shouldn't break unless you change, say, like what the user sees and instead of a submit button, you put upload, and now it's going to like look for submit and it finds upload because that they messed up the translation or whatever, or.
You've removed I or you've removed a field or like you. I mean, there's also good test fails where it's like I made this change and the test shows that I made this change. Now I can remove the test or change the test. Yes, I don't mind it when it is clear on stuff. What I mind is, hey, you know, we wrote three hundred tests around this set of pages, and I changed this CSS structure and every test is vailing now.
And that's the difference between unit tests, right, because you write that test and then you can't really see what's
going on, right, you're stuck in code land. But in Playwright, when you have that failing test, you spin up a trace of your test, and that's like basically be able to go think of a video, but better so a video with dev tools, a video that you can manually step between every single step and go back and forward in time at your own pace, and have a DMB snapshot that you can just pop out and inspect the dev tools and say, right, I know what happened here.
Somebody puts CSS hidden. I'm just gonna use that example because that actually happened to me on one of them that I was testing on the podcast website, and it put CSS hitting them playright couldn't find it because it isn't visible to the user. So if the user can click it, but the code is there, the code didn't change. The CSS change and the CSOs could have been in a different file to the code, right well.
And so that could have gone the other way. Right, you've hid it the test pass and you deploy it, but the users can't work that. Yeah, you want to be able to catch that too. It's it's it's always going to be a battle.
Can can Playwright penetrate to the level where it can see components of the shadow? Don Yeah, it just works.
It just works, iphrames, it just works.
Just wow. Not well that was a simple answer.
Sorry, I try to make it more complicated.
Yes, no, yes, no question, I can't tell. That's great. Yeah, that's great because that that has been a problem.
In the past, you know it has. That's why we had to solve it. Because you know, problems need to be solved.
Yeah.
Do you find that developers are using playwright in their local cycle or is this purely part of the c i CD pipeline.
I think it has to be both, because you need to. I always think that unless a website is finished, which I've very rarely seen, what is product finished exactly exactly, So unless it's something that you are never going to touch ever again in your life and you're going to write those tests and it's just on CICD for whatever reason. But no, nothing has ever finished. Things are always changing.
So running your tests locally as you're building some code, as you're adding additions, as you're changing things always makes sense. Some people don't. Some people push it to CEICD so they can just get their code up there and they're not testing it locally, and then like you know, Friday, they make all the changes, they push and then they
can't go home because, like you know, something broke. So running it locally always always helps, and having the tools like it's it's when when tests are quick and easy to run, it means people don't mind running them locally. When tests, if your test takes forty five minutes to run, as a developer, I'm not going to run it. I'm not going to run it locally. I might run the one test that I'm working on, but it might affect other things the code I'm working on.
So and that's sort of that integration test that after the fact. Yeah, now you go check the integration effects.
So I always say, like new code you're building, I'm building this new feature, then I really want to work on those tests. I'm writing those tests. As I'm building the feature. I write the tests, and then I'm testing that and then I you know, it's now deployed and then it's running on CICD and it keeps running. Yeah, So it's a mixture of things.
I mean, a CICD part is not not optional. Before this heads towards deployment or load testing or the next stage. It must be thoroughly tested. It's just a question of how much of that can you pull down to the local workstation for that DEV cycle, the impacts that would have on them. I guess it would be great to organize it in such a way that you could only need to test the piece you're working it.
Are there any common misassumptions or barriers to entry or gotcha's that people will find misassumptions, Maybe that people will run into things that they assume that it does that it doesn't do, or what's the typical getting started barrier that you see people having.
I think that miss assumption is a word. Look, sorry, you looked up? I did?
I can't.
I just like, what have you met me? You've appreciate met me? Debbie, I'm sorry, We're okay.
So I think what's really important to note is that tools change and evolve over time, and things get better and we improve things a lot. So sometimes people will say I'm going to go write a test and they are too. I'm gonna use the word lazy, but it's just, you know, that's my word, but there could be another way of describing it. They're too lazy to read the documentation. So what they do is they just go and they say, I'm going to write a test. I don't want to
read docs. So what do I do? And what do we all do? We just Google right and you just throw it into Google or now we got the new Google. We got chat gtpt right, and we say, how do I write a test in Playwright? And some times the results we get are wrong and we don't care because we don't know because we didn't read the docs. We copy and we paste that code and we put it into our tests and then we go, my test is breaking, and I don't know why my test doesn't work, and I don't know.
What it works when it shouldn't work exactly.
So this is really important to kind of make sure you're using the latest version that you know, I mean, Playwright is you know, the first time Playwright came out three years ago, the locators were different to what they are now. And you know, sometimes chat GTPT isn't up to date, it hasn't been trained in the latest model, so it's going to give you an old version.
Right.
Someone on stack overflow maybe wrote that a couple of years ago. Maybe doesn't know much about testing. You have no idea of their knowledge level. You're just copying their code and their answer, assuming that they're all experts and that you're not an expert. But really, if you just go into the documentation and did the same search and the documentation, you would get that answer. We get a better answer.
Are there are there really good examples in the documentation of you know, for people getting started their walkthroughs or that kind of thing.
So I'm going to be very very honest and tell you that I did never basically ever install sea sharp on my computer until literally a couple of months ago. Of all guils, I built the getting started docs for c sharp, and I took it from start to finish, and I mean from start to installing on my computer.
Wow.
So if I, as a non developer, can write a playwright test in dot net by following the getting started guide, then a dot net developer should be able to do it ten times faster than me and should be very successful in a very short space of time.
Well done you nice.
Thanks, Yeah, that's.
Actually really easy to install.
Do it?
I realized it's actually a purple button. You just press it and then it just installs. I did not know that.
Yeah, I mean we always had this problem. But there's too many ways and so did some of them are hard and yeah, if you picked it.
You know what I did, I asked chat and you know what it did. It sent me down the rum path.
Oh man, you go to the docks.
Who's lazy? Yeah?
Right, it's like have an intelligent mentor that sometimes loses its mind.
Well, it gave you an answer, it just wasn't the easiest answer, right, Like, that's that's a tough concept for a piece of software to figure out.
Gott to change your system? Prompted to can you not give me any dumb answers? That'd be great. If you don't do that, you're only going to get dumb answers.
I mean the answer sounded good to me if it felt familiar in my world. It was like, you know, yeah, homebrew, just homebrew it all up. Yeah, that's that's me, Mark that that does it. But no, you don't need that was it was?
It was it Hunter that told you. It's like we made a button.
There's a button.
Yeah, why don't you know about the button?
One thing that I've noticed as an aside here that Chat GPT is terrible at is CSS?
Oh boy?
Yeah, don't try to solve a CSS problem with chat GP.
Problem. Now I have two problems.
I have a problem factory.
Yeah, I tried it with Kubernetes and a head of container problems. Yeah, CSS is terrible. I'm sorry. Or did I have a problem orchestrator. It's one of those, you.
Know, I might as well ask the JavaScript JavaScript web queen here. CSS visualization tools exist so that you can see a tree view of your application so you can more easily diagnose CSS problems.
I mean, we just live in death tools, and like the death tools is in me, and even like you know.
You really think so, yeah, you think the dev tools are amazing for CSS. I mean, I guess you do see the tree, see.
Everything you see like accessibility for your components. You can see the color. You can even like modify it and say like that's that a color is not accessible. But if I can just like scroll it across a bit, this is an accessible color, which means you can go back to your designer and say, hey, that gray it's just not going to work for me, but if you want,
can we make it this grey? Because that would work accessibility wise, and then they just kind of get you know, they love their colors and their stuff, so they go, oh, that gray is a little bit different.
Yeah, I guess what I'm really looking for, And maybe the dev tools haven't have just missed this. But you know, when you're several layers down, like maybe ten or eleven or twelve layers down, and you have some CSS that's all messed up and it's not what you expected, can you like querry that and say, yeah, where did this particular attribute come from in the CSS tree?
Why is this hidden?
Yeah? Why is it hidden? Why is it purple? Why is it not purple?
I think a lot of people write very very bad CSS, and that is a weird problem that you need to solve, and deaf tools can't solve that, and there's no tool I can solve that. I tried to solve that in a company and I literally re architected the whole thing in the amount of times, like CSS, this is being used across twenty components and you change it once and it's going to affect so many others, but but you don't know about it because this file live somewhere else,
and it's just being cold and bundled together. I mean, tailwind kind of solves that a little bit. If you wanted to get out on that road, That's what I would have used and used.
Maybe there's a good AI solution out there waiting on the horizons that you can just ask, why is the button not purple? There you go or whatever, and it'll find it in the tree and.
Bob shrunkle AI is going to solve all our problems.
All the problems, all the problems I know. Yeah, co pilot for CSS. Here it comes you laugh, I bet it's coming. Well.
That brings up a great question, Daddy, what's next for Playwright? Like, is how long's your to do list?
I think the too list is never ending and nobody can ever know because like when we last spoke, I never knew that, you know, cloud based browsers were going to be a thing. That was never because we didn't we didn't have that scaling problem. We didn't have that many people using Playwright, so we didn't have all this feedback from all these people and companies and community. I mean, we went from zero members in discord to over thirty thousand members since since we spoke, basically right, so all
of those companies are now using playwrights. You need different solutions, so we basically use GitHub. Playwright is open source and people file issues and say we have this problem, we want this tool, we want this and everything that we built is based on up votes from our users and you just go to that file an issue and up voted.
How organized these issues are technically collecting feedback like I could see you guys are taking care of this part of the inner loop, like because people are asking for features, they things they think are bugs but may or may not be like mm hmm exactly. So really are developing out in the in public. Basically anybody can contribute. Anybody can read lots of comments too, like you're clearly prosecuting these like what do you think of this? What about that? Way?
You know, how would we solve it this way?
Because we have to understand, right, sometimes people say I need this feature, like we'll explain to us because I'm not seeing what you're seeing, so we need to understand, like, you know, what is it that we why do you want this? Why is it important?
Or the question that I'll microsoft people ask, why would you ever want to do that? It's it's not a threatening question. It's a real question, like why, what's your why?
Yeah, because then we can say, well, oh that sounds great, let's you know, put some time towards that and work on that one instead of this feature, right, or fix this boog instead of that bog or whatever.
Yeah, yeah, totally. And I said just and you know, you could write the code too, like if you're if you're that interested like that, you can contribute.
We have had we've had some external contributors, especially in the last releases, and I think now get up has done some really nice thing where you can kind of see their names and their picture and it's really cool. So that even like is nice, So yeah, do yeah.
To contribute to a project like this, that would be really fun because.
We are a very small team, so we can't do everything that people want us to do because it's just not possible, so we can only like say.
It's a triayeah, yes, I get that. That makes a lot of sense.
Very cool, awesome sense, like a fun project to contribute to. If you're looking first, you've.
Had a lot of contributors. Holy man, look at this list.
Yeah, hundreds. We have a great community. I told you. And Discord Yeah, oh, there you are.
I see your commit.
Well, Debbie, it's been an absolute delight having you on the show again, and congratulations on playwright success. And we hope our listeners will check it out.
I mean they have to, it's not we hope. You got to be strict to Carl, you gotta tell them.
All right, guys, my next episode.
You have all should have written at least one test, and I want.
To see some comments on hiver right.
I would say I would challenge I would challenge anyone who thinks or has never written a playwright test. I would say take seven minutes. I won't say seven, not ten's a little bit more than five. Seven minutes. Go to the getting started guide and the documentation of the dot net and basically write that test. And you don't even have to have a website. Just put in any website and just just go through it and write that test. Seven minutes and you'll have a test written to your developer.
You'll get a pay rise in a second.
All right, there you go. Let's take make this about money, all right, Debbie, thanks a.
Lot again, Thank you.
All right, we'll talk to you next time on dot net rocks. Dot net rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facilities located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back the show number one recorded in September two thousand
and two. And make sure you check out our sponsors. They keep us in business. Now go write some code. See you next time you got jas middle Vans
Home the