Changing Testing using Playwright MCP with Debbie O'Brien - podcast episode cover

Changing Testing using Playwright MCP with Debbie O'Brien

Jun 12, 202549 min
--:--
--:--
Download Metacast podcast app
Listen to this episode in Metacast mobile app
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more

Episode description

What happens when AI comes to your web testing tool? While at Build, Carl and Richard talked to Debbie O'Brien about the latest features in Playwright, including Playwright MCP, the model control plane for Playwright capabilities. Debbie talks about using Playwright MCP to buy a table. Ask your LLM what tests should be written for your web page, and then ask it to write them.

Transcript

Speaker 1

How'd you like to listen to dot net rocks with no ads? Easy? Become a patron for just five dollars a month. You get access to a private RSS feed where all the shows have no ads. Twenty dollars a month. We'll get you that and a special dot net rocks patron mug. Sign up now at Patreon dot dot NetRocks dot com. Hey, it's dot net rocks from build. I'm

Carl Franklin A Richard Campbell. We're a Microsoft build in Seattle, and I noticed this morning when I walked in here, I am the only human in Seattle wearing shorts.

Speaker 2

Yeah, of course you are. And I thought, which is weird because normally celebrity wear shorts, do they? Yeah, shorts, sandals and a gortex jacket to go together.

Speaker 1

Yeah, I guess the Birkenstocks, right, yeah, yeah, but you know, because it's little rainy, I guess people put on their long pants. But I don't know what it feels like short. It's spring here, can tell. But I looked all around build, only one wearing shorts. Walk down to the hotel today. Only One'm wearing shorts, all right, he So tomorrow I'll be in pants and I'll be more blending in Okay, all right, so I'll get my shorts on. Yeah before

we do better. No, framework, let's talk about nineteen fifty four. What do you want to say about nineteen fifty four? Well, that is the number of our show today, true, nineteen hundred and fifty four. Yeah, holy crap. So nineteen fifty four was your significant events, including the Supreme Court's Brown versus Board of Education decision, which ruled school segregation unconstitutional. Imagine that. Wow, that's all I'm going to say, nineteen

fifty four. Yeah, other note, they were stupid in nineteen fifty four. What did they know? Don't think that's true? What did they know? All right? Other notable happenings included the mass vaccination of children against polio, the beginning of the Vietnam War, and the debut the first nuclear powered submarine, the USS Nautilus, which has a significance is that of g Yeah, that's right, Groton, Connecticut. My father worked there, my mother worked, my brother Jay worked at the electric boat.

Speaker 2

They built a prototype reactor in Idaho, because that's a good place to put a build a reactor, you know, for a submarine.

Speaker 1

In Idaho. Well, you know where my father and mother went on their honeymoon, Idaho. Interest that's why the reactor. Yeah, that's that's.

Speaker 2

Where ricked Over was. That was where all the experiments.

Speaker 1

Were being done. So interesting, isn't that funny?

Speaker 2

Yeah?

Speaker 1

So yeah, I have a little history with electric boat. I never worked there myself, General Dynamic electric boat. But because I don't know, I was kind of a little hippie.

Speaker 2

He didn't you didn't want to be a nable architect like your brother.

Speaker 1

Well it's not that. I just didn't want to contribute to the war machine. You know. I thought I you know, I thought I knew everything when I was twelve whatever.

Speaker 2

So anyway, you must have some I mean Joseph Salk certainly and important, oh yes, you know. And Plio yeah yeah, and that was huge also the founding of CERN really yeah, the European Organization for Research in Nuclear Materials. Right, and they discovered the god particle, well they possibly they found in twenty twelve they found a candidate in the Bosun class of sufficient mass and they got what one hundred and twenty five GeV that would be comparable to what Higgs had said would should exist.

Speaker 1

Okay, but what I remember from twenty twelve is a everybody lost their minds thinking these guys were going to blow up the world.

Speaker 2

Well it was blown before that. That was we're going to make the black.

Speaker 1

Holes right and thus blow up the world. And then they said they found it, they found the god part. It was a bit more complicated than that, but that's what the news reported. Yeah, they said they simplified a lot. They also found a bunch of other interesting things like pentaquarics, and the configurations didn't even make sense, even locations of supersymmetry. We've every thinks that only Richard Campbell would undersleeve, never done this. Geek out, but no way, I have a

lot of notes. Are we talking end of twenty twenty five? I don't know.

Speaker 2

Maybe it's it's a lot. It's a ton of work to actually put that whole story together. But all right, they've also got that that's the largest anti manufacturer in the world. If you live from any matter, that's where you go.

Speaker 1

And it also spawns the novel Angels and Demons by Dan Brown. Okay, do you remember it? You know what I'm trying. Yeah, Okay, So before we introduce Debbie, let's get to better no framework. All right, all the music? What do you got? So again, I'm really into this. What's trending on githubb list. It's an aggregator that looks at, you know, the most popular GitHub repos and some of them are nuts, but some of them are pretty cool. And this one isn't development oriented. But it's YouTube music.

Speaker 2

And it's literally YouTube music.

Speaker 1

Well, okay, it's under the thhhh account on its YouTube dash music all right, thhch dot GitHub dot io slash YouTube dash music. Or because this is nineteen fifty four, you go to one ninety five to four dot pop dot me. So it's a YouTube music desktop app bundled with custom plugins and a built in AD blocker and downloader. Right, but this is using electron, so it's an Electron wrapper

around YouTube music. Cool. They got all sorts of interesting features including you know, themes and build your own plugins and all that stuff, and an AD blocker and a downloader, which apparently has been the holy grail of spammers, you know, malware.

Would they go out and say, hey, we have this YouTube download because everybody wants to download a video from YouTube, right, everybody has in the past to do that because they don't know how long it's going to be up there, and blah blah blah, and there's all sorts of malware that says they can do that and they infect your machine. It's terrible. I have no idea if this is malware

or not. I did not run it. However, since it's trending on GitHub, I'm assuming that people like it, and they like it because it works, and you know, that's it. But the ad blocker in the downloader by itself is probably a good reason to check it out. Awesome, So there you go. Love it. Yeah, who's talking to us today? Richard Abdy comment off show nineteen seventeen, which you did back in September twenty twenty four. We've talked to one

de Wi O'Brien. Maybe you've heard her. Yeah, yeah, And oddly enough, we were talking about playwright I know, shocking nineteen seventeen when jazz took over New Orleans something like that.

Speaker 2

I think it was also the middle of World War One. But let's talk I complicated now, Okay. A ton of comments on that show, by the way, and Edwin Aughten had this commed. He says, thanks for yet another interesting show. Regarding the challenge. You guys discussed with Debbie about finding which obscure CSS file is making your element in VIE, something we talked about.

Speaker 1

I hate that.

Speaker 2

Yes, well, he says, when using dev tools in Chrome or Edge to inspect CSS, you can go to the computed tab to show the values that the browser is actually applying on the rendered element.

Speaker 1

But you have you still have to crawl through the hierarchy of CSS to find the ancestor or the one thing that set that that is being cascaded down. It's really a pain.

Speaker 2

But in theory you should be able to hover over that and see all the computed values.

Speaker 1

You see the computed bit.

Speaker 2

And then search on that value. You might find that file you're not looking right, So that's the idea. So Edwin, thank you so much for your comment and a copy of music Coba. It's on its way to you. And if you'd like a copy of music Koba, I write a comment on the website at dot rocks dot com or.

Speaker 1

On the facebooks.

Speaker 2

We should publish every show there and even comment there, and everybody on the show we'll send you copy Music Koba.

Speaker 1

Music to Code by of course still going strong. Twenty two episodes music to code by dot Net and you get the entire collection in MP three wave or flack format. And before we get started with Debbie, I want to say that Jeff Fritz and I did an episode of the Blazer Puzzle episode seventy yeah called Playwriter's Block. Ah, what a great name, which we want to know why simple playwright test fails in a standard Blazer server app.

And that's all I'm going to say. You're going to have to go find out why and you can try to fix it yourself before you see the solution, okay. Wi O'Brien is a Principal program manager at Microsoft with over fifteen years of experience in front end development. She's worked as a tech lead and consultant for many important clients with various technologies and often with a strong focus

on performance, static sites and testing. She has led teams both in house and remotely, as well as giving workshops and training. She has many years of experience as a mentor for online learning platforms, Treehouse and open classrooms. Debbie is a Google developer, experting web technologies, cloudingnir Media Developer Expert, and Next Ambassador, as well as a GitHub Star alumni and previous Microsoft most valuable professional and developer technologies to them, nothing,

she said. When she entered the room, right, she said to them, welcome.

Speaker 3

I think I need to just remove some stuff from that profile.

Speaker 1

You know, I only read the first paragraph. There were three, there's more. It's one thing to read it.

Speaker 2

It's you know you've done those things, no question about that. And but it's the other thing to rehab it all read to you.

Speaker 3

Yes, that's tough, intimidating.

Speaker 1

I appreciate that. Who is that person? Wait, it can't be you, but Jeff Fritz and I are a huge fan of playwright. Playwright of course is you know as a fresting tamwork. I almost said a frest where we going a testing framework for websites that goes out and actually executes against your website and you you can essentially write programmable scripts. It's great. Yeah, and we use it all the time.

Speaker 2

If you want to scrape a website, play rights the way to go.

Speaker 1

Well, thanks, that's I mean, that's the point.

Speaker 3

That's something different.

Speaker 1

I have. I have written scripts to scrape websites. Who hasn't, right obviously, and Playwright's great for that because you can essentially what it's for it's not what it's for, but it's still great for that.

Speaker 3

Well, I mean seeing that you've opened up this and so yeah, obviously, like testing is the key focus of play right, and we all know that you shouldn't be you all have to be testing your applications and you shouldn't be spending time manually like doing the stuff that computers can do for you, because at the end of the day, that's just wasting time. And there is still space for manual testers and stuff. But the more you can owe it the better.

Speaker 1

Yeah, but it doesn't take the place of an API.

Speaker 3

And then coming back to your point where it's like basically you're talking about like scraping or browser automation. Have you used the play right MCP? Oh my god, all right.

Speaker 1

So ai'd playwright we ate, Well.

Speaker 3

We mcp'd it.

Speaker 1

So by now, if you've been listening to the last two shows, people know what MCP do.

Speaker 3

They owe amazing great, So we don't have.

Speaker 1

To control program tron is real excellent.

Speaker 3

So basically what you can now do is turn around to get up Copilot or whatever LM you're using and say go to a website and do X y Z.

Speaker 1

Right, So I actually wrote a call.

Speaker 3

Yes, I actually got it to buy a table for me.

Speaker 1

Wow.

Speaker 3

Yeah, I wanted to buy a table for my twins, so I wanted to find out the cheapest option of buying it, and I wanted to know how much the shipping cost was because I live on an island and they always throw on shipping costs and then it's not really the right price and stuff. So I asked AI using the MCP, I said, can you go and check it out? So the play Right MCP went and opened up a browser window. He went to the website that I wanted to buy the table from, the particular brand,

the Stoke Table. It started searching for different combinations. It found the table plus two chairs is a cheaper option, and I wanted to buy two chairs anyway, so I buy the whole thing as a package.

Speaker 1

I say, wait, I didn't buy it for you.

Speaker 3

Not yet, not yet, not yet, not yet. I'm not there yet. It then went to the filling in the forum to find out the shipping detail. So made up an address in MAJORCA and it filled all that in with a dummy email address, and then it was able to find out the actual shipping cost. And then it was able to say, right this is the actual price of it, and then I was able to say do I want to purchase it or not right now? Because it opens up a browser and it does all the work.

It's still just a browser, right, so I want to actually finish the purchasing, I could just go in and press click on the PayPal button and go and buy that table right.

Speaker 1

After you edit your address.

Speaker 3

Well I would have to edit the address because they put in a John do. But that's the power of the MCP. It's basically able to go and open up a browser, do the tasks that you could do yourself that you don't want to do, and then you can still continue because it's a browser window.

Speaker 2

Somehing magic exactly, It's just a pre filled in browser window.

Speaker 1

I got a story for you guys, and I know I've told it on Dotton it Rocks before, but it's been years. So back in the two thousands, when you know, back back in the two thousands, that's all I'm going to say. I was courted by a member of the Connecticut Technology Council. Now I'm in New London, which is in the south eastern Connecticut, and the people in Hartford, they don't like New Londoners, like they're big eastnobby kind

of insurance money people and whatever. So this you know, they meet in Hartford, right, So I say, okay, I will go to a meeting. I went to a meeting. I joined the Technology Council whatever, paid my three hundred dollars whatever, and went to one of their meetings. And it was just ridiculous. I mean, and the stuff that they were talking about was like, you know, how to

log in? What is a cookie? Right? So, but they had a lot of really great members and they had a member's website, and they didn't have a mailing list. And I said, do you guys have a mailing list? No, sorry, we don't, but all our members are listed on the website. Hmmm goodness. So me being it was a kinder, gentler web. I used real sockets. I didn't even use an HTDP. There was no web client. You literally did a manual get.

I did a get on the website, parts looked for the email the email tags that I wanted, logged them, you know, with their names, and blah blah blah. And I said, are you sure you don't I wrote back, are you sure you don't have a mailing lists. No, we don't have a mailing list. I'm like, would you like one? Because I basically scraped all of the emails and names from your website. It's and he said, like this is the best response. Well, you're obviously some cracker

jack technology program. Like aren't you guys the Connecticut Technology Council. Shouldn't they be looking to you for technology solutions? What are you doing? That's one of my greatest superiority moments. Nice. Yeah, it felt very superior, very clever. But yeah they didn't.

Speaker 3

Did they send out a mail?

Speaker 1

No, they didn't invite, But I did send out a mail to all this p and I got some great responses.

Speaker 3

It was awesome.

Speaker 1

That's funny.

Speaker 2

Yeah, this speaks to an interesting path for working with uh. You know you're a copilot of some kind to just be able to go fetch data from from sites through the m CP.

Speaker 3

Yeah, I mean it can do whatever you want it to do because it is access to the browser, which is what was missing when it came to the l MS. They couldn't open a browser. Now they come.

Speaker 1

Now they can open a browser and click, they.

Speaker 3

Can type anything that playwright can do. What can play?

Speaker 1

Right? Do everything. So this MP MCP is part of play right now?

Speaker 3

Is that it's just it's a play right MCP.

Speaker 1

It's open people, you get it off NPM.

Speaker 3

Yeah, it's you can just click a button and and it just installs it.

Speaker 1

I think we're done where this has been dot Net rocks time thinking the amount of trouble we're about to get it. Okay, let's trouble.

Speaker 3

Let's turn it around, let's turn around.

Speaker 1

How can we use these powers for good?

Speaker 3

Well, we can test our applications.

Speaker 1

The crazy espec.

Speaker 3

What we can actually get it to write the test for. So we're here at build and I've just given a couple of demos live at bill showing how you can using natural language tell it to go to a website, and I did a demo of a movie application. So go to a movie's application, search for a movie and ensure that that movie is on the page, and write the test for it. Now, the thing is, what's the

difference between asking COPA, let's just write a test. It can do it, and it guesses what the buttons are called unless it has access to the code, and it has able to then search to your code base, and then it can the MCP. No, the MCP actually go to the browser and it opens the browser page and then it takes a page snapshot, right, and the snapshot has all the context of what that page has on it.

Speaker 1

All. Right, we're really talking MCP is the protocol. You're really talking about an agent? Right?

Speaker 3

Well? Were you just using copilot here? You can use whatever you want.

Speaker 2

Yeah, this is just think of it as an apik PI to play right, to be able to fetch information the CP to play right, right, which from which.

Speaker 1

You can do you have to use an agent with this or can you just use it directly?

Speaker 3

You just go to copilot. I'm using copilot. You could use whatever you want, but just let's, you know, go to copilot exactly that that supports mcps.

Speaker 1

Yeah, that's important. That could be most of them, not all of them, most of.

Speaker 3

Them, yes, And then you basically ask it what to do. And in my case, I was like, I want you to write a test, but very clearly I said, I don't want you to search the code base. I want you to run the steps. Yes, I want you to use the MCP, and I want you to then only after you've run all the steps, then create the test.

Speaker 1

Five I don't have to test my website. Now I can test your website. You can, yeah, because I'm reading the page.

Speaker 3

You know, it's even better if you if you went to Amanda's demo, you can actually say, go to this URL and give me some ideas of what I can test.

Speaker 1

Wow, this is great.

Speaker 3

And then it came up and it said, oh, so you have filtering on the site that could be good for testing, and then it went ahead and clicked around the filtering and then it went and created that test.

Speaker 1

That's pretty awesome. So how specific do you have to be if you tell it you want to gather some like Okay, take my example right where you have pages and pages of people of companies that are in this particular organization, and they're all listed with their names and email addresses, but you have to press next and next and next and next. There's no search. Can it assume how to do that? Can it figure out how to do that?

Speaker 3

Assume it actually clicks and does it.

Speaker 1

But you don't have to tell it to do that, right, you know, it goes in. You could just say give me all the names and address of all the people.

Speaker 3

In them, and it will find the way to do it. So, for example, in my app, it couldn't type the movie name into the search because the search needs to be clicked on first. So when it went to type right, which was the logic of there's a search field all type yeah, it went, I can't type in here. It doesn't let me. Let me take a page snapshot. And then once I had the page snapshot, it said, oh, I see, I need to click the search icon right and then I can continue typing, and then it could continue.

Speaker 1

So it actually figured it out.

Speaker 3

By taking the page snapshot.

Speaker 1

By taking the page snapshot, Wow, that's interesting. I'm my mind. You're immediately going to evil. I hear it in your vote. Yeah, but you know that Harvey Korman from Blazing Saddles where he says something like my mind is a swirling vortex of infinite possibilities or whatever. It was some cheesy one. Yeah, that's where I'm going. It's I'm not actually thinking of evil. Although Dwayne and Patrick were here, Dwayne would immediately tell me what he would use it for evil.

Speaker 2

I'm just thinking about winning, you know, sweepstakes by filling in as many.

Speaker 1

Forms as you want. Well, hopefully the website has some sort of you know, anti denial of search.

Speaker 2

Well, you're claiming they're gonna find out pretty quick, aren't they.

Speaker 1

But but what you're saying is like you could just basically say any website that you can't figure out, you can instead of asking chat GPT, Let's say, hey, how do I use whatever Twitter to change my profile picture? Right? Because I don't want to click through and figure out how to do that. You just say here's a picture, change my Twitter profile picture and it could figure it out. Right.

Speaker 3

I would love to see that in action, but most likely us. I mean, you have to think about the login data. So you've got to decide are you going to log in first and then it's logged in and then you use the LM to continue the process. I would do that because you don't want to give your user credentials to the l.

Speaker 1

You're right, And this is this is something that we brought up with the other guests that we've had so far, which is security.

Speaker 3

Nothing is secure.

Speaker 1

CP isn't very secure right now.

Speaker 3

It depends. It depends on whose hands it's in it has access to. I mean, Carl, I think it's not secure with you, all right, tche.

Speaker 1

All right, Carl zero WI one. But but right now, and Scott Hunter told us this, right now, you know, you have to give it access to your keys in a local file or something like that. But they're working on it, Like by this time next year they'll have some sort of Azure entity where all the keys are stored there. Nobody knows the keys but Azure. And yeah, the MCP probably next.

Speaker 3

MCP is so new and everything is happening so fast that like even just the out protocol for the MCP is like not even a month old, right, So there's everything is just happening and everything is being improved. But for sure, when you're using it, be very careful of what you give to the LM, because you know, you don't want to say, here's my Twitter credentials, go in and and create my profile picture whenever, because you don't know then what happened after that.

Speaker 1

So you can obviously use the same browser instance that you're using regularly, exactly because.

Speaker 3

Of persistent browser context, you can actually just log in and then get the LM.

Speaker 1

To do some of it's beautiful.

Speaker 2

There may be a side by side play, but I know, I like just in your table example, so you're just watching it do a certain amount of work, and at some point you can step.

Speaker 1

In and do what you want to do with it. There was a special there was a story recently about an agent that was created and somebody said, find the cheapest price on ax on Amazon and actually found what it thought was the cheapest price and then ordered it. But it didn't the person didn't tell the agent to order it. Not only did it order it and charge them in an ordering amount of money. However, it wasn't the cheapest price.

Speaker 3

But that's silly because you let them go forward and gave it a credit card. I wouldn't give it my credit card.

Speaker 1

I wouldn't either, but they did. Ye.

Speaker 2

If you're going to let it onto Amazon where you've got.

Speaker 3

You know, oh you're already locked in And that's right.

Speaker 1

You're story. You've got your default credit card? Well you know you know you did it to yourself. Yes, yeah, So it's all in the prompt, isn't it? Careful what you play with, be careful what you tell your AI or not?

Speaker 3

But no, like, let's think positive. There's a lot of cool things that you can do it right, and buying tables is one of them. I kind of like to think of more of what do you spend most of your time playing around to try and find thinking.

Speaker 2

About thinking up good test for a web page. It's not a trivial problem exactly.

Speaker 3

Really, it is you don't know what to test, you don't know where to start.

Speaker 2

Yeah, just getting that list of what should I test on.

Speaker 3

This page and you can do it. I did a couple of demos and you can do like just turn around and say, hey, go to this website and give me a markdown file of those, right, so then you now you go to a marked all file would say maybe seven tests, right with clear English instructions, and then you can decide, right, I'm going to allow use this again and again, so you don't have to just go

and use the MCP to do everything. Now done, you can separate it out how then, right, these are now my marked one files and then you can use I don't know, you can create you can do you know, there's a get up MCP, So you get the get up MCP to create an issue for it, sure, and then continue to create the test and then the get up MCP will create the test after the play right MCP has done it, and you can just MCP all day it's.

Speaker 1

It's clear that MCP is the future and it is the glue. It is going to be the soap and or the JSON or the rest. Like it's going to become the default protocol of AI agents talking to each other. So I'm I'm very excited about this, and I only learned this today.

Speaker 3

Yeah, well a lot of people have only learned it today, and I think people feel like they're behind and it's actually not. It's just so new and it's so new, but everyone's talking about mcps and it is just very cool.

Speaker 1

But the anthropic blog post is from.

Speaker 3

November exactly, and that's the first.

Speaker 2

I don't even know they'd implement at that point. It's just that is the blog post.

Speaker 1

Yes, so it's clear that with playwright testing with the MCP is the obvious choice. Because who likes to write tests?

Speaker 3

Nobody, as much as we want them all, do you. I like to press the green button and see it running and it's playing.

Speaker 1

Now.

Speaker 3

I don't mind writing tests, but there's too much to do that. I want to actually do more creative stuff, so I want my app to work. I know we're all vibe coding, right.

Speaker 1

I don't know about you, but I'm not vibe coding.

Speaker 3

Oh Carl, you're missing out. No, no, no, you're missing out. It's so much fun. So you're just vibe coding away and you're like, yeah, that looks good, looks.

Speaker 1

I can't let this go. Vibe coding is a pejorative. That's that's like when some of you doesn't know what you're doing, talks to an AI. No, no, no, Now, I use AI to help me write things.

Speaker 3

Maybe your vibe coding and you don't know it.

Speaker 1

Well, maybe our definition of vibe coding is different.

Speaker 3

I'm just a fun person. I like the vibes. So I'm vibe coding along and I'm like, you know, I'm doing stuff faster and I'm getting the agent to kind of do it, and I'm going with the vibe. I'm like, yeah, this looks good. Yeah, keep going, keep going, keep going. Now, got this application. But I really have not written the majority of the code, and that is their new reality. A lot of people out there are not writing all the code themselves. Therefore, can they trust the code that

they've written that they haven't even written? And that's where testing is really.

Speaker 2

Was just talking about this idea of do an experiment where you write them. You write a prompt for code and don't check the code, just run it and keep iterating and asking more questions so you get to a place. It's for experimentation. He wasn't aiming it at the world. He was aiming at developers.

Speaker 1

Yeah, and he was aiming it at a way to experiment with a new tool or a new environment and a new idea. But it has become a pejorative to about well, people who don't know what they're doing. People indevitably abuse it. Yeah, right, but how many times over the decades has your prototype been shipped? Well, you're asking me, this.

Speaker 2

Is what I'm saying, like, and what are you doing when you're doing vibe coding? But you're playing with prototype. Prototypes never shipped, but certainly they went on to become reading implications. And thereon lies this alien point, which is because it's a relatively painless, like afternoon experiment, you don't mind chucking it out either, but it changes your thinking. You now have seen some ideas and seen them things, and you might come at it again from a different angle.

So I just wish it hadn't been soiled, you know. Yeah, And we need a better term car party worked really hard to make this idea come true. Like he tried a few different names he's I think at one point he was talking about I'm coding in English like, but it was vibe coding was the one that memed up and took off.

Speaker 1

Yeah sounds cool, So it does, but it's been soiled. Well I think if it's abuse, it's.

Speaker 3

Not going to go away. No, all right, stop vibing if.

Speaker 1

You say so, Debbie.

Speaker 2

Yeah, but just don't do exactly what he said they the responsibility element still exists.

Speaker 1

Yeah, all right, And with that, we're going to take a little break and we'll be back after these very important messages don't go away. Did you know that you can work with AWS directly from your ide AWS provides toolkits for visual Studio, visual Studio code, and jet Brains rider Learn more at AWS dot Amazon dot com, slash net, slash tools. And we're back. It's Dot in a Rocks.

I'm Carl Franklin. That's my friend Richard Campbell, and that's our friend w O'Brien, and she's an advocate for Playwright, the testing framework for dot Net for web applications, and we're talking about how playwright has embraced AI and now you can use a MCP. Did you work on the MCP? I automated, worked.

Speaker 3

On the actual advocacy for it and ensuring it happened in the sense I wanted this. I was pushing them. They're like, you know this to be a thing. Yeah, because it was so early days. It was like the first one I did was the Blender one. I played around the Blender MCP and I just I had made a leprechaun at a Blend.

Speaker 1

This is not cliche at all to me. I can't imagine, Like, what do.

Speaker 3

You mean thinking I needed to go back to the team and prove that I did something so nobody else.

Speaker 1

Is going to do. You did an MCP for Blender. Does that mean you have mcps? Are there mcps for say CSS out there in general? No?

Speaker 3

No, no, But for a Blenders a tool, and I don't know how to use the tool. I don't know how to use Blender, but I was able to. I was able to blend a leprechorn together by using the MCP, and that's the power of the MCP. But you don't have to have the knowledge to be able to use things. So then I played around with a load of stuff and you know, brought that back to the team and kind of said, hey, guys, I don't know how.

Speaker 1

To use this tool, but I was able to get this tool to do this white.

Speaker 3

Yeah, why don't we have a Playwright one?

Speaker 2

Back to you know, you put on your product, manage your hat, and it's like, I want more people to use my product. Why wouldn't I make it easier for more people to use my product?

Speaker 1

Yes?

Speaker 3

Well, also there were other third party mcps by Playwright wow right, which we're doing really good stuff, and I was like pressed by what they had done, and why do we have an official one? Because we can then tailor it to what people's needs are. But you know a lot of clients were using the third party ones and asking us do we not have an official one? So all this was like, you know, pressure.

Speaker 1

So how did those third party repo managers feel about Microsoft doing an official one?

Speaker 3

I don't know. I think because it's open source, we were able to build stuff, they were able to learn from that, we learned from them.

Speaker 1

I think it's like, you know, the way everybody's great, you don't make these things for fun. You make it because you're trying to get something done. You just took a bunch of work off their play.

Speaker 3

We're improving things.

Speaker 2

Yeah, that's and he probably too accused from them because they'd already come.

Speaker 1

Up with something.

Speaker 3

They've done some amazing stuff.

Speaker 1

Yeah.

Speaker 2

And in the end, it's better for there to be one comprehensive one that does every everything that everybody wants.

Speaker 3

I think as well, people want to trust that this yes trustworthy.

Speaker 1

You guys aren't planting malware on my machine exactly, I'm pretty sure. I think in general, I think in general, Microsoft is kind of late to the agent game, and I think that they're finally coming around to it. Why they relate probably because you know, things are changing so fast, But I really want them to succeed at this. I

don't think anything within a year is late of anything. No, no, no, But they were there as certainly other things popping up in the last couple of years, a year and a half that you know, people are glombing onto and especially people in FVNX. You know they're early adopters, but they're they're all in agreement that they want to see Microsoft kick this thing right. We want them to be the dominant force. Yeah, I don't know.

Speaker 2

I like how open everything has been so far. Yes, you know, arguably except for weights. But for the most part the weights that people are protecting don't seem to be all that valuable anyway.

Speaker 1

Can you explain what you mean by that?

Speaker 2

Yeah, So when I say weights, well, the values inside of the neural net that make it perform the way it does. So, you know actor value that they talk about CLAW being open source. Sure, the code's open source, but the cloud that you're using actually has a training set in it, and those sets of values, those weights

are in theory, that's the asset. Those aren't public, and so you don't you're not literally sitting on the set of you're sitting on the code where you can do your own training set to make a cloud like thing which you don't have their training set, and you don't have the results of that.

Speaker 1

It's about the training data.

Speaker 2

And one of the ways it was explained to me the other day that really amused me is that these ms, these neural nets, they are the jpegging of data. They are a lossy compression format for a large quantity of data. Right cool, you tokenize it, you turn it into a set of symbols that you can then compress into a model that then you can pull images from. But it's a lossy format, so you're trying to lose the things

that aren't important and hope for the best. But and it's opaque, like the only thing you can't do from a JPEG is get the original image. The loss is the law us unlike FLACK, which is an audio format, and if you want a flack of the Internet, use the Internet or ZIP is a better example, right, Yeah, yeah, a non lossy formula on this. So I mean, we're playing in this space where we have this language model and now we're figuring. You know, in some ways the

science is over. This is all engineering, and the engineering we're doing is how do we couple this to things that are valuable. And that's where MCP just emerges and

grabs on so quickly. You know what it reminds me of typescript, Yeah, because when typescript templates like typescript was an interesting idea until everybody made a template for everything for it, right, And what you're seeing with MCPS is same effect that all of a sudden, all of these tools that are especially part time tools, like tools you don't use all the time, You're never going to be good at that. Suddenly you gave them a prompt interface and it makes it a lot easier to do those.

Speaker 1

So how long has the Playwright MCP been available? Minutes?

Speaker 3

No months? Which is an entirety in AI. I think we were about January February when we released it and that time.

Speaker 1

Have you seen people using it for? Interesting ideas? Can you give us some of those things that people used it for?

Speaker 3

No, I don't think I'm looking on I'm looking on tables when I use it. No, obviously we do. We have a lot of people using and for testing, especially inside Microsoft. That's what it should be for, and that's what should be for, and there are other people using it for other things. There are many things you can use it for. But yeah, it's a good question. I'd love to hear from. We are the number one playwright or the number one MCP server. So, no kidding, It

is insane because of the capabilities it can do. And it's not just it's not just about testing, it's about browser automation.

Speaker 1

MCP to the Web. What could be better? Yeah? Yeah, really, you've got ten thousand and stars already, ten thousand stars. Heypesky website doesn't have an MPI and API. Don't worry, don't worry.

Speaker 3

We've got it cover and we don't even have a documentation page website. We have literally just to get up thirty. It's crazy.

Speaker 1

Wow, that's great, It's madness, it's awesome.

Speaker 3

I put a video out on Thursday and has twenty eight kve yews.

Speaker 1

That holy crap.

Speaker 2

No, no, it's this is a very very powerful one and a bunch of people know it like they've figured it out.

Speaker 3

What I'm wondering is, you know, you probably heard the keynote from Satya and we've got Windows as an MVP. Yes, how could we leverage playwright into the Windows MCP?

Speaker 1

Okay, this show's the first check in is March fifteenth, twenty twenty five. This is two months old?

Speaker 3

Is that all I told you? Months?

Speaker 1

Two months? All right? So here's the thing. Ten thousand stars, two months.

Speaker 3

I think we got the first five k in like three days or something. Literally, yeah, I literally slept and then I wake up and it was like another two k stars. It was insane.

Speaker 1

Is this going to be the new phone app? In other words, is MCP is going to be Starbucks going to have a Starbucks MCP where you can just to order a coffee. To order a coffee, it's McDonald It's going to have a McDonald's instead of going to go to the drive through? Are using the mobile app? Are they going to say, are using the MCP.

Speaker 2

I don't think they're going to ask. I think what's going to happen is you're going to be able to You're going to ask you a perplexity from your phone to order from Starbucks, and it's going to tap the m CP. Get here's a wonderful.

Speaker 1

Thing about Starbucks. You go up, they say may I help, and you say yes, I want X, and they give it to you as opposed to you drive it to Starbucks.

Speaker 2

Go, my name is Carl. Here's your bag. Yeah, it's already done, but I don't want it already done. I want it done fresh.

Speaker 1

While I wait, No, it was done. Adget I'm the I'm the the Luddite tear and we've done fresh. On your way there. Richard's the guy who wants Wi Fi in your refrigerator because he wants your refrigerator to phone the grocery store in order milk.

Speaker 3

I want, I want that done too.

Speaker 1

I think. I think that's what And the answer is the ultrasonic sensor in the cape down into the bottle that much milk is licked and break. That's how it works. It's really brilliant. You know what's awesome going to the grocery store with a list and shopping. I have seeing what's fresh. I have stuff to do, but I have to be clear shopping.

Speaker 3

Try shopping shopping trolley.

Speaker 1

Twins not a thing, all right. I want things to be easy.

Speaker 3

I want things to be fast.

Speaker 1

Why, by the way, I'm not a lute I am. I am like very technological. If you understand, if you listen to Dot it Rocks, you know I do so much with technologist day. You're just trying to be a contrary. I am trying to be a contrary. It isn't working, but you're trying. Okay.

Speaker 2

My my family in New Zealand owns a dairy farm. So I was just down there my cousin and we had to go shopping, yes, and we had and we had to buy milk. Oh no, And I'm like, I find it hilarious that you're buying milk. She goes, Oh, you don't even know the half of it. We're in the one half of one percent of the best milk produced in New Zealand, and my milk is in this bottle somewhere, mixed in with everybody else's back because I can't buy my milk. Listen to the big country by

talking heads. It's like on the plane and he's looking down, he's look at that kitchen and all that food. They go to the grocery store and they put it in their car trunks and they bring it back home, bring it to the point.

Speaker 1

I don't know, it's crazy, It is crazy, but you know that's what happens, all right.

Speaker 2

Well, I mean with the interesting truth about the playwright. MCP is not just I can do better testing on my website with less effort, without a doubt, but also this is a remarkable tool for fetching information off the web. Yes, full stop web scraping with AI. Yeah, except you're not even thinking about scrape anymore. You're just getting information exactly.

Speaker 3

I think webscraping sounds makes it sound is a derogatory. Yeah, it's about giving the LLM the power to open a browser and do whatever you need to do.

Speaker 2

Yeah, and move from there, and that play right, but then save.

Speaker 3

It in its memory and then do something with whatever that is, be it read a buck to you or create a test for it.

Speaker 1

So so you can tell chat gpt, you know, you can ask it to you know. What is the is when I ask chat ept about a problem that I'm having and it says, well, I don't know, you know, or here's something that doesn't work. I say, is anybody else on the internet having this problem? This is a great prompt for chat ChiPT and it will go search and sometimes it will find yes. On stack overflow, I found this. This particular user said do this. This particular

user said do that. You wouldn't necessarily have to do that anymore. Like, is chat gpt using something like that to scrape websites? Using Google searches and then looking at the websites? And it's yeah, of course it's different. It doesn't have play right.

Speaker 3

It's not opening the browser.

Speaker 1

It's using an HDP client kind of thing. Yeah, it's interesting.

Speaker 2

I just like that pattern of what tests should be written for this web page? Why don't you write those tests for me?

Speaker 1

Yes, it's great.

Speaker 3

Continue to keep going, keep going.

Speaker 1

It's like, why are you waiting? Keep going.

Speaker 3

It's very powerful. We're only in the early days. Like literally you mons, I know, it's so cool and there's so much more, and that we have some really cool ideas that we want to try and work with to improve the whole testing scenario in the MCP world. Improve it, make it better, give it access to more things. What can we what else can we let it have?

Speaker 2

Yeah, well you hit on the point of now you plug the GitHub MCP into this as well, and so now you're pushing issues for tests to fail, right, instead of fatting around with anything else. Just to automate that whole problem, that entire workflow.

Speaker 3

Just talk to the LM, the whole timelilet this.

Speaker 2

Feed that engine over there, because you know the other remember when get hubs you sort of claim to fame its the scriptor was social coding, right, it was the social media for coding.

Speaker 1

This flow of.

Speaker 2

Creating issues and then resulting in pull requests and so forth is about a documentation chain of how an app evolved. The fact that we're going to insert these tools into the process, they should contribute the documentation.

Speaker 1

They write better.

Speaker 3

Documentation, and we do anyway better English than my house.

Speaker 2

Right and thorough the number of times now that I've turned to get hub Copilot after tinkering with some co for a while and said, what have I changed? And it's batback stuff I've completely forgotten about already. It's like, here are all the changes you did, so, I mean it just speaks to all of those you know, those things singing together means we have a far better picture of how software can be evolved and and different tools

to evolve it. Because of course this show and we've now done podcasts leading into our conversation about this agenic AI role helping us write code.

Speaker 3

It's further reality. Yeah, it's not even the future, it's yeah, it's reality.

Speaker 1

Right, it's right now, right now. Yeah. I guess your next question is like, how would you make this web page better? How would you make it faster? I gotta tell you something I don't know about you, Richard, but I'm getting.

Speaker 3

Vibes you see, you see your vibe coding already.

Speaker 1

I'm getting vibes like when we did a show with Miguel da Casa after Microsoft bought them about Zamoran, about Zamorin, and you realize the future and how it was going to change, and it turned into done at Core. Right, Core was already well underway. Bye, But I mean it really, I mean they got it. They got it really rolling you think about we signed to Miguel in twenty eleven, yeah, right, which is when Andrews had left to see sh our team was going to typescript.

Speaker 2

Yeah, and like we were worried about about C sharp, we were. We didn't know what was typescript. We just knew he left and suddenly the that Miguel came and talked to us. And I think I said on the show at that time, I think the keeper of C sharp doesn't work for Microsoft. It's Miguel dia Cap absolutely, because he was pretting C sharp into iPhone and Android, which was craziness, right, and that completely I think Miguelda

Causes saved. Yeah, I wonder if that's true. I think it was gonna be saved anyway, but he certainly proved it had more potential than.

Speaker 1

Anybody had thought of yeah or believed it. Yeah. And without his work on Mono and everything, it would have taken a lot longer to dot necor what it is today. So I'm getting the same vibe right now with you, Debbie, with playwright.

Speaker 3

You all the world.

Speaker 1

You are the new media because MCP and MCP. I'm sorry, but an AI interface to the web generally is full stop, full stop the future. I'm I mean, you can have an MCP to GitHub into Azure and all that stuff, and that's all great for your resources, but for the web click type the bit of stuff on the web. I don't know if you've noticed, there's some stuff. There's some stuff. I think some of it's horrible. Some of

it is horrible. You should not click Google there if you get a link in an email that says, hey click this, don't click it, don't do that, don't do that. But I think this is one of those times right now today where we will look back and I think we're probably going to be doing this in twenty years. Oh good lord, I think so. I think, all right, ten years from now, we're going.

Speaker 2

To years you know what, two years that we're talking about show three thousands, right, ten years from now, we're going to look back and see.

Speaker 1

Remember that show we did with w O'Brian on playwright that changed the world. I'm literally thinking that this is that moment.

Speaker 3

I always want to change the world.

Speaker 2

Wall blame you, Yeah, well, it's all your fault. Supposed to be a nice little show.

Speaker 1

What have you done? Look, we've all been around for a long time. We can recognize big change when we see it. And this is big.

Speaker 3

Now, this is big.

Speaker 2

You're really running out of excuses not to write tests, really really running out of it.

Speaker 1

Yes, the test.

Speaker 3

Yeah, they're literally saying like, we've got no time, we've got no money, we've.

Speaker 2

Got no this that's okay, this is free. And all you can do is write a prompt and then say yes.

Speaker 3

And you can write a reusable prompt that you can save into your get a prompts folder. That's right, that therefore you and after the work once.

Speaker 1

Prompt libraries, yeah, Hunter told me about this at lunch. Prompt libraries, like you have code libraries. Now there's prompt libraries.

Speaker 2

Yeah, obviously, right, a well crafted prompt that will really have that will walk through all the steps of writing out an entire test plan.

Speaker 1

For our new website. Forget about substacks, subscribe to my prompt library, right, anything else you want to talk about before we end this world changing?

Speaker 2

Everything's done. Now remember your life before.

Speaker 3

No, it's all mtps, I would say, I would say, still people are still testing, and that's also fine that this is not taking over.

Speaker 2

Oh no, this show is called changing testing with play right MCP Okay, that's what this is called.

Speaker 3

Okay, But you can still write test if you want to.

Speaker 1

You can write that never went away. But I'm what I'm doing is taking weeks not to write test exactly.

Speaker 3

So still write your test still, you know, when you're developing. But if you have a website out there, you're a URL. You just want help. You don't have an idea what a test. Let the AI do it for you.

Speaker 2

Let it guide you down the path. I mean, in the end, you're still going to agree with what it did.

Speaker 1

But it's got good ideas, definitely good ideas.

Speaker 2

Which is mostly just scraped from the internet.

Speaker 3

Anyway, right exactly someone else has done not.

Speaker 2

Thinking any originally. But you're basically you've now found a mechanism to tap the best thinking and testing to give you ideas on how to test your site thoroughly.

Speaker 3

But it's a little bit different because you were asking it to give ideas based on someone else's ideas. This is actually the MDP going to the website and saying, oh look, I see a button here. I'm going to click it, right. So it's kind of like what would a user do when it first sees a website. It starts picking buttons, and that's what this is doing. It starts clicking buttons and then seeing, Oh I see this is a filter existance. Oh I see this is this because it has access to that page not shut.

Speaker 1

So, Debbie, how long have you been in town?

Speaker 3

Two days?

Speaker 1

Maybe? Have you been to the Elephant in Castle? Irish pub? A British pub? No, it's good if you're if you're hankering for a pub, good goods and chips.

Speaker 2

Yeah you don't, hankers are here.

Speaker 1

Yeah, you brought the fam. Yeah, like cold and wet.

Speaker 3

It's my garden, that's true. How old are they seventeen months?

Speaker 1

Oh? My goods, they're not eating fish and chips yet.

Speaker 3

Oh they're reading everything.

Speaker 1

Oh yeah, Well give them some mushy peas. He'll be happy. Yeah, okay, it's fantastic. All right, Well, this is a life changing moment for me and I hope I hope everybody else too.

Speaker 2

So if you're excited because cool things are happening.

Speaker 1

We are very excited, and we'll talk to you next time. Dear listener. Dot net rocks. Dot net rocks is brought to you by Franklin's Net and produced by Pop Studios, a full service audio, video and post production facility located physically in New London, Connecticut, and of course in the cloud online at pwop dot com. Visit our website at d O T N E t R O c k S dot com for RSS feeds, downloads, mobile apps, comments, and access to the full archives going back to show

number one, recorded in September two thousand and two. And make sure you check out our sponsors. They keep us in business. Now go write some code. See you next time. You got Javans Then of Texas a reading h

Transcript source: Provided by creator in RSS feed: download file
For the best experience, listen in Metacast app for iOS or Android