Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 282, recorded May 3rd, 2022. I'm Michael Kennedy. And I am Brian Okken. It's great to have you here, Brian. It's just us, just the two of us. Yeah, just like old times. I know, but we have our friends out in the audience, so we're not entirely alone. It's great. So let's kick it off. I know you have a particularly exciting announcement.
It's a very important topic to cover here. So let's go do it. Okay, so PyScript. So this was an announcement at PyCon US by Anaconda's CEO, Peter Wang, during a keynote. I wasn't there, but everybody was tweeting about it. So it almost felt like I was there. But I haven't seen the presentation, so I can't wait till that goes online. I know. Are the videos, I have not seen the videos for the presentations at PyCon out yet. Are they out yet? And I just missed it? I haven't looked.
Is my YouTube broken? It should be full of this stuff. Like, what's up with, is it supposed to be next day or something? I don't know. I know, I know. Anyway. I would have loved to live stream it, but I didn't see an option. So anyway, I'm looking forward to watching this one in particular when it comes out, because this is big news. So PyScript is Python in the browser. So what does that mean? It is built on top of Pyodide, which is a port of CPython based on WebAssembly.
I'm pretty sure we've covered Pyodide before. So this is a pretty neat thing. And one of the things that this, so the PyScript.net, you go to it, it's got a little, it's kind of actually, it's like hype and it sounds neat. And you can do Python in the browser. Neat with the PyScript tags. But what does that mean? So there's a, if you go down to the bottom, there's a GitHub repo that you can go look at. This is what I suggest. And this will talk about, there's a getting started guide.
But what I did is just followed this. I cloned the repo and then I went in and did the, into the JavaScript area and then did NPM install and then did this dev run, run dev thing. So this only take me like five minutes to get this far. And, and what you have is you've got one of the things that it has is it has an examples folder and you can just open this up now in your local, your local browse, local host. And there's all these cool demos.
Like there's a, a REPL where you can just do, it's kind of like a Jupyter where you can say like X equals three. Let's do this. And then X. And then if I do shift enter, it evaluates it. How neat is that? That's pretty neat. That's awesome. Yeah. To do app here. So make sure you listen to our podcasts, go buy Python testing with pytest. We'll check that. Cause we know you already bought that. So, and then there's an example with D3 graphics. This is neat. I don't think I've ever done this.
There's an Altair example. And this is pretty fun. Cause you click around and it changes the above. It's like an interactive thing. This is fun. I, we, we use Altair with a project at work. So this is neat. The Mandelbrot set. So there's some code. So all of this code is in the repo. So you can look at the examples and look exactly how the code is done. There's a HTML file and a Python file for all of these. So you can check it out. Actually, I don't know about the Python thing.
It's a, it's, it's HTML and Python within the HTML code embedded. So there isn't a separate file, but you have, you can do imports and all this sort of stuff too. Um, Oh, I went too far, but I wanted to bring up, there's also an article that we're going to link to in the show notes that is called, PyScript, unleash the power of Python in your browser. Um, this is by Eric Lewis Lewinson and, it runs through it's, it's a pretty interesting, a little quick read of what it is.
If you're not familiar with, WebAssembly and pyodide. So it's nice. What do you think, Michael? So excited. I am very excited. You know, there's been progress on the WebAssembly plus Python side on several occurrences that were, they give you a sense of what's possible, but they didn't give you a thing to build with. You know what I mean?
Yeah. So for example, pyodide is awesome, but it's kind of like, well, if I want to sort of host a Jupyter kernel in my browser, like I can, I can kind of do that. Right. Um, the WebAssembly Python itself is great, but it doesn't specify a way to have a UI of your webpage interact with Python. It's just, oh, you could execute Python over here. Well, like, and then what, you know what I mean?
Which is, which is still good, but there's not something where like, I can have a button on there that like wires up to this thing in Python. And I can have this list that binds in that way and so on. And this looks like we might be there.
Like one of the things they talk about on the page is not just running Python in the browser and the Python ecosystem, as you pointed out, but really importantly, two more things, Python with JavaScript, bi-directional communication between Python and JavaScript objects. Yeah. So you can wire into like events on the page and other, DOM type of things.
Yes. And then a visual application development ties in with that with, use readily available curated UI components, such as buttons, containers, text boxes, and more. Oh yeah. Yeah. Yeah. I mean, like these are just a little quick examples, but I'd love to see some, some, uh, bigger examples of things like that. Like, being able to connect, it, you know, yeah. JavaScript interaction with, stuff on, on the Python side. That'll be neat.
Yeah. It's weird to see Python written just straight in the browser, you know? Yeah. Like here you have like angle bracket, pi dash script, and then just import anti-gravity, anti-gravity dot fly. Like, wait, what? Well, so this, this is a good example. I, I picked this example for one is because it does, it does do an import. So this, it, there's like a path thing you see, you set up. So you can put code, you can put code, all your code doesn't have to be in HTML.
It can be in, in a Python file. So you can debug it there, which that's where you want to debug it. And then you can import it and call it within Python. And so this is probably more where I would use it is, putting most of my code somewhere else. And then. Yeah. That's what I want to see. I would want to see just Python files and just, effectively a script tag for it.
I mean, you probably, maybe you can't do it directly as a script tag, but you could do, you know, bracket, high script, and then just import and run. Right. Yeah. So the point, basically. I haven't looked at this before. So the anti-gravity.py that is, bringing in is bringing in some pyodide stuff and, uh, to be able to work it. So I'm seeing some from, doc, this is Python code from document or sorry, from JS import document. Yeah. And set interval. And so those are the things you do there.
Uh, let's see. Are there any, any callbacks? I don't see any callbacks there. Oh yeah. Yeah. This set interval as a callback self.move when the interval, the JavaScript interval fires. So under, under fly, that is, hooking into a timer there. Yeah. Timer callback. So we should check that out. So where's, where's that? Um, so the, I should have done this ahead of time. The anti-gravity is not linked to, but I'll just like bring it up. Anti-gravity based on. Wow. Oh my gosh. This is so amazing.
People have to do this. Oh, this is cool. We all know import anti-gravity and we've got to know the XKCD that comes up, but yes, this is so good. It's great. It's alive. It's not just, is the person who, who says, how are you flying? The person says, I'm playing with Python. Like that thing is alive and cruising around. I love it. Yeah. And that's based on the callback, right? That's, that's calling Python based on the set interval, timer callback in JavaScript.
Yep. Yeah. And, and to me, that has been the missing piece. Like how do I wire up? It's like great if I can just execute Python and have, you know, like a number of things. But what I want is view and Python or reactive. I want to build the UI and Python and just not deal with JavaScript and be able to do so many more things on the front end. I mean, this opens up stuff like, progressive web apps, which could be really amazing for the Python space. Right? Like I'm here in Vivaldi.
If I go to my email client, just in the browser, I can right click and install. It gets its own app that works offline. It like pull this data down into local DB or whatever. Theoretically you could do this, right? You could pull down the CPython WASM. You can pull down the 5k I script file and then just somehow use JavaScript to Python to talk to local DBs. I mean, what if we get like ORMs in Python going, oh yeah, we have one of our backends is the web browser, local DB.
Yeah. Or something that would mean, this is great. I would love, I'm very excited for where this might go. Sky's the limit, right? That's what that little flying character is saying at least. Yeah. Okay. So, well, good job Anaconda folks. And I believe this was Fabio and crew. So really, really nice. That was super psyched. How am I going to follow that one up, right? I mean, come on. It's just, I'll, I'll give it a try. No, I've got some good items. They're just not flying around.
Amazing Python in the browser. Amazing. So Bloomberg has a lot of Python going on and Bloomberg actually has a pretty cool, like tech engineering blog where they talk about some of the stuff going on at Bloomberg, right? Yeah. One of the really good articles I read from this, from them was about how to really set up and run micro whiskey in production. And it was like this huge, long, deep list of like, here's a bunch of flags you probably never thought about.
And here's why you should care about them in Python. Really good stuff. So they're back with another thing that they use that is cool called memory, like memory, but memory. It is a memory profile for Python. So if you want to understand the performance of your application, especially around memory, here's a pretty neat tool. Now, let me just get that right out of the way before I forget. Linux only. So if you're not using Linux, just close your ears. No, just kidding.
Like you could all, if you're on Windows, you could just run your Python app under WSL and then profile it and then go back to running on Windows. Or if you're on Mac, just do a VM or something, right? Anyway, it only runs on Linux. But because Python is so similar across the platforms, I'm sure you could just test your code there, even if that's not the main use case. All right. So you get all these different visualizations of memory usage.
It can track allocations for Python code in native extension modules, like NumPy or something like that. And even within CPython itself. So you get sort of a holistic view of the memory, which is pretty awesome. Yeah. And it'll give you different memory reports. We'll talk about them a little bit. And you can use it as a CLI tool, just like kind of like time it or whatever. You can just say memory run my app. And then when your app exits, it's like, and here's what happened.
One of the things that's super challenging about complicated applications and web apps and stuff is you want to focus on a particular scenario. And there's so much overhead of like startup and other things. So for example, if I just want to profile a FastAPI API call, if I just say run it up and then I go hit that API, all of the infrastructure starting up UVicorn and FastAPI and Python, it just like, it just dwarfs whatever that little thing is usually.
So there's also a programmable API that says, you know, you could create like a context manager. Like, I don't know if it actually is that way, but you could certainly build it if it doesn't exist. Like with memory profile here and just do a little block of code and then get an answer, which I think is pretty neat. Alvaro asks if it accepts an entry point. I suspect you could call an entry point because you just do the run on the command prompt. So you could probably pass it over.
Whatever you run. Yeah. Yeah, exactly. But the problem is there's still like the startup of just CPython itself, right? Like I always find just the imports and all that is just way more overhead than, you know, it clutters it up. Anyway, let's hit some notable features of Memray. It traces every function call as opposed to sampling it. So instead of just going every millisecond, what are you doing now? What are you doing now? Let's just record that, right?
It actually exactly traces so you don't miss any functions being called, even if they're brief. It handles native calls in C++ libraries. So the entire stack is represented in the results, which is pretty cool. That's pretty neat. Yeah, that's pretty dope. Apparently it's blazing fast. There's some kind of character. I think it's a race car there. It causes minimal slowdown in the app if you're doing Python tracing.
If you do the native code stuff, it's a little bit slower, it says, but that's optional. You get a bunch of reports. We'll see those in a minute. It works on Python threads. So you can see, I know all these people watching, but you check out the webpage. There's a little thread, like a sewing thread emoji. Or a Twitter thread. Yeah, dude. So it also works on native threads, like C++ threads and native extensions, which it represents as an alien plus the thread icon. I love it.
Alien threads, yeah. Yeah, yeah, yeah. So let's look over here real quick. We'll look at just, I guess, the reporting, right? I mean, the running is super simple, as I said. Memory run Python file with arguments. Or memory run dash M module with arguments. These are the places you could put your entry point and so on. And Dean, in the audience, says we've had a rich spotting. Okay? I haven't pulled that up yet, but very nice. So there's different ways in which you can view it.
And the first one that I ran across, which is pretty interesting, if you're familiar with glances or you want to go old school, like top, or one of these things you can run in just the terminal and get, like, not really with rich, not rich, not rich with top, but rich output like glances, is you can run it in a live mode where, while it's running, it'll show you what's happening with the memory. That is so awesome. That's pretty cool.
Yeah. Yeah. Yeah. So instead of just showing you a memory graph, it's like, guess what? We're running here right now with this many allocations and so on. Yeah. Like that looks super neat. Yeah. And if you've got something interactive, you can interact with it and watch the memory change then. Yeah. Yeah. You can cycle through threads.
You can sort by total memory or its own memory of the, that's a common thing you do in profiling like this and all of the stuff it's called, or just this method itself, sort by allocations versus memory usages, all kinds of stuff. So that's really neat. It will track the allocations across forks, as in process, sub process. Oh, okay. Why would you care? Because multi-processing. If you want to track some kind of multi-processing memory workflow, it'll actually do that.
Just you do dash, dash, follow fork, and it'll like aggregate the stats across the different processes. Kind of insane. Let's see if we can get down here. You can do, they have the summary reporter, which is kind of a nice, just, you know, this is probably what you would expect. Flame graphs. If I can get down here somewhere, it'll show like sort of the color and the width of these bars. It'll show you how significant it is.
there's a nice tree version that'll show you the biggest 10 allocations. And then a call stack sort of in and out with trees and like how much memory is being allocated in each one of those and so on. That's nice. Yeah. This is a nice app, right? Nice utility. Definitely. Cool. Yeah. Indeed. Indeed. Indeed. So if you want to track down memory leaks or you're just wondering like, why is my program using so much memory? Fire it up. Let it run for a while. See what happens. Yeah. Cool. All right.
Back to you, Brian. Well, I want to bring up a, a pie test tool. So, um, it was a, I, I have a recent, I've often used a pie test X dist, uh, for parallel. So X dist is a way you can just say that it's, it's, it's the one that I heard about first for running pie tests in parallel. So you've got, you know, like tons of tons of unit tests, maybe, and you want to just speed them up. Um, you can throw them, throw a dash in for something like that at it.
and it'll, uh, and it'll, it'll just throw them, uh, and, um, and there was, uh, I think it was Bruno Olivier, um, suggested a couple of alternatives. And one of them was pie test parallel, which, um, I, I, I know I've run across, but I haven't played with it for a while. So I tried it out and it's actually like really cool. So one of the, one of the pie test X does does a lot.
One of the things it does is it not just, uh, uh, it's not just multi-processor, but it can be on different actual different computers. So you can launch them on. Oh, nice. Like grid computing almost. Yeah. You can SSH into different systems and have it run in parallel. Um, but that, you know, you don't, I don't usually need that kind of power. Um, the one thing it doesn't do is thread. So it's, it's process based and pie test parallel does both.
So you can say, um, you can give it, uh, you can give it, uh, well, where we have, I'm going to go down to the examples. So you can give it number of workers and it'll tell it to, um, that's how many, uh, processes it'll spin up or how many CPUs. Um, now you can also give it test per worker and then it'll run in multi-threading mode. Um, and you can give it auto on both of these. And it's, uh, this is extremely useful for, you have to, by default, this is turned off by default.
The, the, the features, if you just say workers equals five or something, it won't do multiple threat, multi-threading. And the reason is it, because you need to make sure your tests are thread safe. Um, and many are not. So I tried it on a couple of my, even if they're isolated, they might not be thread safe, right? Yes. Um, that's, that's another level of consideration.
However, if there are, there's a lot of small, especially small, uh, not really unit like system tests, but a lot of unit tests are just testing a little Python code. If you've got a part of that is a lot of projects, that's a big chunk of the test load.
So being able to do multi-threading is really nice, but you know, even with just multi-processing, I tried this on a few different projects and there were like, I tried it on flask and the, um, uh, the, the parallel version using pytest parallel was like three times faster than the ex-dist version. So, um, so based on your, I, there's, there was another one that, uh, Bruno mentioned, but I think these two are really solid ex-dist and parallel.
So if you want to speed up your test run times, I would try both on your project and just see, play with them. And see, see which one's faster on, uh, many of the projects I tried parallel was at least as fast or faster than ex-dist. So it's kind of nice. Yeah, it's cool. This looks great. I like it. And having your test run faster is always good. Do you do anything crazy? Like, do you set up your editor to auto run tests on file change or anything like that?
Um, sometimes, uh, one of the things that I've always, I've done it a few times, but it always makes me nervous. I'm like, ah, just like it's unnerving to me that it just keeps running. One of the things that I really like around that was added to PI test not too long ago was, um, is stepwise. So that's not really all the running it all the time, but, um, stepwise will, and this would be a handy one to, to run all the time.
So what stepwise does is it takes, you can run all your tests in stepwise. And when you run it again, it'll start at the first failing test because it assumes you're trying to fix something. It'll start at that and then run until it finds a failure. So if you, if you haven't fixed this first failure, it'll just keep running that one until you fixed it and it'll go to the next one. Um, and, uh, so I do that a lot while I'm trying to debug something.
Um, and, and hooking that up with like an auto, like a watch feature. There's a bunch of ways you can watch your code to, to do that. Um, yeah, it's fun. Nice. Very cool. So let's do some real time follow up here. First, Alvaro is being all mischievous asking, I wonder what would happen if I install both plugins, both X, just in parallel. I, you can, I don't know if I've, you can run them at the same time. I should try. I have it installed on like the flask one. I ran it.
I installed both of them. And then try it in both, but not at the same time. I'll have to try the forks. It's going to go so fast. And then just going back to PyScript, there's like tons of excitement about PyScript. Yeah. Dale's excited. Brandon's excited. Um, and David says, I hope someday I, I can say back in my day, you couldn't just learn Python. You had to learn JavaScript too. Yeah. Indeed. Indeed. Uh, let's see. So I got one more to cover that is going to be fun as well.
And this one comes to us from former, uh, guest co-host, uh, Michael Feigert, uh, sorry, Matthew Feigert. And Matthew is a great support of the show. It sends all sorts of interesting things in to help us out and good ideas. And this is yet another one coming from the data science side of things saying, you know, one of the things you have to do often in say a Jupyter notebook is go download a file off of an API or just some link or S3 bucket or whatever. And you want to process it.
And if you use requests while great, you end up making the request, verifying that it worked, reading the stream into bytes, writing the bytes to a file, uh, picking a file name and then using that file name to open it. And then say, now you can process it. Right. Yeah. So there's this thing called pooch, a friend to fetch your data files. All right, pooch, go get, go get my files. Like a little, a little, a friendly dog that also seems to hold a snake in its mouth. So that's pretty cool.
Anyway, who wouldn't want a dog that can wrangle snakes to go help you with your notebooks? Anyway, the idea is you can do all of what I described with requests. You can do that in one line of code. Oh, wow. Yeah. And you get other cool features as well. So, um, it says, look, you can just make this one function call and it'll save it. And it'll also cache your files locally. So some of these files that data scientists especially work with are massive, right? You know, it's like a gig.
And every time you run the notebook, you don't want it to download the gig again. You just want it to run more quickly. So you can set up a location for it to cache it. Uh, you can pass in a hash of the file to say, I want to get this file and I expect it to be this MD five or whatever the heck the hash is that they're using so that you can be sure it doesn't change. Right.
So if you're doing like reproducible data science, you say, what you do is you download this file, then you apply this algorithm, then you get this picture. Well, if the data changes, I bet the picture changes. Right. And so you can put it like a layer of verification that it's unchanged from the last time you decided what it should be. That's pretty cool. You can do multiple protocols. So not just HTTP, HTTPS, but FTP. Oh my gosh. S FTP. Oh yeah. It's what else. Basic off.
It'll also automatically resolve DOIs, digital object identifiers, which are used in places like big share and Zen Zenodo. And this is about the reproducible science. Like here's that, here's the file. And like, we've been assigned an immutable ID that we can always refer back to it. So you can just say, here's the ID and it'll actually get the file and it'll even unzip and decompress files upon download. Neat. Pretty neat. Huh? Yeah. Yeah. Pretty straightforward.
Let's, let me see if I can find an example of. I love, I like the, the, the section of learning about it. It's called training your pooch. That's cute. Oh, nice. I love it. Apparently it has progress bars. Always download actions, logging, and you get multiple files, but the main use case is just file equals pooch.retrieve URL done. That seems pretty nice. Yeah. That's great. It's my data. Here it is. Oh, cool. So Pamphill Roy on the audience says, Hey folks, funny.
We're adding this to scipy optional to have a scipy dataset sub module. scikit image is using this as well. I had no idea. Very cool. Thanks for the extra background there. Cool. Yeah. But I think this is great. In fact, I know it's sells itself. It bills itself as being for data science. I also like to download files sometimes and not go through five or six lines of code. I could use this.
Yeah. Yeah. There's, there's a lot of stuff that data science people are doing that we can use in lots of other fields. So. Indeed. I do think that's actually one of the really interesting aspects of Python is we have so many people from these different areas that it's not just all, you know, CS grads doing the same thing. Yeah. Yeah, for sure. All right. Well, those are my items for today, Brian. Nice. I don't have any extras today. Do you have any extra information in stuff? I do.
I do have extras. So this one I'm very, very excited about. I have a new course that I just released called up and running with Git, a pragmatic UI based introduction. So I'm really excited. I just released. I haven't really even announced it yet, but I finished getting it all public and online and turned all the GitHub repos public and all that stuff right before we jumped on the call today. And the idea is there are tons of Git courses. So why create a Git course?
Well, I feel like so many of them are just like, okay, we're just going to work in the terminal or the command prompt. And you're just going to assume that like, that's the world of Git that you live in, like kind of a least common denominator approach. And while that, that is useful. Like, I don't think that's how most people are working, right? If you're in Visual Studio Code or PyCharm, like there's great hotkeys just to do the Git stuff and see the history and whatnot.
And there's other tools like SourceTree and Power and others. So it kind of takes this approach of like, well, let's take all the modern tools that give you the best visibility and teach you Git with that. So super fun. Which GUI tools are you using then? Which ones are you using? Visual Studio Code. Okay. PyCharm. SourceTree. Okay. Those are the things. And so I've done a lot of work.
I've tried to take some of my experience from doing some work on YouTube where I was experimenting with like setup and presentations and stuff. And I think I have a really neat, polished experience for this course with like lots of cool visuals and graphics and video and stuff. So hopefully people really enjoy it. Anyway, this is my extra. I just sent this out to the world. I'm pretty excited about this. Nice. Congrats. Yeah. Thanks. Thanks so much. You have no extras.
Does that mean you're ready for some humor? Yes. Always. All right. All right. This one. I chose this. Honestly, I just chose it just because of the title. So there's Robert. Is this Robert Downey Jr. looking at somebody in like some kind of wizard situation? Right? Like. Yeah. This is like Endgame or something. Okay. Yeah. I don't know the movie. Like apparently I stopped watching movies at some point. Now I don't. I'm out of touch.
So anyway, the title is when your code stopped working during an interview or it could be a demo presentation or whatever. Like you want to, you want to tell us what this is about, what that's going on here? So he's, he's, he's looking back at Banner. So who's the Hulk? He says, dude, you're embarrassing me in front of the wizards. Yeah. Because, yeah. Because Banner wasn't able to become the Hulk. So at the time. Right, dude. Don't, don't embarrass me in front of the wizards.
I just, I love to think of programmers as kind of like the modern day wizards. Like we can think of things and then poof, they, they kind of come into existence. Yeah. It's good. And also while working on that Git course, I had this pretty fun experience. Like right while I was recording it. Nice. And I'm just sitting there and then. Git was down. How often does GitHub itself go down? But no. Oh no. There's like an, the Octa cat is falling like with a 500 sign in its hands. Toops.
Which of course made me. I love the. Redo that section of the course. Yeah. I like the expression on your face for that. It's like. Yes. Exactly. People seem to really like that tweet. I'll, I'll put it in the show notes if people can check it out. Anyway, dude, don't embarrass me in front of the wizards. That's what I got for you. Yeah. Good. Good, good. Well, thanks. Thanks a lot again. It's a great show. Yeah. Sure was. Thanks. Thanks, Brian. Thanks for everyone who came.