#265 Get asizeof pympler and muppy

Hey there, thanks for listening. Before we jump into this episode, I just want to remind you that this episode is brought to you by us over at Talk Python Training and Brian through his pytest book. So if you want to get hands-on and learn something with Python, be sure to consider our courses over at Talk Python Training. Visit them via pythonbytes.fm/courses. And if you're looking to do testing and get better with pytest, check out Brian's book at pythonbytes.fm slash

pytest. Enjoy the episode. Hello and welcome to Python Bytes, where we deliver Python news and headlines directly to your earbuds. This is episode 265, recorded January 5th, 2022. I'm Brian Okken. I'm Michael Kennedy. And I'm Matt Kramer. Matt, welcome to the show. Thanks. Happy to be here. Yeah, welcome, Matt. Who are you? Oh, so a huge fan. I've listened to every episode. I actually, I'm one of these folks that started their career outside of software. I've heard a similar parallel

story a bunch of times in the past. So I have my degree actually in naval architecture, marine engineering, which is design of ships and offshore structures. In grad school, I started, I was started with MATLAB, picked up Python, thanks to a professor. And then over time, that's just grown and grown. Spent eight years in the oil and gas industry and using Python mostly for doing engineering analysis,

a lot of digital type stuff, IoT type monitoring work. And about three months ago, I joined Anaconda as a software engineer. And I'm working on our Nucleus cloud platform as a backend software. Very cool. Awesome. Yeah. Congrats on the new job as well. That's a big change from oil and gas. A couple of years. I mean, it is in Texas and all, but it's still, it's still on the tech side. Yeah. No, it's, it's related, but obviously a different focus. I wanted to make writing code

my job rather than the thing I did to get my job done. So. Fantastic. I'm sure you're having a good time. Yeah. Well, Michael, we had some questions for people last week. We did. I want to make our first topic a meta topic. And by that, I mean a topic about Python bytes. So you're right. We discussed whether the format, which is sort of, I wouldn't say changed. It's, I would rather categorize it as drifted over time. It's sort of drifted to adding this little

thing and do that different thing. And we just said, Hey, everyone, do you still like this format? It's not exactly what we started with, but it's, it's where we are. So we asked some questions. The first question I asked, which I have an interesting follow-up at the end here, by the way, is, is Python bytes too long at 45 minutes? That's roughly the time that we're, we're going these days, probably about 45 minutes. And so I would say, got to do the quick math here. I would say

70, 65%, let's say 65% are like, no, it's good. With a third of that being like, are you kidding me? It could go way longer. I'm not sure we want to go way longer, but there are definitely a couple of people that get, it's getting a little bit long. So I would say probably 12% of people said it's too long. So I feel like it's actually kind of a decent length. And one of the things I thought it's like, as we've changed this format, we've added things on, right? We added the joke that we started

always doing at the end. We added our extra, extra, extra stuff, but the original format was the six items. You covered three, I covered three. Now it's two, two. And we got Matt here to help out with that. So what is the length of that? And it turns out that that's pretty much the same length still. So the last episodes, 39 minutes, 32 minutes, 35 minutes, 33 minutes, that's how long are our main segments up to the

end of the minute. So it's kind of like, for people who feel it's too long, I wanted to sort of say, like, feel free to just delete it. Like you hear the six items, like delete it at that point. If you don't want to hear us ramble about other things that are not pure Python, you don't hear us talk about the joke or tell jokes, no problem. Yeah. Just, just stop. It's at the end for a reason. So if you're kind of like, all right,

well, I'm kind of done, then, then be done. That's totally good. Yeah. we'll put the important stuff up first. the other one was, do you like us having a third coast like Matt or, shell or whoever it is we've had on recently? And most people love that format or, you know, it's okay. So that's like, I think that that's, that's pretty good. I do want to read out just a couple of comments as well. There's stuff that you always get that are like, you just can't balance it. A couple of people

are saying like, you just got to drop the joke. Like, don't do that. The other people are like, the joke is the best. Who doesn't want to stay for that? So, you know, like, well, again, it's at the end. So, you, you can do that. But I also just wanted to say, thank you to everybody. They, they wrote a ton of nice comments to you and me at the end of that Google forum. So, one is, I can't tell what counts as an extra or normal, but it's fine. I love it. By the way,

it's such an excellent show. Fun way to keep current. Brian is awesome. Oh, good. I asked my daughter to submit that. She's good. I think your third grass, having a third guest is great. Like I said, drop the jokes, keep the jokes for sure. Ideal. I, so anyway, there, there's a bunch of, nice comments. I think the other thing, that I would like to just speak to real quick and get your thoughts on and maybe you as well, Matt, cause you've been on the receiving end of this a lot is us having the

live audience. Right. I think having a live audience is really interesting. I also want to just acknowledge, like we knew that that would be a slight drift of format, right? So if you're listening in the car and there's a live audience comment, it's kind of like, well, but I'm not listening to it live. That's kind of different, but I think it's really valuable. One time we had four, maybe four Python core developers commenting on the stuff we were covering relative. Like that's a

huge value to have people coming and sort of feeding that in though. For me personally, I feel like it's, yeah, it's a little bit of a blend of formats, but I think having the feedback from the audience, especially when people are involved in what we're talking about, I think that's worth it. Brian, what do you think? Well, we, we, we try not to, to let it interrupt the flow too much, but there's some great stuff.

Like if somebody, if we say something that's just wrong, somebody will correct us. And that's, that's nice. the other thing is, sometimes somebody has a great question on a topic that like, we should have, we should have talked about, but we didn't, we didn't, we didn't. Right. We don't know everything. We certainly don't. so I do want to add one more thing. the, there was a comment like, Hey, we as hosts should let the guests speak. We should be better interviewers. I'm like,

this is not an interview format, you know, like talk Python is a great interview format. Oh, that's where the guest is featured. Testing code is a great form interview format where the guest is featured. This is sort of just three people chatting. It's not really an interview format. So, and, and we always tell the guests to interrupt us and they just, they don't much. So, yeah. Yeah. So Matt, what do you think of this live audience aspect? Like, do you feel like that's tracks or is it good?

Well, yeah. First of all, thank, I'm, I'm, I'm glad that, people generally like having a guest. Otherwise this would have been very awkward. but no, I, I do like it. I think. Well, where'd Matt go? Oh, he must've disconnected. There was one. Occasionally there is a kind of a, a little bit of a disruption,

but I think in general it's been great. Yeah. I've definitely been listening when times when, um, you know, a bunch of people are chiming in because there's always, as you know, that you, you mentioned a GUI library and then there's about 12 other options that you may not have covered in. Instead of waiting 12 weeks, you could just get them right out. so I think that's

great. And I, I'm, I'm generally a audio listener. I listen when I'm walking my dogs, but, but I love having the video because when I am very, when I'm interested in something, I can go hop to it right away and see what you're showing, which I really like. So. Yeah. Awesome. Thank you. two other things that came to mind. Someone said it would be great if there's a way where we could submit like ideas and stuff like that for guests, and whatnot. right here at the top in our menu,

it says submit. So please, reach out to us on Twitter, send us an email, do submit it there. The other one was, if we could have time links, like if, if you go to the, the, to listen and at some certain time, a thing is interesting that's mentioned, be cool. If you could like link at, at a time, if you look in your podcast player, it has chapters and each chapter has both a link

and a time. So, like the thing that Brian's going to talk about next interpreters, if you want to hear about that during that section in your podcast player, you can click the chapter title and it will literally navigate you to there. So it's already built in. Just make sure you can see it in your device. Yeah. All right. I think that's it, for that one, but yeah, thank you for everybody

who had comments and took the time. Really appreciate it. Yeah. And just the comment, if you, if you want to be a guest, just email on that form and you might be able to do it. That's right. That's right. Yeah. Great to have you here. Um, actually I didn't want to talk about interpreters. No, that's me. Oh, wait, you're right. Well, you're talking about it now because I've changed. No, let's talk about Adder. Sorry. I saw the wrong screen. Go for it.

Apparently we're not professional here, but, no, it's okay. I wanted to talk about Adder's. We, we haven't really talked about it much for a while because there are lots of reasons, but Adder's is a great library and it just came out with Adder's, came out with a release 21.3.0, which is why we're talking about it now. And there's some documents. There's a little bit of change. There's some changes and some documentation changes. And I really,

in an article I wanted to cover. So one of the things you'll see right off the bat, if you look at the, the overview page of the Adder's site is, is it, is it highlighting the, the define, decorator. It's a different kind of way that if you've used Adder's from years ago, this is a little different. So the, there's a, there's, there was a different weighted to a different API that was added in the

last release. And this is, or one of the previous releases. And now that's the preferred way. So this is what we're calling modern Adder's. but along with this, I wanted to talk about an article, uh, that Hinnick wrote, about, about Adder's. And it's a little bit of a history and I really love this discussion. So, and I'll try to quickly go through the history. early on, we didn't have data classes. Obviously we had, we could handcraft classes, but there were problems with it. And there

was a library called characteristic, which I didn't know about. This was, this was, before I started looking into things, that, and then glyph and Hinnick in, in 2015, we're discussing it ways to change it. And that begat the old original Adder's, interface. And there were things like Adder.s and Adder attribute that were partly out of the fact that the old way of characteristic attribute was a lot of typing. So they wanted to something a little shorter. and then it kind of

took off. Adder's was pretty, pretty popular for a long time, especially fueled by a 2016 article by glyph called the one Python library. Everyone needs, which was a great, this is kind of how I learned about it. and then, there was a, you know, different kind of API that we were used to for adders and it was good and everything was great. And then in 2017, Guido and Hinnick and Eric Smith talked about, at, in the Python 2017, they talked about how to make something like that

in the standard library. and that came out of that came PEP 557 and data classes and data classes showed up in, in Python three seven. and then, so what, then a dark period happened, which was people were like, why do we need adders anymore? If we have data classes? Well, that's one of the things I like about this, this article. And then there's an attached article

that is called why not, why not, why not data classes instead of adders? And, and this is, it's, it's important to realize that data classes have always been a limited set of adders. Adders was a, is a super set of functionality and there's a lot of stuff missing in data classes, like, like, equal equality customization and validators. Validators and converters are very important if you're using a lot of these. and then also people were like, well, data classes

kind of a nicer interface, right? Well, not anymore. the, the pound defines pretty, or they at defines really nice. This is a really easy interface now to work with. So anyway, yeah. And it has typing and it has typing. and, and I'm glad he wrote this because I'm, I kind of was one of those people of like, am I doing something wrong? If I'm, if I'm, using data classes, why should I look at adders? And one of the things that there's a whole bunch of

reasons. One of the things that I really like is adders, has, slots, the, the slots are on to by, by default. So you have, you kind of define your class once instead of, keeping it growing. Whereas the default Python way and data classes is to allow classes to grow at runtime,

have more, more attributes, but that's not really how a lot of people use classes. So if you, if you came from another language where you have to kind of define the class once and not at runtime, adders might be a closer fit for you. I like it. And it's whether you say at define or at data class, pretty similar. Yeah. Yeah. Adders is really cool. I personally haven't used it, but I've always wanted to try it.

Um, we're using FastAPI and, and Pydantic. So I've really come to like that library, but adders is something that looks really full featured and nice. definitely something I want to pick up. Yeah, it's cool. And Pydantic also seems very inspired by data classes, which I'm learning now. I suspected, but now learning that is actually inspired by adders and they kind of sort of leapfrog each other in this, this same trend, which is interesting.

Yep. So yeah, cool. Good one, Brian, Matt. I thought Brian was going to talk about this, but you can talk about it. This would be me. Yeah. so this one's not strictly Python related, but I think it's very

relevant to Python. so I mentioned earlier, I, I came from a non CS background. and I've always, I've just been going down the rabbit hole for about 10 years now, trying to understand everything and pick it up and, and really connect the dots between how do these very flexible objects that you're working with every day, how do those get actually implemented? and so the first thing I did, if you heard of this guy, Anthony Shaw, yeah, I think he's been mentioned once or twice. He wrote

a great book, shout out CPython internals. Really? Anthony's out in the audience. He even says happy, happy new years. Hey, happy new years. So this book is great. If you want to learn how CPython's implemented. but because I don't have a traditional CS background, I've always wanted, you know, I felt like I wanted to get a little bit more to the fundamentals and I don't remember where I found out about this book, but crafting interpreters,

um, I got the paperback here too. I highly recommend it. It's, it's, it's a, implementation of a language from start to finish. Every line of code is in the book. it's a dynamic interpreted language, um, much like Python. but I really like how the book is structured. So it is, it was written over, I think five years in the open. I think the paperback may have just come out last year, but you walk through every step from tokenization, scanning, building a syntax tree, and all the

way through the end. But what I really like about it is, is you actually, you develop two separate interpreters for the same language. So the first one is written in Java. it's a direct, evaluation of the abstract syntax tree. so that was really how I got a lot of these bits in my head about what is an abstract syntax tree. How do you start from there? How do you represent these

types? But the second part is actually very, where I think it becomes really relevant for Python because you, the second part is written in C it's a bytecode virtual machine, with garbage collection. So it's not exactly the same as Python, but if you want to dig down into how would you actually, you know, implement this with the types that you have available for you in C, but get

something flexible, much like Python, I really recommend this. so again, it's not directly, there's some good side notes in here where they, he compares, you know, different implementations between different languages like, Python and JavaScript, et cetera, Ruby. But I really liked this book. I devoured it during my time between jobs and, yeah, I keep telling everyone about it. So I thought it would be good for the community to hear. Yeah. Nice. Yeah. I didn't study this

stuff in college either. I mostly studied math and things like that. And so understanding how virtual machines work and all that is just how code executes. I think it's really important. You know, it's, it's not the kind of thing that you actually need to know how to do in terms of you

got to get anything done with it. But sometimes your intuition of like, if I asked the program to work this way and it doesn't work as you expected, you expect, you know, maybe understanding that internal is like, Oh, it's because the, it's really doing this and all, everything's all scattered out on the heap. And I thought numbers would be fast. Why are numbers so slow? But okay. I understand now. Yeah. I, I, I really liked the, I mean, it answered a lot of questions

for me. Like how does a hash map work? Right. That's a dictionary in Python. What is a stack? Why would you use it? What is the, when you do a disassemble and you see bytecode, what does that actually mean? Right. I really, I really enjoyed it. And he's got a really great, books open source. It's got a really great build system. If you're interested in writing a book, it's very cool

how the adding lines of code and things like that are all embedded in there. And he's got tests, written for every part where you add a new, you know, a new bit to the code, there's tests written and there's ways where he uses macros and things to block them out. It's pretty, pretty interesting. Nice. Testing books. That's pretty excellent. Yeah. Yeah. So Matt, now being at Anaconda, like that world, the, the Python world over in the data science stack, and especially around there has so much

of like, here's a bunch of C and here's a bunch of Python and they kind of go together. Does this give you a deeper understanding of what's happening? Yeah, for sure. I think, CPython internals gave me a really good understanding a bit about a bit more about the C API and why that's important. Um, it is, I'm sure you, well, as you know, and the listeners may know, like the binary compatibility is really important, between the two and dealing with locking and the, the,

global interpreter lock and everything like that. so it's definitely given me a better conceptual view of how these things are working. As you mentioned, I don't, you don't need to know it necessarily on a day-to-day basis, but I've just found that it's given me a much better mental model. Having an intuition is valuable. Yeah. quick audience feedback. Sam out in the live audience says, I started reading this book over Christmas day and it's an absolute joy. So yeah, very cool.

Uh, one more vote of confidence for you there. Cool. Brian, are we ready for my, my next one? Yes, definitely. A little, Yamale. Yeah, I'm hungry. So this one is cool. it's called Yamale or Yamaly. I'm not a hundred percent sure, but it was suggested by Andrew Simon. Thank you, Andrew, for sending this in. And the idea of this is we work with YAML files that's often used

for configuration and whatnot. But if you want to verify your YAML, right, it's just text. Maybe you want to have some YAML that has a number for a value, or you want to have a string, or maybe you want to have true false, or you want to have some nested thing, right? Like you could say, I'm going to have a person in my YAML. And then that person has to have fields or value set on it, like a name and an age

with this library. You can actually create a schema that talks about what the shape and types of these are much like data classes. And then you can use Yamaly to say, given a YAML file, does it validate? Think kind of like Pydantic is for JSON. This is for YAML, except it doesn't actually parse the results out. It just tells you whether or not it's, it's correct. Isn't that cool? I think it looks neat. yeah.

Yeah. So it's, it's a pretty easy to work with, obviously requires modern Python. It has a CLI version, right? So you can just say, YAML, give it a schema, give it a file, and it'll go through and check it. It has a strict and a non-strict mode. It also has an API. So then to use it, just say, YAML.validate schema and data, either in code or on the CLI. And in terms of schemas, like I said, it looks like data classes. You just have a file like name:str, age:int. And then you can even add

additional limitations. Like the max integer value has to be 200 or less, which is pretty cool. then also, like I said, you can have, more complex structures. So for example, they have what they call a person, but then the person here, actually you could nest them. So you could have like part of your YAML could have a person in it and then your person schema could validate that person.

So very much like Pydantic, but for YAML files, like here, you can see, scroll down, there's a, an example of, I think it's called recursion is how they refer to it, but you can have like nested versions of these things and so on. So if you're working with YAML and you want to validate it through unit tests or some data ingestion pipeline or whatever, I just want to make sure you're loading the files correctly, then you might as well hit it with some YAML-y guessing.

One of the things I like about stuff like this is that, things like YAML files, sometimes people just sort of edit it in, in the Git repo, instead of making sure it works first and then it gets, and then having a CI stage that says, Hey, making sure the, the animals valid syntax is, is pretty nice so that you, so that you know it before it blows up somewhere else with some weird error message. So yeah, exactly. Yeah. This is really cool. The validation of these types of input files,

especially YAML files is really tough. I've found just cause it's indentation based and, white space is not a bad thing, obviously, but for YAML, it's tough. I can't tell you how many hours I've banged my head against the wall in the past life. trying to get Ansible scripts to run and things like that.

So this is really neat. And anytime I see something like this, I just wish that there was one way to describe those types somewhere like, and if preferably in Python, just cause I like that more, but this is really cool. Yeah. I wouldn't be surprised if there's some kind of Pydantic mapping to YAML instead of to

JSON. and you can just kind of run it through there, but yeah, I think this is more of a challenge than it is safe for JSON because JSON, there's a validity to the file, regardless of what the schema is, where YAML less so, right? Like, well, if you didn't indent that, well, it just, that means it's, it belongs somewhere else, I guess, you know, it's a little, a little more freeform. So I guess that's why it's popular, but also nice to have this validation. So yeah. Thank you for Andrew. Thank

you to Andrew for sending that in. yeah. So next I wanted to talk about Pimpler, which is great name. and I, I honestly can't remember where I saw this. I think it was a post on or something by Bob Belderbos, or something he wrote on five bites. I'm not sure. anyway, so I'll give him credit. Maybe it was somebody else. So if it was somebody else, I apologize. But anyway, what is Pimpler? Pimpler is a little tiny library,

which has a few tools in it. And it has, one of the things that says is, one of the things I saw, it does a few things, but what I, it measures monitors and analyzes memory behavior and Python objects. but the, it's the memory size thing that, that was interesting to me. So, you've got, like for instance, it has three, three tools built into it, a size of and muppy, which is a great name, and class tracker. So a size of is a, provides a basic

size information for one or a set of objects. And muppy is a monitoring. I didn't play with this. I didn't play with the grass, the class tracker, either class tracker provides offline analysis of lifetimes of Python objects. Maybe if you've got a memory leak, you can see like there's a hundred downs of my hundreds of thousands of this type. And I thought I only had three of them.

Yeah. And so one of the things that I really liked of, with a size of is, it's, it, I mean, we already have, sys get size of in Python, but that just kind of tells you the size of the object itself, not of the, like later on. So a size of will tell you not just what the size of the object is, but all of the recursively, it goes recursively and, and, looks at the size of all the stuff that it contents of it. So, right. And people haven't looked at this, you know,

they should check out Anthony's book, right? But if you've got a list and say, the list has a hundred items in it and you say, what is the size of the list? The list will be roughly 900 bytes. Cause it's 108 byte pointers, plus a little bit of overhead. Those pointers could point at megabytes of memory. You could have a hundred megabytes of stuff loaded in your list. And if it's really only a hundred, like, no, that's 900 bytes, not 800 megabytes or whatever. Right. So you really need to,

if you actually care about real whole memory size, you got to use something like a size up. It's cool that this is built in. I, I had to write this myself and, it was not as fun. Yeah, this is awesome. I, I also, I, I hit this, sometime in grad school, I remember when I was going to add a deadline or something. And, just, I hit the same thing about the number of bytes in a list being so small and just writing something that was hacky to try to do the same thing,

but to have it so nice and available is great. And the name is awesome. I love silly names. Yeah, for sure. Uh, one of the example, and I was confused that the example we're showing on the screen is, just a, there's, you've got a, a list of, a few items, some of it's a text to, some of them are integers and some are lists of integers or tuples of integers and being able to go down and do the size of everything. But then there's also a, you can get more detailed. You can, give it,

a sized, a size, with, with a detail numbers. I'd have to look at the API to figure out what all this means, but the example shows each element, not just the total, but each element, what the size of the different components are, which is kind of cool, but it lists like a flat size. And I'm like, what's the flat thing? So I had to look that up and a flat, the, flat size returns the flat size of a Python object in bytes determined as the basic size.

So like in these examples, it's, like the tuple is just a flat, the tuple itself is 32 bytes, but the, the tuple and its contents is 64. I see. So flat is like sys.get size of, and size is a size of that bit. I think that's what it is. It is. but yeah, not sure, but that's what I'm thinking. Yeah. So for people who are listening, they don't see this. You should check out the docs page, right? Like a usage example, because if you have a list containing a bunch of stuff,

you can just say, basically print this out and it shows line by line. This part of the list was this much. And then it pointed at these things. Each of those things is this big and it has constituents and, and so on. my theory is that the detail equals one is recurse one level down, but don't keep traversing to like show the size of numbers and stuff. Yeah. Yeah. Cool. Yeah. I love it. This is great. Yeah. Oop. All right. Take it all.

Okay. So, I'm going to talk about HV plot and, HV plot dot interactive, specifically. so this is something I actually wasn't very aware of until I joined Anaconda, but one of my colleagues, Philip Rodeger, who I know is on talk Python, work point, um, is our, is the developer working on this. And there's basically there's, you know, when you're working in the PI data ecosystem, there's pandas and X array and tasks, there's all

these different data frame type interfaces, and there's a lot of plotting interfaces. And there's a project called hollow views or HV plot, which is a consistent plotting API for that you can use. And, and the really cool part about this is you can swap the backend. So for example, pandas default plot, we'll use dot plot and it'll make a mat pot lib. But if you want to use something more interactive, like bokeh or hollow views, you can just change the backend and you can use the same

commands to do that. so that's, that's cool. And you set it on the, on the data frame. Yeah. Yeah, exactly. So what you, what you do is you import HV plot dot pandas. And then on the data frame, if you change the backend, you just do data frame dot plot. and there's a bunch of kind of, you know, rational defaults built in for how it would show the different

columns in your data frame, versus the index. And then I like that. Cause you could swap out the plots by writing one line, even if you've got hundreds of lines of plotting and stuff, right. And it just picks it up. Exactly. Yeah. And, and the common workflow for a data scientist is you got, you're reading in a lot of input data, right? Then you want to transform that data. So you're doing generally a lot of method chaining, is a common pattern where you want to do things like

filter and select a time and maybe pick a drop a column and do all kinds of things. Right. At the end, you either want to show that data or write it somewhere or plot it, which is very common. Um, now there's interactive part. Philip demoed this or he gave a talk at PI data global about two months ago. I think, it kind of extends on that. And this blew my mind when I saw it. So, if you had a data frame like thing and you put dot interactive after it, then you can put your

method chaining after that. So this is an example where you say, I want to select a discrete time and then I want to plot it. And this is, this particular example is not, doesn't have a kernel running in the backend, so it's not going to switch. But if you were running this, in an actual live notebook, it would be changing the time on this chart. And again, this is built to work with the, a lot of the

big data type APIs that match the pandas API. Nice. so for people listening, if you say dot interactive and then you give the parameter that's meant to be interactive, that just puts one of those I Python widget things into your notebook right there. Right. That's cool. Yeah. So, a related, library is called panel, which is, it is for building dashboards directly from your notebooks. so you can, if you had a, a Jupyter notebook, you could say panel

serve and pass in the notebook file. and it'll make a dashboard. That's the thing I want to show in a, in a, in a second here, but the, the way the interactive works is really neat. So wherever you would put a number, you can put one of these widgets. And so you can have time selectors, you can have things like, sliders and you can have input boxes and things like that. And all you do is you would

change the place where you put your input number at, put one of those widgets in. And then it sort of, it, it, I actually don't know how it works exactly under the hood, but from what I understand, you put this interactive in, and then it's capturing all the different methods that you're adding onto it. And anytime one of those widget changes, it will change everything from that point on. Um, and so the, the demo here was from another panel contributor, Mark Skoff, and that's in,

and I'm just going to play this and try to explain it. So we have a data pipeline on the right where we've chained methods together. and what he's done here is he's just placed a widget in as a parameter to these different methods on your data frame. And then this is actually a panel dashboard that's been served up in the browser. And you can see this is all generated from the, a little bit of code on the

right. So if you want to do interactive data analysis or exploratory data analysis, you can really do this, um, very easily with this interactive function. And when I saw this, I kind of hit myself in the head because the, normally my pattern here was I had a cell at the top with a whole bunch of constants defined. And, you know, I would manually go through and okay, change the time, start time from this time to this time, or change this parameter to this and run it again. And over and over.

You got to remember to run all the cells that are affected. Exactly. So the fact that the fact that you can kind of do this, interactively while you're working, um, I could see how this would just, you know, you don't break your flow while you're trying to work. And the method chaining itself is, I really like too, because you can comment out each stage of that, um, as you're going and debugging what you're working on. So, yeah, this is really neat.

And I definitely, I put a link in the show notes to the actual talk, as well as this gist that Mark Skobmatson put on GitHub. And, yeah, it's, it blew my mind. It would have made my life a lot

easier had I known about this earlier. So, yeah. And one of the important things I think about plotting and, interactive stuff is it's not, even if your end result isn't a panel or an interactive thing, sometimes getting to see the, see the plot, seeing, seeing the data in the visual form helps you understand what you need to do with it. Yeah, no, exactly. I mean, I did a lot of work in the past with time series data and

time series data, especially if this was sensor data, you had a lot of dropouts. you might have spikes and, and you're always looking at it and trying to make some judgment about your filter parameters and, and being able to have that feedback loop between, changing some of those and seeing

what the result is, is a huge game changer. So yeah. Yeah. And you, you can hand it off to someone else who's not writing the code and say, here, you play with it and you, you tell, you know, give it to a scientist or somebody.

Oh, that's exactly right. That's what panel's all about is what the, the biggest challenge that I always had in many data scientists have is you do all your analysis in a notebook, but then you got to show your manager or you got to show your teammates and going from that, going through that trajectory is, can be very challenging. these new tools are amazing to do that, but that's how I

turned myself into a software engineer because that's what I wanted to do. But I went out, went down the rabbit hole and learned Flask and dash and how to deploy web apps and all this stuff. And yeah, well, I'm glad you did. Yeah. Maybe I wouldn't be here if I hadn't done that, but,

but yeah, this is really cool. And I definitely recommend people look at this. there was also another talk this, sorry, this is an extra, but, there was another talk at PI data global, um, hosted by Jim, James Bednar, who's our head of consulting, but he leads pyviz, which is a community for visualization tools. And it was a comparison of four different, dashboarding apps. So it's panel dash, voila and streamlit. And they, they just had, you know, main contributors from the four

libraries talking about the benefits and pros and cons of all of them. So if anyone wants to go look at those, I definitely recommend that too. That sounds amazing. All those libraries are great. Nice. Thanks. Oh, speaking of those extra parts of the podcast that make the podcast longer, uh, we should do some extras. We should, we should do some extras. Got any? I don't have anything extra. Matt, how about you? Yeah. two things. So first, you can show my

screen. last year at a kind of hired the piston developers, piston is a faster implementation fork of c python. I think it was at Instagram first. I can't recall, but anyway, before, right before the holidays, they released, pre-compiled packages for many of a couple hundred of the most popular python packages. So if you're interested in trying piston, I put a link to their blog post in here. Um, they're using conda right now. They were able to leverage a lot of the conda forge recipes for

building these. this is that binary compatibility challenge that we talked about earlier. So, yeah, I, I know the team's looking for feedback on, on that. If you want to try that, feel free to go there. And it mentions in the blog that they're working on PIP. That's a little harder to just because of how, you know, the build stages for all the packages aren't centralized with PIP. So it's a little more challenging for them to do that. and then just the last thing is,

um, you know, I, I don't want to be too much of a salesman here, but, we are hiring. It's an amazing place to work and I definitely recommend anyone to go check it out if they're interested. so fantastic. Yeah. And you put a link in the show notes. People want to, yeah,

it's anaconda.com slash careers. and we're doing a lot of cool stuff and growing. So if anyone's looking for work in, in data science or just software and building out some of the things we're doing to try to help the open source community, and bridge that gap, spelled it wrong, bridge that gap between, enterprise and open source and data science in particular. Yeah. And definitely seems like a fun place to work. So cool. People looking for a change or for

a fun Python job. Yeah. Yeah. Cool. People do reach out to Brian and me and say, Hey, I really want to get a Python job and doing other stuff, but how do I get a Python job? Help us out. So we don't know, but we can recommend places like Anaconda for sure. Yeah. It looks like there's about 40 jobs right now. And, so check it out. Fantastic. Oh, wow. That's awesome. All right. Well, would it surprise you if I had some extra things? It would surprise me if you didn't.

All right. First of all, I want to say congratulations to Will McGugan. We have gone the entire show without mentioning rich or textual. Can you imagine? But no, only cause I knew you were going to talk about this. Otherwise I would have thrown it in. Yeah. So Will last year, a while ago, I don't know the exact number of months back, but he was planning to take a year off of work and just focus on rich and textual. It was getting so

much traction. He's like, I'm just going to, you know, live off my savings and a small amount of money from the GitHub sponsorships and really see what I can do trying that. Well, it turns out he has plans to build some really cool stuff and has actually all based around rich and textual in particular. And he has raised a first round of funding and started a company called textualize.io. How cool is that? Well, we don't know because we don't know what it's going to do.

All you do is if you go there, it's like a command prompt. You just enter your email address. I guess you enter something happens. Let's find out what happens. Yes, I'm confirmed. Basically, just get notified about when textualize comes out of stealth mode. But congrats to Will. That's fantastic. Another one we've spoken about tenacity. Remember that, Brian? Yeah. So tenacity is cool. You can say here's a function that may run into trouble if you just put at, you know, tenacity.retry

on it and it crashes. It'll just try it again until it succeeds. That's probably a bad idea in production. So you might want to put something like stop after this or do a little delay between them or do both. I was having a race condition. We're trying to track when people are attempting to hack, talk Python, the training site, the Python byte site and all that. And it turns out when they're trying to attack your site, they're not even nice about it. They hit you with a botnet of all sorts

of stuff. And like lots of stuff happens at once. And there's this race condition that was causing trouble. So I put retry, a tenacity.retry. Boom, solved it perfectly. So I just wanted to say I finally got a chance to use this to solve some problems, which was pretty cool. That's really cool. The other one that's similar to this, which I've used, and I think, I don't know if you've used Brian, but it's called pytest Flaky. And it's awesome because

I was working with this time series data historian. I had a bunch of integration tests in my last job, but you know, network stuff, it would drop out occasionally. And so you can do very similar type things and wrap your test in an @flaky decorator and do similar type stuff and, you know, give it three tries or something before you make it fail. Yeah, exactly. That's cool. That's what mine, I think mine does three tries and it's like randomly

a couple of second delay or something. Remember that part, Brian, where we talked about, it's really cool if people are in the audience while we talk about stuff and then get a little feedback. So Will McGooghan says, "Hey, thanks guys. Can't wait to tell you about it." Yeah, congrats, Will. That's awesome. Glad to see you out there. All right. A couple of other things. Did you know that GitHub has a whole new project experience?

That's pretty awesome. Have you seen this? I haven't. I haven't seen this. So you know how it's like this Kanban board, Kanban board, where you have like columns, you can move your issues between them. So just last week, they came out with this thing called a beta projects where it still can be that, or it can be like an Excel sort of view where you have

little drop down combo boxes. Like I want to move this one in this column by going through that mode or as a board, or you can categorize based on some specification, like show me all the stuff that's in progress and then give me that as an Excel sheet and all these different views you have for automation. And then like there's APIs and all sorts of neat stuff in there. So if you've been using GitHub

projects to do stuff, you know, you can check this out. It looks like you could move a lot of, a lot more work towards that on the project management side of software they used to. This is really neat. Yeah. In my previous job, I was using Azure DevOps. I was always wondering when some of those features might move to GitHub. I don't know if that's what happened here, but being able to have this type of project management in there for this type of things, it's really,

really great. Yeah. Super cool. Yeah. One of the things I love about stuff like this is because even, I mean, yes, a lot of companies do their project management on or projects on in GitHub or places like that, but also open source projects often have their often have the same needs of project

management as, as private commercial projects. So, yeah. Yeah. I personally, I only have a few open source projects that are kind of personal and no one would probably want to use them, but even just keeping notes about to-dos and future stuff and it would be really nice. Yeah. Just for future you, if nothing else, right? Yeah. Awesome. Okay. So this is cool. Now the last, yeah, this last thing I want to talk

about is Markdown. So, Roger Turrell turned me on to this. there's this new Markdown editor, it's cross platform. Yes. Cross platform called Typeora. And we all spend so much time in Markdown that just, wow, this thing is incredible. It's not super expensive and it looks like a standard Markdown editor. So you write Markdown and it gives you a whizzy wig, you know, what you see is what you get style of

programming, which is not totally unexpected. Right. But what is super cool is the way in which you interact with it. And actually I am going to show you real quick. So you can, you can see it and then you can tell people like, what do you think about this? here, I think that's it. I'm back. Waiting. There. Okay. Yeah. So here, here's, here's Mark, here's a Markdown file for my course, just the practices

and whatever you can say, you know what? I would like to view that in code style. Right. Well, that's kind of cool. We want to edit this. You click here and it becomes, Ooh, comes Markdown becomes Markdown. That's, but this is a boring file. So let's see about, it has a whole file system that navigates like through your other Markdown stuff, correctly. So like here, chapter eight, it's a good one. So we go over to chapter eight on this,

and now you can see some more stuff. Like you can go to set these headings and whatnot. But if you go to images, like you can set a caption and then you could even change the image, like right here, if it were a PNG, it's not, but so put it back as JPEG and then it comes back. You can come down and write a code fence. use the right symbol and you can say def, a, right, whatever. And then you pick a language. Isn't that, isn't that dope? Oh, this is so good.

So if, if you end up writing a lot of Markdown and if you need to get back, you just, go back and switch back to raw Markdown and then go back to this fancy style. I think this is really a cool way to work on Markdown. I'm actually working on a book with Roger and, it's got tons of Markdown and it's been a real joy to actually use this thing on it. So yeah. Does it have VI mode?

VI mode? Probably not. I don't know about that, but it has themes. Like it has, it has, like you can do like a, like a night mode or I could do like a newspaper mode or, you know, take your pick. It's, it's pretty cool. The weirdo grad student in me is upset that this isn't LaTeX. It has, it has built in LaTeX. It has like, you can do, yeah, you can do like inline LaTeX and you can, there's a bunch of settings you can set for the LaTeX. It's got a whole, a whole math section

in there. Oh, that's sweet. Okay. Yeah. Let's see. So am I the only person that went all the way through college pronouncing it LaTeX? I did too, but I just learned that the cool way of saying LaTeX. LaTeX. Yeah. Yeah. It's French. No, I don't know. No. Yeah. It has, it has support for like chemistry settings, like inline LaTeX and math and all sides of good stuff. So yeah, it's, it's,

I'm telling you, this thing's pretty slick. So, all right. Well, I got to do my screen share back because so you all can see the joke because the joke is very good and we're going to cover it. Where's the joke? But it's at the end. It's at the end. So if people don't want to listen to the joke, they don't have to. Brian, I blew it. You did? I blew it. I blew it. Before I move off, the Markdown thing though, Anthony Shaw says editorial for iPhone and iPad is really nice too.

Cool. So, but let's do, let's do the joke. So I blew it because I was saving this all year. I saw this like last March and I'm like, this is going to be so good for Christmas. Yeah. And then we kind of like had already recorded the episode. We're not going to do it. We'll just take a break over. So we didn't have a chance to do it. So let's do it now. People are going to have to go back just a little tiny bit for this one. Are you ready? Yes. Matt, you ready?

Yeah. So this goes, this sort of a data database developer type thing here. And it's on a, I don't know why it's on a printout. Anyway, it's called SQL clause as in SQL clause. So it's, he's making a database. He's sorting it twice. Select star from contract contacts where behavior equals nice. SQL clause is coming. Nice. It would have been so good for Christmas, but we can't keep it another year. I got to get it out of here. You're going to sing it. SQL clause is coming to town. Yep, exactly.

Okay. I want to share a joke that I don't have a picture for. All right. Do it. But, but my daughter made this up last week. I think she made it up, but it's just been cracking me up for, and I've been telling it to everybody. So it's a short one. Imagine you walk into a room and there's a line of people all lined up on one side. That's it. That's the punchline. I love it. Nice. We've got my, we had my, my cookie candle last time. Nice. My, I can't

all these cookies. We've got a dad joke of the day channel in our Slack at work. And it's, it makes me oof every time. Nice. Nice. Okay. All right. Nice to see everybody. Thanks, Matt, for joining the show. Thank you for having me. Good to see you, Michael, again, as always. Yeah. Good to see you. Thank you. Thank you. Thanks for listening to Python Bytes. Follow the show on Twitter via @pythonbytes. That's Python Bytes as in B-Y-T-E-S. Get the full show notes over at

pythonbytes.fm. If you have a news item we should cover, just visit pythonbytes.fm and click submit in the nav bar. We're always on the lookout for sharing something cool. If you want to join us for the live recording, just visit the website and click live stream to get notified of when our next episode goes live. That's usually happening at noon Pacific on Wednesdays over at YouTube. On behalf of myself

and Brian Okken, this is Michael Kennedy. Thank you for listening and sharing this podcast with your friends and colleagues.