#331: Meet the Python Developer in Residence: Lukasz Langa

I loved what you're doing there. Then we talked about the Python language summit. You gave us a wrap up on that or a rundown on that for 2018 and then diving into CPython 3.8 and beyond. So here we are again. Thanks for being back. Yeah. Happy to be here. Like 3.8 is now already an old release, right? It's already in security only fixes mode. So it was new at the time. Lucas Lange: Yeah. I think when we were talking about

it, it's like, what are the new exciting features? I guess, you know, while we're on that topic, just real quickly is, you know, what are the new features that you're excited about now, either 3.9, 3.10 or 3.11? Like what's out now, what's just about out and what's coming soon? There's quite many. The one that I'm kind of living with the most right now is actually kind of just better error messages. So this is kind of like a trend that is still kind of

happening, right? So there's further changes for 3.11 later, which is already like the main branch right now of development, right? So 3.10 is already in release candidate mode. So like we're going to get the second release candidate pretty soon. And then, you know, the final release. So all new features now go to 3.11 already, but the better error messages like really kind of, they're kind of this icing on the cake that

make stuff much nicer for you. There's a bunch of typing changes that I was really looking forward to like back in PEP 484 times, like when we first added string and well annotations to Python. I always kind of wanted them to feel more native inside your code. And now with Python 3.10, you can use, I guess it's PEP 604, like union types, you know, like that kind of are not so verbose. They use the pipe operator instead of spelling out, you know, capital case union of

whatever, which is kind of more compact and also more readable through this, right? Because now types that aren't super trivial don't look as scary anymore, right? Because like you kind of don't have to have the square brackets so much and whatnot. Yeah. Yeah. For people who haven't explored this deeply, there previously was basically a parallel, a mirrored hierarchy of types that weren't actually the types, but they

describe the types. So from typing import capital L list, that would say you're returning a lowercase L list as your type. And then it got even more interesting when you have like optional, right? I have an optional user that's returned from this query. Now you can just do things like user pipe none and things like that. Yeah, absolutely. So this was fun, but also like what

they really kind of like to use is that like the lowercase types just feel more natural, right? So, you know, kind of just putting dicts and lists and whatever, like just straight out of built ins and just applying generics to them just kind of looks right. Like feels right. So yeah, it always felt weird. Like why do I have to do an import just to talk about this thing I'm already using, right? Yeah. Because like, if you actually look at how the feature was enabled to

allow lowercase list of string and whatnot, like this is now part of the interpreter. This was a kind of changed like deeply in the C code of the interpreter back in 2014 when we first released PEP 484. The value of typing wasn't really so clear cut yet. It was a new idea. We wanted to have this out. And we wanted to have people actually tested and say, you know, and is this something that Python could actually see value in?

So it was a little too early to make, you know, drastic changes to the interpreter just because it would be handy to have for typing. Like it's kind of, you know, icky in the sense that between here and there, we clearly have this transition period where typing just becomes more natural to use. And you know, more ergonomic and obviously the tooling like mypy matures, but it's what you have to do if you're dealing with a language that is 30 years old. Right?

So we didn't have this clean room experience because the language was already successful. It was already out there. So yeah, like a bunch of new languages now have this comfort, like have this luxury of just being able to just, you know, can look at an empty page and just design from scratch. Like we had to evolve what we had. Yeah, yeah, for sure. I mean, Python is 30 years old, right?

But you know, language that gives me sort of that comes to mind when you say that is Swift. Right? Swift took a bunch of these ideas like you have to explicitly. Say here's a nullable reference type and whatnot, but like all of that stuff was done looking back at Python, at C# at C++ and all these languages that, you know, and then like sort of put together the pieces. Right? Yeah, absolutely. Like Swift in particular, I kind of like, you know, kind of it fits my brain.

I talk with Corey Benfield every now and again about it. Like he's now working on it actually at Apple. So yeah, I do feel it's, it's a good evolution of Objective-C since, you know, kind of it's where it all Yeah, you know, started from. It's definitely an improvement. I don't want to go too far down the rat hole, But I do wish it had something like a better standard library like Python has. I feel

Like the language is good. But then when I want to do stuff, I'm like, oh, it's, There's so much missing from like the foundation of it in terms of the functionality. Anyway, long story short, because it still depends on the whole Objective-C sort of like really funky class library stuff. Oh yeah. Yeah. I'm sure I'll get mail about that. All right. So sorry, I super derailed you. What else is exciting that's coming in the near term?

Of course, like the biggest kind of feature that was most controversial, probably is the match statement, right? So pattern matching in Python, it is a huge feature. It kind of changes how, you know, some of the code that you're dealing with looks. And also it kind of introduces for the first time this concept of context specific

keywords, right? Like that's something that maybe not so many what's new in Python 3.10 posts It's kind of touched on, but just notice that you can still use match and case as regular variable names. You can still name your functions that you can still use this as argument names and whatnot and whatnot. But when the Python grammar discovers when the parser sees that, oh, this is in a place where it fits a construct that looks like a match statement,

it'll deal with that and it'll parse that successfully. That is what we could do, having this new peg parser that it started in Python 3.9 like it was released then already, but without any new features, right? Like now in Python 3.10, we're getting the match statement. I'm still kind of, you know, to be honest with you, before this area where I can tell you that I have a lot of experience writing match statements, it's like super useful for me

and whatnot. Like I've written a few of them where it actually made sense and made the code. Well, I couldn't really say shorter because usually it actually doesn't, but much more readable because it's kind of flatter. It's easier to parse. Like visually, I mean by a person. But it's not something that I really use daily. So like whenever somebody is really worried about like, hey, like what is Python becoming? It's such a complex feature.

Like, you know, what about the kids? Think of the children. Like I always tell them like, hey, like how many descriptors did you implement in your last week or month or year? Like probably none. True. And how many meta classes? Exactly. So like how many meta classes have you worked with? That's a good point. It's almost like another feature like this, where sometimes it will make perfect sense,

but most often, just like with assignment expressions, like most often you won't see them. You You won't be using those. Yeah. Okay. Well, fantastic. You know, you're talking about the foundational stuff like the peg parser, right? That's not a feature that a bunch of people are saying, you know, I can't wait for a better parser, right? But it's one of these enabling things. And I feel like this whole role that

you're in is sort of in that same idea, right? The same vein that that it might not be super public or generate a bunch of new features, but it might be making things a lot smoother for everyone who wants to come along and do those things. What do you think? The way I look at it is that there's a lot of us on the core team and even more people around the core team who are kind of, well, we call them drive by contributors, right?

Like, you know, they would find an issue, just produce a bunch of pull requests and Yeah. maybe then kind of disappear. Or maybe they're around, but like not producing more for us. And we are obviously very happy with any contribution. And the sad reality is that like, we have more of those than we can realistically review. Yeah. So the problem isn't even that, you know, you need more people to actually be able to,

actually produce value. We need people to evaluate the value that we're already getting from the community and from each other to be able to, you know, actually merge them into mainline and to see how the language evolves. Obviously every year, you know, this changes over the years how Python is developed. Like we're going to have a bunch of people who are super invested and they're going to be spending crazy amounts of time, including on weekends and

whatnot, like to work on Python, even for free. I know I did that for like, you know, a decade. So those contributions are super valued, but usually those really, those people change, right? Like, you know, you cannot really do this in a consistent manner day in day out for a long period of time. You know, your life situation changes, you know, your job changes or whatnot. And you know, like you stop contributing. And what happens

to Python then? Well, we lose some value. Currently we have over 1400 open pull requests, and I've been on a mission to kind of bring that number down. Currently, as I'm looking at it, it's 1421. I was looking at this morning here, I put it up on the screen for people watching live and wow, wow, wow. That is a lot of information coming in and not just the sheer number of open ones. But the thing that struck me is I'm reading down 26 minutes ago, an hour ago, two hours

ago, three hours ago, three hours ago, four hours ago, five hours. Like that's hard to keep up no matter what, right? Yeah. Some of those are already reviewed, like the faster implementation of Gzip decompress that you have like somewhere in the screen, like a little lower than the first one. Yes, like this on lowest one at the moment.

So, so the thing is like there's so many of those that, you know, kind of somebody actually has to come in and you know, day in day out, like butts in seats, just review the value that people are already providing for you. Like this particular change actually extracted from an older rejected change, like is actually providing clear value, right? Like now we're going to have Gzip compress and decompress be faster, right? But it is actually not change that is

obvious, right? Like you have eight files that are changed. You really need to see that, Oh, there's a lot of code that changed. Some of it actually wasn't, it was just moved so that we could reuse it better. Some of it actually has subtle changes. So this isn't something that many volunteers will be willing to do on their free time because it's mostly kind of, well, it's work. It really is work. It's the janitorial typo. Just keep it clean. Keep it going. Yeah. Exactly. Yeah, exactly.

So let's go ahead and actually introduce this main topic here, the big announcement for you. On your blog here you have, I am the new CPython developer in residence. Congratulations. Ah, thank you. I'm still very excited about this. Like back when we only had volunteers doing this and maybe, while Guido being, you know, kind of tolerated as the BDFL at Dropbox, because obviously he was being paid to actually move my

PyPy forward. Right? That was his team. But obviously he did a lot for Python as well. And that was Kind of understood by Dropbox that it's fine. And many of us were in the same situation. Like I was tolerated as a CPython core developer at Facebook and some others were at their own respective companies. So I was super frustrated by this because, you know, there's tremendous value that we're giving to the entire community, including multi-billion dollar corporations.

Yes. And we only thought like, hey, like the expectations are high and we want to provide the best tooling that we can for the industry. But it is kind of a hard sell, you know, when your kid wants to go out and you're like, oh no, I'm fixing this bug. But at the same time you're doing this for free. So you're thinking, you know, like, where did I go wrong in life? Like, you know, my values are somewhere, not where they're supposed to be.

So Red Hat actually hired Victor Stinner like full time to work on Python, but it was still Red Hat. He's still an employee of a company that is for profit. So now, Yeah. the CPython developer in residence is different, right? It is sponsored by a PSF sponsor. In this particular case by Google, like Google as a company reached out to the PSF

specifically to sponsor stuff like this. And that is amazing. I am very happy that they did, because you really believe that this is something that might, you know, kind of alter how we think about maintenance of community driven projects like Python. Yeah. And thank you to Google for doing this. It's so easy for them to say, Well, maybe we could just hire some core developers and they could work on speeding up the Google Cloud stuff for Python and we'll let them also contribute back.

Right. That would be the sort of natural historical way of doing something like that. Yes. But here it's like, no, let's actually directly sort of effectively hire a person, but put them in an independent role that just makes Python better. And that's interesting, Right?

Yeah. Like amazing thing about this is that, you know, while I do hope that some of the work that I already did was useful for Google as well, like there is no job description written by them and there are no like tasks that they're directly giving me. Right? Like it's really independent in this sense.

And this is what makes it super healthy and like, you know, kind of, I think, well, sustainable for the future that it is actually me being able to talk to the steering council, which I do regularly and to the PSF about like, Hey, what do we actually need? And we don't have to take into consideration, you know, kind of direct value for our sponsors. Like we are providing indirect value, like, but you know, we don't have a roadmap where

there are certain milestones that we need to reach like it's not like that. So I'm very happy about this difference because we did have sponsors before that did have either projects that they really wanted to move forward. Right? You know, and we still have those like, so for example, packaging and whatnot, or events that happen. Right? So Facebook sponsored, you know, core development

spins, sprints. So we had those at the campus two years in a row and then Microsoft took over and we had Bloomberg and then we had COVID, you know, so like those things do happen and I'm very happy for those. But this is a kind of a game changer. Like we have not done this way of sponsoring a project before where we're actually thinking about the software word in Python software foundation where we directly sponsor work on the code on the source

code. Yeah. The PSF has what seven, eight, nine full-time employees, but you are the first employee who is in a sort of software developer role. I think the closest we had before was a project management role for the new pip. Is that right? Yeah. So like we, we have that, like there's also E who works on our infrastructure, right? So what is also important for Python and this is the dealt with by the PSF is to keep the website running and to keep my PI running, right?

Yeah. And so on. So obviously all of this requires constant supervision. And it's also something that you can spend your life like working full-time on. And this is what E is doing. However, that wasn't contributing to CPython as a repository directly. And this is the entire change. Yeah, that's fantastic. Quick shout out to just the two other visionary sponsors. We've got Bloomberg

and Microsoft through Azure. So I think that's awesome. I think this whole visionary sponsor, Google was the first one in that category, right?

Yes. That is a change in how the PSF was looking at sponsorship before. Like, I'm very happy to see that like this entire developer in residence position was something that I was pretty stressed by when I first got to know that, Oh, this is actually going to be happening because I do believe that my particular performance kind of will make or break future ideas on whether this should be extended to more people, right? Or just closed down altogether. So it's not only providing,

value to the project. It's literally providing kind of proof that this development model works. So yeah, there's certain responsibility around it. I think I already caught a rhythm that I can sustain and I think, you know, kind of splits my own coding with actually reviewing and accepting other contributions, which are the bulk of what I'm doing. Because like, you know, otherwise, if I just focused on coding all week, I might maybe if I'm lucky and

other people review my work, make maybe five, maybe six changes to see Python. Like we're doing careful reviews because like even simple changes might have actual far reaching consequences. But when I'm reviewing other changes, we sometimes have like 50 or over 50 changes that I can actually merge into mainline. So I'm very happy about that. You know, like that that is obviously a much better deal for everybody.

Yeah, it's one of these enabling roles, not just sort of heads down coding roles, which I think is the important thing. Absolutely. Yeah. Paul Everett out there in the live stream says, you mentioned the stress from the responsibility. Agreed. It takes some bravery to do it. Lucas Lucas Lucas Lucas: Oh, thank you. At the same time, I was literally giving talks before about, you know, yeah, we should do this. So, you know, like I kind of put myself out there even before this, this was a

position. So at least applying because like, obviously it wasn't at all obvious to anybody like who is going to be chosen for this. I have this applying was, was sort of in the cards. Like I had to man up and do it, you know? Yeah, that's cool. Let's sort of explore a bit your background. Like,

It's not just that you are a user of Python. I don't know, even a really good Python developer themselves might not be a good fit for this role because it really is like closely working with people contributing to the code base and the core developers and all that. So, you know, maybe give people a sense of some of the things that you did previously to to this in the cPython space. Like for example, you create a black. I think maybe people have heard of black.

Yeah. If anything, this, this was actually probably on something that made it harder for me to get this position. Because obviously everybody is scared. Like, hey, are you going to reformat all of this library now? Like, you know. Oh, you're bringing the black guy. He's going to totally format it different. yeah. The biggest PR ever. No, like, so joking aside, like I'm not new in the project. And that was obviously important. It's kind of hard for me to

you know, kind of now enumerate the reasons why I was chosen. Like, you know, kind of, I can only tell you what I told the PSF when I applied, because that was actually like the only time in my life where I actually had to write like a proper cover letter and actually kind of, you know, do some soul searching and say like, hey, am I actually a good fit for this? And you know, I think that's my mission statement.

Exactly. Like, yeah, pretty much. So, you know, I've been around for a while. I've done some changes, big and small. There's a bunch of peps with my name on them. So I guess, you know, kind of, it's not just, you know, tiny typo fixes and introducing new bugs. It's also introducing peps. Some of them might be more clear cut wins than others. Yeah, you appear in a lot of the peps. You were the release manager as well, right?

Yes, I still am for 3.9. 3.9 is now currently already pretty late in the game in the sense that we will be releasing Python 3.9. 7 just next Monday. So probably not when, you know, the listeners of the podcast are going to, well, like they're probably going to see this already released, but that for now, like for the ones like watching the live stream. Yeah. Next Monday is when we're going to release 3.9. 7. So that's already the seventh

bug fix release of Python 3.9. There's going to be a bunch of more of those, and then it's going to graduate or retire into the mode where only security fixes are considered for inclusion. This is where Python 3.8 is already. I've been managing the releases of that. How long do you remain the release manager until 3.9 is retired or does it get handed off at some point? No, no, no. So like for, You become a release manager for the entirety of a life cycle of two releases,

right? This was true for a long time. Like I didn't dig up like, you know, kind of prehistory, but since like the Python 3 releases, I looked at, like that was the case. Python 2.7, I think was, you know, Benjamin and 2.6. I'm not sure that it was him. Like it was probably Barry Warsaw. But the thing is like 2.7 was, you know, the longest release managing job ever. So, you know, kind of you could just burn yourself out on that one release. So that's fine.

For us currently, we're managing two releases. Like, so how long is that? Well, with the development of Given Python version where 3.11 is already being developed right now, it takes around a year and a half

for it to actually, you know, get out there as the final release. We do have a cadence where we release new versions every year. But that's only because once one release becomes beta, we can already switch the branches. And now new features go to a yet new version. So currently, it's a little weird. I think that happened on your watch, right? That happened on your watch. Wasn't it 18 months before?

Yes. So 18 months of a cadence was never really a hard calendar event. Like it was more like, you know, a guideline where like, yeah, we're going for 18 months. Sometimes it was 19, sometimes it was 20. But it was problematic for many reasons. So yes, like one of the peps that I authored was a PEP saying, hey, we should change the cadence of development to make it on annual release cadence, which helps with downstream

distributors like Linux distributions. And for many other reasons, like, you know, for us, like that's in the pep. But yeah, like to just close on your original question, like, it takes around eight years from starting the job to like actually saying, oh, now I'm a retired release manager. So it's a long term contribution. That's a big commitment, isn't it?

Yeah, wow, that's amazing. Well, thank you for that. That's, that's wild. So when you touched on this a little bit, but on your original blog post,

announcing it, which I'll link to in the show notes, of course, you talked about what can one person do is the developer in residence. And you called out a bunch of specific things here about how you can basically be a multiplier effect for people who want to contribute, who are not core developers, you want to maybe like sort of give us the rundown on those things?

Sure. So like, the blog post, I'm still happy with it, because like, it kind of shows that it wasn't just winging it, like I actually thought it through, but like, you know, kind of it lists a bunch of things that are ideas, and then you actually start doing the job and some things become clear, right? So like, all of the things I wrote are true. But what I for example, didn't quite like word out in the blog post that I realized is important is that very many changes that I touch, are kind of 90% there, you know, where I cannot just merge with the

them as is, but really what I need changing is the doc string is just bad, or the comment is misleading, or whatever. So like, you know, a bunch of simple things that, you know, kind of, it feels just wrong to just review the change and say, hey, like, I request you do this small knit, you know, because I cannot merge it as is. Yeah, you got to do the line break, because this code is 99. Yeah, and 90 characters long, like you could just break that line and then commit it, right?

Yeah, so this is what I do, actually, very often right now, I just kind of take the change locally, you know, run the tests that, you know, kind of are important for this particular case, I do the change that I would ask for just myself, you know, like without actually looping in the original author, most people actually leave that tick on their GitHub pull requests, making my job easier, like, so like, yeah, thanks for that. And then, you know, kind of, yeah, maybe they forgot.

I just did a PR last night, and I allow maintainer to edit the PR. Yes.

Yes, this is something I'm using very often right now. Like, because like, if you forgot to, for example, put a news entry for this, and you've never done this before, like, yes, there's a bot for this that Mariada wrote. So like, you know, you just need to go and read how to do this, and you will be able to do this from now on. But you know, when will you do that? Maybe tomorrow, maybe in a week, maybe you're just gonna leave this pull request, you know, to go stale. So instead, I just go and do this for you.

This kind of level of multiplication really, I think is kind of, you know, closing the gap, like because very many changes, and sometimes it's just sad to look at, you know, like you have a change from 2017 that is clearly great, you know, like, clearly valuable. And then there was some knit that, you know, prevented it from being merged. And now it's 2021. And it no longer merges, because there's conflicts with other unrelated things.

It's so out of date. It's no longer just a knit. It's now like a big now.

Now it's hard to merge. And obviously, not just that, maybe that original person who created the change and worked hard on it now is discouraged or moved on to other projects. Because you know, like it wasn't merged for four years. So this is what we want to change, right? We want to make the, not just the first time contributor, but like any contributor, well, life or experience, like much better. Like at least providing feedback. Like,

Like not all feedback will be positive. Like, this is already a mature project. And sometimes very good and easy ideas actually aren't if you look close enough, you know, like, this is the frustrating part where even kind of clear cut advantages might be disadvantages for a different group of users, right? Or on a different platform that you're not a user of, but you know, we have to actually look at the big picture. But at least you're going to know from somebody like me that, hey, like,

This is good, but we cannot take it because of X, Y, Z.

You know, it's super frustrating or disappointing to put work into doing a PR, submitting the PR, and then it's not that it was rejected. It gets no response. And that's, I think, really frustrating. Like I spent a whole day because I was thinking this would make this thing. I almost works for me, make it perfect for me. And here's the thing. And the answer could be no, it's not the right direction, but no answer at all is just like, ah, I'm not sure I want to do this. You know? Yeah.

Related to that sort of, Rodrigo out in the live stream has a pretty good question. I think let me throw that out to you. And you know, what are some of the advice to potential future drive by contributors? How could they make your life easier? Well, like in the future, we'll move to GitHub issues. So a bunch of this will become easier for the contributors themselves. But currently that we still have bugs, Python.org. There is a required step, pretty much for the

required step pretty much for anything that isn't a trivial typo, which is you need to have a bugs, Python.org account where we know that you signed the CLA. Right? So the contributor license agreement saying I release my, you know, code under Apache, for example. Right? So that we can actually use it and merge it and it's fine. So this is something that, you know, kind of is the important original step. And once you're already on bugs, Python.org. And your change is not trivial.

Please click that, you know, open new issue button and write what you're doing and why. Maybe it's already there. Maybe you already know that you're fixing a particular thing that was already reported by somebody else. But if not, like spend this five to 10 minutes describing what is happening and why you want the change applied. Very many changes simply do the thing that they want to do. Many of them don't even write in the pull request name, like what they're doing. It's simply the original GitHub update file.

Which really, you know, is a puzzle now for anybody else because we don't really know why this thing was touched in the first place. So essentially just describe like where you're coming from. Like what is this change doing? Like that is already, that's already great. That already kind of gives context to us.

Would you suggest that people create an issue and get a little bit of buy in from you all before actually doing the change and the PR? Like, would it make sense to say I'm considering this? I'm happy to do a PR. Here's the plan. Would you accept this if I did it? Depends, right? Lucas Lange: Okay. Because like that is the worst centrist, bleak answer that anybody can give. Depends, right? When you're doing bug fixes, you should just go ahead and, you know, do your best and make the bug fix.

Yeah. Here's the failing test. Here's the fix. Yeah. Yeah. But if you are saying, hey, this library should have this new function that I am using in my own project. So I just want to put it in a safe functools or whatever. So before actually going ahead and creating that pull request, like it's probably might sense to discuss it with us because a growing standard library, well, isn't so clear cut a value as you think, right? Because it's a maintenance burden. Yeah, absolutely.

And more importantly, you know, now it has to consider all the usage groups that the original author might not care about, but we have to, we necessarily have to. So yeah, for features, definitely file an issue, you know, describe your problem. You might actually, if that makes it easier for everybody to discuss, just submit a pull request saying, hey, this is the feature if you want to talk about it. So we actually talk about something concrete instead of an idea. But if you, for example, say that, hey, we need to rewrite multiprocessing.

So it does this other thing instead of creating a multi file gigantic pull request, like let's discuss first. Talk Python To Me is partially supported by our training courses. At Talk Python, we run a bunch of web apps and web APIs. These power the training courses as well as the mobile apps on iOS and Android. If I had to build these from scratch again today, there's no doubt which framework I would use. It's FastAPI. To me, FastAPI is the embodiment of modern Python and modern API.

You have beautiful usage of type annotations. You have model binding and validation with Pydantic and you have first class async and await support. If you're building or rebuilding a web app, you owe it to yourself to check out our course modern API's with FastAPI over at Talk Python Training. It'll take you from curious to production with FastAPI. To learn more and get started today, just Just visit talkpython.fm/ FastAPI or email us at sales at talk Python dot FM.

As you get better and maybe automate or go down as you hit the heart of Another question out there I think is good from Paul is your PR review velocity. Do you expect it to go up as It's like clearing out your inbox a little bit, right? Like some of them are using Hard to say. Yeah, I think it's a little bit. Yeah, I think it's a little bit. Yeah, I think it's a little bit.

Like, oh, that's a whole day. Oh, no. Essentially, it is not very easy to kind of, you know, foresee what's going to happen a given week. But one thing for now has been constant. Like, you know, I need to get the number of pull requests down like below the number that we have right now because like there's there's going to be some that are on page like 35 of the, you know, open pull requests page. So I will probably never get to them otherwise. So I need to bring the number down.

And so far, just by myself, I was unable to do it. Like just I cannot do it. Like I had one week where I really said like, hey, just, you know, 40 hour work week be down. Like I need to see if I can really bring this number down, you know, and I almost merged 100, you know, so that was not

something I can do every week. And I still didn't meet the, you know, the goal because I did leave the account below 1400 when I went to sleep and I, you know, woke up the next day and it was already over, you know? Yeah, we saw the rate of incoming ones. That's a lot. Yes. So I'm doing around 50 ish a week right now. And you know, kind of it sounds super high, but you know, some of them are backboards to branches that we maintain. I list those because some of

those backboards aren't trivial, right? Like they will conflict because, you know, the files changed in between or they require some push, you know, from a person because the bot didn't do what it's supposed to do. So, you know, it's also work. But 50 isn't really, you know, kind of me kind of flexing. I can do so many changes. It's really the rate of change that you can see. So there are enough new changes,

right? That I guess I can always forever just stay on page one and just deal with those. But that would feel unfair, right? You know, that, you know, kind of that is some reversal of priorities where the last that comes in. Yeah. You almost need to go to the last page and work your way forward. Exactly. So, so sometimes I would look at the older ones, you know, basically just asking a interesting query to the search, you know, pull request search on GitHub and see what's there for, for example,

this file or, you know, what's already approved and that I can, you know, kind of maybe push over the line and so on. So there's going to be some balance there. I don't expect to go any faster than I currently am because I can't really automate much there. Like there might be some instances where if we analyze the pull requests better, we find that, oh, there's an entire genre of pull requests.

We can just bulk close, but I can't really, you know, okay, now I like put this on the report. Like it's like, oh, I closed 1500 pull requests. I'm so great. Now, like the ones that I put on the weekly report are all that I like with my own eyes look at and review. So I don't see how automation can really help me, you know, do that faster. Like we need to be careful, right? Like it's still a language used, you know, for some important stuff. And, you know, in

Yeah, it's still a language used in our industry and beyond. So, you know, kind of a higher rate of change sounds nice, but it also, you know, is risky, right? If you don't do it responsibly. So I think that the 50 ish number that I'm currently at, like, is enough that I don't feel like, you know, I could be doing more. It's around what I can reasonably with a clear mind, like review day in, day out. And I still have some time to actually code, which is also fun because now you're contributing something of your own. But yeah,

Like I don't think I can get to 100 every week, like regardless of the automation I can come up with. Yeah. And maybe they get harder. Oh yeah, they might. Because you could go, all right, this one, I can quickly close this. I can quickly close this. But as you sort of cycle, iterate over that list, like you get stuck more with the hard ones. Yeah, absolutely.

Yeah. So you mentioned the weekly report, you're doing a weekly report that's just out in public. So for example, I'll link in the show notes to your weekly report for July 26 to August 1st. Was this like your second or third week? Something like that? Yeah. So if people want to keep up with what you're doing, it's right out there, isn't it?

It would be out there anyway, right? Because like, even if you don't look at the reports that I'm writing, like you can still go to my profile on GitHub and just see like what the stream says there. And if it doesn't say anything, like, you know, that's kind of worrying. It's like, hey, is this guy even working? What is he doing? Check your Instagram, see if you're on the beach, just hanging out.

Exactly. And I was actually last, just last week. Yes. So the weekly reports are kind of, you know, thought of by the way.

I mean, personally, since like the PSF wants me to blog about what I'm doing, but not at this cadence. Right. But I personally just want to kind of tell you what I'm working on. Because ultimately what I'm doing is work on an open source project for the community. Like Lily hired by the Python social foundation. So there is no agenda that is personal, like, you know, to me or driven by some corporation. Like I'm literally maintaining this for you. So like, why not just let you in?

On what I've spent my time on most of those I expect are going to be pretty boring after a while. Because like a big part of it, like at the bottom is always just a list of PR so we can actually click through and see like, Oh, what did, what did I do there? But you know, I always try to like keep it interesting by, you know, kind of finding some highlights that, you know, I found particularly interesting a given week.

Yeah, I think this is really valuable. I don't help people a lot to sort of see what the role is doing. So originally this role was based on Django's fellowship program, right? Oh, yes.

Yeah, that's really cool. So you're able to, Django had this idea of bringing on people to help deal with the lower, not lower level, the more maintenance oriented stuff, right? Like exactly what we've been talking about. And I know one of the things you talked about somewhere in one of your articles. I can't remember which.

Was basically, you're very excited about this role, but you're also a little nervous because you want to make sure that you're the first one doing it and you want to make sure that it is going strong for the next year and the year after and whoever comes along you want. This is a good thing. You want to keep going. Right. So that's a little bit. Oh, yeah. You're kind of breaking the ground there, right?

Yes. So this is important for many reasons, obviously. Right. Because like, you know, we can go ahead and talk about like how this can transform Python if there's enough people doing this full time and whatnot. Yeah.

And this is all true. But you know, there are also kind of more like down to earth reasons why I would like more people to be able to experience work as I am right now, because it's just a wonderful way to work. And not only am I working on something that was my hobby for like a decade now. So I can just day in day out, just kind of, you know, I feel like I'm just kind of just doing the hobby right now. So it's amazing. But also, and this is something that, you know, like kudos to

Google, like the sponsorship level for this is Bay Area level of salary for a software developer. So like, it doesn't really matter that I'm here in Poland, you know, kind of where very many companies would apply some location specific salaries to a position. And now like this is Lily kind of the like, well, like I don't want to say premium grade, but like industry strong salary here. Like the sponsorship level for this is Bay Area level of salary for a software developer.

So like, it doesn't really matter that I'm here in Poland, you know, kind of where very many companies would apply some location specific salaries to a position. And no, like this is really kind of the like, well, like I don't want to say premium grade, but like industry strong salary here. It's not like you had to go and say, I'm really going to take a hit, but I really want to do this. So let's just do it. Right. So it's a proper job. Not at all. Not at all.

Which is exactly why I think, you know, it's both important to the project, but also important to all those people who are already contributing. They're already providing value. And now if they got paid, like, you know, actual industry strong rates for their work, like this has the overall kind of opportunity, or this has this enough value that it can change a person's life, right?

Like some of our contributors are not in the US or in countries that are like, you know, I don't know, Eastern Europe or East Asia or wherever else. And there, like this kind of amount of money, like might actually make or break, you know, kind of somebody's, well, entire livelihood. So it's amazing that this opportunity exists. At the same time, like for now, like, you know, let's not get ahead of ourselves. Like this is a contract for one person for 12 months, right? It might not get renewed.

Like we might not get a sponsorship for next year. So the PSF might actually have to be on the lookout for more sponsors later, or maybe for a new developer, if they just don't feel like, you know, kind of the current one is doing the job like well enough. So like, you know, all those things might still be in motion. So I myself don't really kind of, you know, already put this in the bank. They're like, Hey, like I'll be doing this now for like, however long.

Like now I'm really spending this first year of really producing like the most I can, a reason we can't, right? Because it might just be a 12 month gig that I will be having like fond memories of, you know, in a decade, or maybe it's a start of something much bigger. So why not take the chance and just, you know, try your best? Yeah, absolutely. So let's dream just a little bit and then maybe I'll put a call out to it. Would it make sense?

Would you want, could you do more with like a team of people if there were five developers in residence? So would the PR look like 20 open ones, the oldest one a month open, things like that? Like, would that be good? Would you want to see that? Having five people available for work like this would entirely change how I'm approaching this because currently, like, I don't really have a much in terms of coordination with others, right?

Like I just look at whatever current work is happening and try to help out there. But if you had five people, you could actually hit this problem head on. Like, you know, we have 1400 open pull requests. Like, it's not a question now, like, if we get them down to zero, but like, but when, right? Now we actually can plan to do this, you know, like methodically. And a lot of bigger style things that we always wanted to have in Python, we could now just plan to implement ourselves.

Now, if I'm just the only person doing this, it feels irresponsible to just, you know, hide somewhere, you know, in a basement in Poland and start churning out my favorite feature, right? Like, that's not what this world is about. But if you have five people, you can reasonably expect that some of them might actually work full time providing new features or finishing unfinished ones, which there are plenty of for inclusion in a future version of Python.

It could be somebody says, I'm the one who just goes to all the bug PRs and just checks those. And there could be somebody working on performance and like, you could specialize and things like that even. Oh, you specialize, but I really kind of think that if all of them are developers in residence, it would be great if they like kind of had like this on-call rotation for doing things as I'm doing now. So just responding to the external community, right?

Because like the important thing that I wrote in the original post and I still believe in is, you know, like the drive-by contributor experience is extremely important, right? So like we would always have to have somebody doing this, but at the same time, nobody really wants to do only that. Like everybody has to provide, well, wants to provide value of their own so that they can, you know, touch it and say like, ah, I did this. This is good.

Yeah. So yeah, I already have a bit of this, you know, like not as clear cut, not sponsored by the PSF directly, but Pablo, the release manager of Python 3.10 and 3.11 is both sponsored by Bloomberg to work halftime on Python performance. And also by him doing the release managing job for Python 3.10 and 3.11. He coordinates with me very often right now, especially that now we are in the release candidate stage of Python 3.10.0. And the greatest thing about this is that he's in London, right?

So yeah, maybe, maybe far if I were to walk to him, but in terms of time zones, that's amazing. Like he's just one hour away. So, you know, we mostly have very synchronized work days. So I would talk to him like essentially every day. And that is also something that I missed doing remote work before, because, you know, kind of if you are working for a company that is in the Bay Area and you're the only one in your time zone, it kind of gets lonely, you know?

Having more people around that now I have like Pablo, I have Victor, other, Sergey, Storchaka. And most of our communication is async, right? So it's not like everybody's talking when I'm not around. Like everything is happening on the mailing list anyway. So it feels like I'm much more integrated into the team compared to my previous stints at, you know, doing remote work. So I enjoy this a lot. So it sounds like it would make a big effect. That's fantastic.

We look at that list of the sponsors, the visionary sponsors, and there's three companies doing really good stuff. But you gave a implicit sort of call out to this. And I've said it plenty of times. There are many companies out there who their entire core of their business is built upon Python and they make billions of dollars. Shouldn't they consider contributing something to build that foundation a little bit stronger, right? Like maybe make Python a little faster.

Their developers go a little, you know, more efficient or whatever. Yes. So like the good thing about this is that, you know, we always can use more money. So obviously we need more sponsors. Many of the companies are trying. Simply the thing that they need is, you know, some sort of like plan or well, not like a legal contract, but at least some sort of understanding what the money is going to go for. What do they get out of it, right? Because in their balance sheets, they have expenses.

And they have revenue. They don't usually have charity. Right. It's tricky for them to make it fit. I think it's kind of, you know, now with the developer in residence position, this kind of changes the game because you're no longer just sponsoring the Python social foundation and kind of, you know, yeah, you might have expectations, but maybe that will go for PyCon. Maybe they'll go for sponsoring of events or minority groups or whatever else.

Like if your company really is interested in providing, you know, like support to the core development, like now you can. And I think like, well, hopefully like that is my hope. Like we're going to see more sponsors of that kind of. Yeah, absolutely. People who are actually interested can go to Python.org slash sponsor slash application. And it shows you what the tiers are, what they cost, what you get.

You get a lot of inside access and a lot of visibility into Python at the visionary level that otherwise you wouldn't get. So there's, there's something to be said for that. People should check that out, right? You got to meet with the PSF board. A lot of cool stuff there. Workshops, job fairs. I think there's a lot of things on here that could go down as this is a legit value. Get a meet with the Python steering council, the sprint recognition and so on.

So I just want to encourage companies out there. Like we don't need tons of them. If we had four others, right? You said that would make a huge difference, right? Absolutely. And also kind of like, let's be clear, right? Like, you know, kind of if a sponsor of a developer in the residence job, right? I came in with like a laundry list of like, this is the things that I want you to do. Like we'd probably have to kind of, you know, come to an understanding that this is not what this is about.

But if people who happen to be employees of a company come in with changes that make sense and actually provide value for us, and now we have people to do full-time code review and inclusion of those changes, it's more likely that things that you care about are going to be included, right? So you can be ethical and impartial to, you know, kind of, well, sources of, you know, money flowing into your account, like, and also provide value to the sponsor. It's not a zero-sum game.

Like, you know, let that be clear and explicitly said, because I feel like this is also important to know that if, for example, there is a company that really, really focuses on an operating system, like, I don't know, maybe they're running everything on Windows or macOS, right? Like, you know, and now they're like, hey, like, we really need somebody to make AsyncIO work better on Windows, or we need to fix this fork problem on macOS once and for all or whatnot.

Like, you know, if they provide, you know, some changes and now there's people to do the review, more likely than not, we're going to have better support for Windows and macOS. Like, you know, everybody wins. Yeah, absolutely. Another benefit I'll throw out there, just thinking about this, is a lot of companies have, like, continuing education programs and credits. Like, hey, you have a bachelor's degree. Do you want a master's degree?

If you're willing to go to school outside of work, we'll provide that for you. And the reason is, now you have people who work for you that have master's degrees and not bachelor degrees, right? So this could be a similar thing, like, hey, we want to provide a position for people to come work as the DIR or part of the DIR, and then maybe they come back and work for us afterwards. But they would be so much more knowledgeable about Python, having spent a year in this role.

There's a lot of benefits like this. You know, you won't believe this, but like, obviously, the project is a million lines of code, right? Like, half of it is in C, half of it is in Python. So I don't know about most of it, right? Like, I've seen much of it, but like, I probably don't have expertise in most of it, but nobody cares when they submit pull requests. So then I look at them.

So for many of them, it's my first actual stint at, you know, like touching a part of Python that I might know as a user, you know, from the outside, right? Yes. But then looking at like, how does Dunder main actually work in Python? It's this special module that, you know, like your execution starts with, like, how does this actually work?

So you go and see, and you actually kind of dig and you understand like, ah, okay, like, I understand this because you have to take into account, you know, starting your Python interpreter, like, you know, with, I don't know, code from the standard input, or maybe somebody is starting a ripple, or maybe you're from a file, or maybe it's a zip file or whatever. So now I know more than I just knew like two weeks back about Dunder main. So very true.

Like, you know, you bring somebody in who is motivated and maybe knows a bit about, you know, what CPython already is built on. But necessarily, you're going to get experience in parts that you would otherwise not touch, right? Because now you're reviewing the bulk of pull requests that are coming in, not the ones that you would write yourself. Yep. Absolutely.

I remember one listener sent me a message and said, I was really surprised to hear you say there was like, this whole section of the standard library you didn't know about. That describes a lot of, there's a lot of stuff out there, right? You might just barely touch it. You might have no use for it. There's a lot. You would, this would force you to just give that depth. It's big. It's big. Absolutely. So Rodrigo asked previously about what drive-by contributors can do to help you.

What can, you know, this is speaking to a smaller audience, of course, but still I think it's relevant. What can the core developers and the steering council and the people on the inside do to help you be more successful and efficient? Most of them know me. So the most important thing for them, and I really mean it, is like, just tell me what I should be rather doing.

If you feel like there's something else I should be touching on, or if I did something that you don't like for any reason, just write me, just tell me, right? Like, because like, it's different from the external contributors. Most of the core devs I know one way or another, so we can be direct with one another. And I think like the make or break of this position is for me to provide value for both groups.

So if there's anything that a particular person needs or wants, or even feels like, Hey, you know, you're focusing too much on this thing or, Oh, have you looked at this? Like, this is not something you've touched yet, but this is extremely important. Just let me know. So communication is key for us right now. And another thing that I kind of still don't quite know how to tackle is that I don't feel comfortable just now, like assigning a bunch of things to volunteers. You know what I mean?

It's like now there is a bunch of pull requests where they're fixing something that actually broke because of a change that some core developer did five years back. So of course they have the best context on this change, but I feel kind of icky just, you know, going and doing this 10 times a day, you know, because it really feels like now you're being this manager of Python telling people, Hey, go fix your shit. Like, no, that is not what I mean. It's like, as I said, the project is huge.

And, you know, very often it is really a bunch of people, like a low number of people who have the best context on given part of Python. So yeah, like if I do this too much again, like, you know, tell me if I'm not doing this and instead of going ahead and doing changes, you know, that you don't like in your particular favorite piece of Python also let me know, like, this is still something I'm pretty new at. Like, I'm just going to like issue my second invoice like next week.

I'm pretty excited about that, but you know, like it's still a new thing for me. So I guess, you know, for the core devs. Yeah. Let's talk like whenever something is clearly well, improvable, if that's a word, like let me know. Yeah, absolutely. How much input into this role did the steering council have and how much did Guido have? Or was this a PSF plus Google sort of thing? So I actually spoke with nobody from Google.

So directly, I don't know the people behind this position, like on their side. I know a bunch of core devs who happen to work at Google and they're very supportive of the position. So that's the extent of it. The steering council meets with me every second week. So it's mostly kind of me talking, hey, this is what I spend my time on. You know, they ask me like, how can we improve your output or whatever?

Like, you know, or just they will, would just tell me about things that, you know, they find interesting or important that I might look at, but it doesn't feel like they're really kind of going and telling me what I should be doing. Like, you know, point blank. They're just having like suggestions on and how this role should be moving. So this is kind of, this is a short, well, 20, 30 minute meeting every two weeks. So kind of high level.

I do have one just like it with Eva from the PSF and E. So it's really, you know, kind of the Python server foundation internally also needs to communicate because they have all the keys to say, being able to restart a bot, right? Or, you know, kind of make changes on a given server or maybe change some workflow on GitHub that affects everybody and so on and so on. So we just also meet every second week to make sure everybody's on the same page. And also there's some very low level stuff.

Like for example, figuring out like, hey, what calendar am I working on? Like is Labor Day a day off for me? Or is some other holiday that is here in Poland, but not in the US a day off for me? Or don't I get any day off at all? And I work on Christmas Day. I don't know. So all of those, all of those things, you know, have to be actually discussed with the PSF directly. Yeah. And Kordevs, like, you know, kind of, we do talk a lot.

And Guido in particular was very supportive of me starting this role. So I'm very grateful for this because it's kind of like, you know, kind of just getting, getting a little bit of, you know, like an email, just a bunch of sentences, just actually kind of saying like, hey, I believe this is a great decision. You're going to do great. Like, okay. I don't know. Like, I still feel like I'm a beginner in a lot of this. It's really reassuring and kind of lifts your spirit.

So, so yeah, like, I guess we are in constant contact using super old ways like email and discourse to an extent. And now like VCs. So yeah, like we're doing Google meets and Zooms with others. They actually have too many, well, communication methods right now. So like me talking with Kordevs, I'm on IRC right now so that others can actually reach me as well. Others would talk to me through WhatsApp and whatnot.

Like I have like 10 chat, like message apps right now open just to be able to talk to everybody I need to talk to. So that needs fixing, but I don't know who is supposed to do that. Yeah, I don't either. It's a growing problem, not a shrinking one, but it sounds like you're doing a good job. Even Rodrigo out there in the live stream says, listen to this hypes me up to go and contribute to Python.

So hopefully that sentiment is multiplied across all the people who are contributing work that, you know, it'll, it'll get more attention sooner. So well done. Absolutely. I'm happy to hear that. Yeah, absolutely. All right. Well, we're about out of time to talk about the new role. So let me ask you the two final questions, which I've asked you before, but it's been a while since you were on the show. So time for an update.

If you're going to do some Python, you know, you always run across different external packages, things on PyPI and whatnot. Have you found one that you're just like, wow, this thing is amazing. We should talk about it, get more awareness of it. It's already growing, so I'm not going to tell anything that is like now like extremely, you know, novel, but I'm extremely digging rich, right?

Like the kind of text formatting, like library for CLIs, like anything that we write, like would use their progress bars and, you know, a bunch of other stuff that that's there for column formatting and whatnot. Like I find it very approachable and the results are like, yeah, this is a console. This is a text console. Just to be clear, right? Like it just looks fantastic. And the code for it is very nice. Like it's, it's a very declarative. So Rich, I like a lot.

And Will, the author of it is now working on a text UI library that is much more Pythonic than the previous propositions that we had to say, like, you know, a curses or you were with. So yeah, like I'm hyped for that as well. It's called textual. Yeah. Textual looks really interesting. Yeah, exactly. So those would be my easy propositions for the day in terms of third party packages.

Cool. The one that I actually thought of Rich earlier, when you talked about the better error messages, there's a cool article called FastAPI and Rich Tracebacks in development that talks about how to take Rich and then plug it in. So whenever there's a stack trace, you get like super colorized error messages and all sorts of cool stuff. And yeah, pretty neat. Pretty neat. This changes a lot, actually, because, you know, I find that, for example, between using GCC, right?

The C compiler and using Clang, like just the fact that it colors my, you know, errors, like, you know, tiny thing. But it makes Clang just feel nicer to me because it just values my time. It just makes me, makes it easier for me to read the relevant parts of the message. So this sort of thing, like might just feel like, ah, you know, this is not really that important in my app.

But like, it might save you a lot of time later on when you're like actually debugging stack traces that you're looking at like at 2am or whatnot. So I think this is extremely important, like not just because it's aesthetically pleasing, which is important in itself, but also functionally, right? Like, you know, this is the reason we are using syntax highlighting, right? Because it actually lets you see more information at the same time, right?

So absolutely, like Rich provides this in a very smart way, like to your application. So not just your Python stack traces, but whatever you're displaying, like might be like this. It will automatically format numbers. It will automatically highlight, you know, dates and files and file paths and whatnot and whatnot. So tremendous value added. Yep. Tables, all kinds of cool stuff. Yeah, it's definitely getting a lot of love out there.

And then if you're going to write some Python code, what editor are you porting these days? I am deeply invested in VS Code right now because it is in a very, very peculiar position where it's actually very good for CPython development, like core development, because it's both a good C editor with, you know, support for just jumping through definitions and whatnot. And also like has a great Python plugin, right? Like, you know, PyLens, right?

So it's both a type checker and, you know, this language server that does a lot of help, like for you specifically. So it's just a good package all around, like to just do a CPython core development. Is it my perfect editor? Well, almost like if the Vim plugin for it was official, officially maintained and a little more sturdy, like I would be the happiest person there. But well, clearly this is super low key feature. Like it's not a very popular thing.

Must not be because otherwise, you know, like looking at what Microsoft is now doing with GitHub and whatnot, like that would have been implemented if more people asked for it. So I might be of a dying breed of Vim users. I don't know. Still, like gets the job done, like for with the current Vim plugin and system, like, you know, with the window splits and whatnot. I'm very happy with it.

And now seeing that my development environment can actually be run in a browser, like what else could you ask for? It's pretty amazing. Yeah, exactly. You're doing your podcast live stream recording from a browser and you're editing in a browser. All right, Lucas. Well, it's been great to have you here. Final call to action. You know, people want to learn more about the DIR. Maybe want to help you out or consider sponsoring this in the future or more people. What do you say?

First of all, if you are a Python user, like you are very, very welcome to bring issues to us when they happen. Right. So whether this is on bugs, Python, the dorg, obviously a very good place so that your report will historically, you know, maintain there and we can have a discussion that is open and actually work towards a solution. But also in informal ways. You know, if you reach to us via RSC or ask stuff on the mailing list or discourse, like we're very happy to hear your feedback.

So you don't have to write code like on day one. Don't worry. You might never write code for us to fix your problems. Just report them. It's great. If you have stuff that works very well for you, you might also spend a minute, you know, telling us because this is something that nobody gets enough, you know, in this industry. We only get bug reports, you know, and yeah, like if you do want to contribute, there's many ways to do this. If you want to code, absolutely.

Just don't think it's super, super trivial to get large changes into a 30 year old programming language. I could make an entire talk just about that. Right. Every little change has huge implications for some small percentage of people, but there's millions. And so that's a lot. Sometimes very surprising ones. Yes. But yes, absolutely. We need more people doing the work and there's plenty of it. So you might still have tremendous impact for the world.

You know, maybe your code is going to run on Mars. You don't, you never know. Yeah. Even if you're just reviewing PRs and making the little tiny fixes or stuff like that, that could be a big contribution. Totally. And finally, for the current sponsors, thank you. You make this possible. So like, really, thank you. And for people and companies considering sponsorship for next year, if there's anything that you're not seeing that you would like to see that would convince you that it's worth it.

Also let the PSF know, like we might adjust what we're doing so that you're more comfortable with sponsoring because really like, you know, this is a very unique programming language in the sense that it is community driven and community owned. So yeah, please talk to us. Yeah. Awesome. Well, thanks so much for being here. Catch you next time. Thank you. Bye-bye. Bye-bye. Bye-bye. Bye-bye. Check it out for yourself at training.talkpython.fm.

Be sure to subscribe to the show, open your favorite podcast app, and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss on talkpython.fm. We're live streaming most of our recordings these days. If you want to be part of the show and have your comments featured on the air, be sure to subscribe to our YouTube channel at talkpython.fm/youtube. This is your host, Michael Kennedy.

Thanks so much for listening. I really appreciate it. Now get out there and write some Python code.