It's been a fantastic year for Python. Literally, every year is better than the last with so much growth and excitement happening in the Python space. That's why I've asked two of my knowledgeable Python friends, Dan Bader and Brian Okken, to help pick the top 10 stories from the Python community for 2018. Join us on this episode of Talk Python To Me to Count Them Down. This episode was recorded on November 27, 2018.
Welcome to Talk Python To Me, a weekly podcast on Python, the language, the libraries, the ecosystem, and the personalities. This is your host, Michael Kennedy. Follow me on Twitter, where I'm @mkennedy. Keep up with the show and listen to past episodes at talkpython.fm and follow the show on Twitter via at Talk Python. Brian, Dan, both of you, welcome back to Talk Python. Thank you. Hey, thanks. Great to be back again. Yeah, it's great to have you back as always. It's going to be so much fun.
And it is time to look back on 2018. Can you believe it's the end of 2018? I just remembered how to write 2018 and not 2017 on all the stuff I have to sign. Yeah, it's gone by fast. It definitely has. It definitely has. So it's good to have you back. We're going to go through what the three of us have decided are some of the top stories in the Python space for 2018. And I just want to quickly throw out there for folks listening. This mostly does not include data science stories.
There's a little bit here and there, but it's mostly pure Python. And the reason is, I'm also doing a show on data science year in review. So save those big stories for that one. Don't want to overlap. All right. Well, we're, we're going to start and do a top 10 countdown. And we're going to go from really interesting to the absolute big news at the end. And you can see what story we have picked to be the number one story of 2018 for the Python space. Let's do a little warm up here. And Dan,
I'll let you kick this off. What was our number 10 item? Sounds good. So our number 10 is the Python 3.7 release and more specifically the performance optimizations in there. And I know you had a really good show on the podcast here with Anthony Shaw, where you guys talked about, you know, some of the changes in 3.7. And I just thought that 3.7 should be on this list because it was a really, really good release. You know, there's a lot of like
free performance improvements. It's always nice to see that. And it's just been awesome to see Python 3 and see Python 3 specifically gets so much love and that it's under active development and it's making this forward progress. And I think it's also the release, or I think it was either 3.6 or 3.7,
where Python 3 is now faster in all of the benchmarks than legacy Python, except for startup time. So I thought, you know, that's kind of a nice milestone for Python 3. I think that's a huge milestone for Python 3. I mean, this is sort of the year that the tide has turned turned for Python 3 in a big way. I think it was starting to turn before, but really this year, I think that's, that's one of the themes. And this is a major aspect of it, right? That Python 3 is now
faster. There were always these couple of reasons people say, I'm holding out to Python 2. You guys, with your fancy Python 3 and your new keywords can just go play because my code is faster and faster beats everything. Right? Well, maybe not. Yeah. Well, I mean, faster never really beats everything. It's, it's faster development time too. And when you have a better data structures, you can sometimes
develop faster. However, you're right that a lot of people were holding out because of the, some of the speed and, you know, and for certain, some applications that makes sense. But yeah, I think 3.7 is definitely faster. And, and the other thing is a lot of people had already, some people for some of their projects had already switched to Python 3.6. And that was probably from 2.7 to 3.6 was not an easy switch.
Having a switch from 3.6 to 3.7 be like, oh, I don't have to do anything. It's just better now. That was a nice thing to have. Yeah. That's super nice. And I think that's a really great point you raised, Brian, that, you know, careful what you're optimizing for, right? Like if it was really all about speed, wouldn't we just be writing assembly code still, you know? Yeah.
I bet you, I bet somebody good at writing assembly could crush the performance of Python or a lot of C++, but you know, they would release an app, a small app every 10 years and it would be buggy, right? Like there's just, it's not the most important thing. That's, that's for sure. One of my, my favorite examples there is, what's this, it's, what's this game called? it's rollercoaster tycoon, which was written by one person in wind 32 x86 assembly. It's like a hundred
percent assembly code. And it's actually an amazing, amazing game. And it's, it's just, it just blows my mind. Like everything is, was written in assembly language. Wow. And it probably runs like totally smooth on like a 286 or something, right? Yeah. No, I think you needed a Pentium or something, but, now somebody's worked on like a Python going to need, I don't know, multi-core CPUs and whatnot to run at the same performance.
Yeah. Who knows? But you know, another, another interesting thing that's coming out is more and more talk about actually compiling Python, which I think, you know, didn't make the list, but it's definitely something that, that came up like my Pisces, for example, and Cython and whatnot. Yeah. Yeah. So if we're going to release a new library or an awesome game, like roll rollercoaster tycoon or something like that, and we were going to do it in open source, probably the main full release
after it's like totally stable and spent out for a year, it'd be like a zero dot. Oh one or, or O three, maybe an O three, like a zero dot 12. I don't know, something really, really small, right? Like that. Right. And that's been a theme that's gone through a lot of, a lot of open source. So I think I'd like to highlight a little bit of shift in just the versioning that seems to be
happening around projects. I want to kick off this part of the conversation with talking about Mahmoud Hashemi's project called zero over Brian, do you remember covering that on Python bites? Yeah. He released it like as an April fool's thing, but it actually highlighted a lot of, a lot of things about different projects that were zero over after like, you know, years. Yeah. So the idea is kind of like I was describing at the beginning, like so many projects just never
even reach version one dot O. And in some sense, it doesn't matter, right? Is the version number smaller or bigger? That tells you whether you have an older or new one, right? So that's fine. But there's a lot of people, especially coming from the enterprise space or from the commercial software space that
sees things like zero dot 20 and go, Oh, that thing's like a super alpha. We're going to stay away from that until that reaches one O. And I think it has a tangible effect on turning off a certain category of folks, especially people who are maybe unfamiliar with open source and things like that. Yeah, I think definitely. We, I mean, a lot of people do a commercial software development and like I do, and, the zero, zero versions always, or almost always mean
the API is in flux. You can't trust it. You shouldn't program against it because it can, the developers can change their mind at any time. And it isn't until the 1.0 version where we have a, a stable, a stable API that people can count on. Yeah. And so I think a lot of people perceive it
that way, right? Yeah. And it's interesting to consider the version number as part of the user interface, you know, for, for an open source project, which makes absolute sense because of those, those effects where people are like, yeah, you know, is this, is this really ready to go here?
Is this an early release or whatever? I mean, I'm wondering if it's really going to have a big impact, you know, if people are moving to something like, like dating their releases, like just calling them 2018 dot something, something, and you sort of get the November release, but it's,
it's an interesting experiment for sure. I mean, it, it definitely looks a little bit nicer than, you know, having, I don't know, like somebody put pandas, pandas here, which had like 83 releases and it's been running or it's been active for eight years and it's sitting at 0.24. 0.0.dev. 0.dev. 0.0. Nothing wrong with that, but I can see how it turns some people off. Yeah. I don't know how dev zero means production, but you know, to me that just says like, dude,
stay away from this. Don't ship this. That's really scary. Yeah. I think the, you know, maybe the grandfather of zero for it might be putty, you know, that little, SSH tool you get for windows cause it doesn't come built in. that'll trade app. it's 20 years old and it's 0.7.
One concern that I have with moving to the date based or like year based version numbers is how, like if you have the semantic versions, you can do things like, okay, I want, give me all the minor patches for this package and all the security updates, but I don't want to jump to the next major version, which maybe has a breaking changes in it. And I know that's not a perfect system. in general, you want to pin your packages, but it's kind of nice to have the ability to do that.
And with, you know, who knows what's going to happen in like July, 2018 or something 19. Yeah, absolutely. So if you look at some of the tools that I know you use and I use, for example, I use pyup.io and use something slightly different, but they let you go to your requirements file or your pip file and say, I want you to keep this on the latest pin version possible with this constraint where you could say like 0.1.star or something like that. Right.
Right. Yeah. But like, how would that work with, you know, date based. Exactly. So let's dig into that a little bit. So what are the two options? I guess the two options are just like to start shipping like full versions, like Django is now Django two and the Django two, the main change from one to two is we're dropping Python to support, which is kind of ironic. Maybe it should be Django three, but whatever, you know, it indicates a major change, like you were saying
there. Right. So one of the projects that's switching to calendar based version is Sanic, right? The async web framework. And what they're doing is they're trying to ship a version, a major release every quarter. So they might be, you know, 2018.07.0, then .1.2 to, for like incremental changes, but they're also doing LTS releases around their calendar version. So every December is an LTS version. So 2018.12.0 and so on would be supported for a year and would
be the LTS equivalent, the long-term support equivalent. If you don't want to switch to something beyond that, that's how that group is handling it. Interesting. Yeah. I kind of like that. Then you can glance at a version and, I mean, if everybody sort of supports something like that, you can glance at all of your versions and go, Oh, everything's within a year or so. So it's, it's probably fine. Exactly. Like Brian, you know a lot about testing. If I told you I'm using
web test.2.0.32, is that new? Is it super old? Is that a beta version? Like what is it? Like, what is that? Right. You don't know. Yeah. But if I told you I was using web test 2019.1.0, like, Oh, you must be like testing the new one that's coming or, you know, something I, it's just communicates it. So Sanic is doing this. pip is doing this. and then you threw some others in there, Brian, what are these? I'm not familiar with them. There's some descriptions of
calendar version and semantic version, calver.org and semver.org. I love that those are domains. I think Mahmoud did at least one of them, maybe both. He has this knack for creating these little sites and these little projects that just pop out of nowhere. But the, the, the, one of the things I want to point out is I was talking with, somebody about calendar versioning and that versus semantic versioning. And one of the things is that the major number, whether it's two or three or it's a date
based often means, you're not going to, you're not going to remove features. So you promise not to break customer code within a major version. The date base kind of says we're at least going to stay. It's safe for a year. Yeah. Whereas a semantic versioning, you know, a development team can go through major versions as fast as they want to, if they feel like breaking things fast. That's true. That's true. It's just when you get the feeling to switch the major version from two to three or
three to four, you just do it, right? Yeah. But it has impact. So, yeah, I think there's some interest there. So I think another thing that was really interesting about the calendar versioning that I learned talking to Adam Hopkins from the Sanic project was that there's this sense of you need to keep shipping, right? Like if you have a calendar version and it's from last year and that's the latest version that sends a message in and of itself that this project is, is not getting a lot of
activity. Whereas without it, you know, you kind of got to be more deeply familiar with the project to understand its rate of improvement. Yeah. I think even it'd be cool to have people, even if there's not really any changes needed just to release it and say, Hey, we tested it. It's, this has all been tested on the new versions and you're good to go. We updated like the readme file or something. Right. Well, there's almost always like some minor little thing that can be,
can be improved and shipped. Right. Another story of 2018 and maybe slightly before, but really was coming in strong in 2018 is Python's kind of popular. Yeah. So, Python's very popular. It's taking over the world. There was a neat economist article that came out and that, I guess that's just sort of related to the, they're highlighting a trend. And the article was, Python has brought computer programming programming to a vast new audience.
But some of the things they point out within there is that, right now, according to a stack overflow survey, 40% of professional developers use Python and another 25% wish they did, which is interesting. It just, just go do it, man. Also the, they interviewed, Codecademy, which is an online training site and their biggest increase in demand, for 2018 is with Python. Yeah. That doesn't surprise me. I mean, you think about a very fast growing technology that also has
a lot of career possibilities. Definitely people are going to be wanting to learn that, right? So if you go to somewhere like Codecademy or any other online trading place, like there's going to be, unless they're doing something wrong, there should be a lot of demand for Python. They noted that since 2014, at least in a, from a survey in 2014, Python was the most popular language, intro language at American universities, but that was amongst, people, studying
sciences or computer science or something engineering. But what we're seeing now in 2018 is the growing movement of people that are not traditionally think of themselves as programmers, primarily learning Python. And, we'll see that in data sciences, embedded development is now moving towards Python and in, in, in a lot of maker senses. And then also the article also pointed out that more and more people are just sort of worried that all jobs will be taken over by automation. So they want
to be part of that. So in order to, to learn some automation on your own, Python is a good way to, to be part of that. You either control the robots or you get replaced by the robots. Take your pick, right? Yeah. Also to throw in there, I saw this on, Nina Sakarenko's Twitter feed recently. so Python is becoming, or has been nominated as the official programming language for education in France. So I think it's going to be part of the high school curriculum there, which is, that's super cool.
I think it's, we're seeing the snowball effect, you know, where, where people, it's really becoming the dominating language for beginners and educational language. And I think that's super cool. That is so cool. And when I saw that on Nina's post as well, I'm like, Oh, that is amazing. And you know, I've been thinking a lot about this. Like why is, is Python both popular as a teaching language and popular as a professional programming language? Like C++ is popular as a pro professional
programming language, but they don't suggest that kids start with it. Right. Things like this. I think there's this special balance that when the language was created, it has this way to get started easily and to pull in just the aspects that you want. Right. So you can start out with not even functions and then like, okay, I need some structure. I'm going to put some functions in here. Oh, let's start working with classes and things like generators, but none of that, you don't have to
understand or use or care about that until you're ready. So like Python has a special ability to start easy and small, but expand into professional type of apps. Whereas a lot of them are either, you're always a beginner type of language, or you're always like, yeah, that's a void star star kids. Let's talk about that. Totally. Yeah. Actually, Mike, I think you, you called it a full spectrum language way back when,
when we did an interview in my, my blog with you. And that just really stuck with me because I think that's, that's so true. You know, it's, it's not just, it's not like you're learning, you know, like the little like turtle, like logo or something like that, where it's like, you learn the basics of programming, then you're sort of stuck and you have to move on and learn something else. But no, you can, you know, all the skills you learn working with Python, they kind of scale and,
and grow with you. And that's, that's just a beautiful thing. It's special. There's not many other languages like that. What a time to be alive. It really is an amazing time to be alive. This portion of Talk Python To Me is brought to you by Cloudbolt. Friends don't let friends violate PEP 8, nor do they let them spend their days in an unfulfilling work environment. Good news. Your friends at Cloudbolt want your help developing their state-of-the-art
cloud management software. Built with Django and ranked as the number one product of its kind, Cloudbolt's looking for talented engineers of all kinds. Located in beautiful Portland, Oregon, Cloudbolt is an hour from the Pacific Ocean and Mount Hood. You're not in Portland? Not a problem. Cloudbolt offers a relocation stipend to the Pacific Northwest and is also hiring solution engineers everywhere. Whether you're interested in containers, hypervisors, or just writing clean
performant Python code, Cloudbolt would love to hear from you. Visit talkpython.fm slash Cloudbolt for more information. Brian, you, you touched on that a lot of the growth of Python was centered around people who are using Python as a tool, maybe not as their identity, right? Like Python developers are like, oh, I'm a Python developer, but other people are just using
Python, maybe for data science or things like that. And so number, number seven in our list is that I'm going to declare 2018 as the year that the number of data scientists have come to equal the number of web developers in Python for professional Python developers or users, let's say. Okay, you're going to have to back that up a little bit.
All right. So there was a really cool survey done by the PSF in conjunction with JetBrains. So JetBrains did the infrastructure for sort of analyzing it and hosting it and so on, but it was promoted just through the PSF and other public sources. So it's not like tied to JetBrains, but they did this really great survey. And it's called the Python developer survey 2017, which might sound like I'm wrong in my year
and sort of I am, but this is at the very end. This is like November, December or something like that of 2017. So it's pretty much there. And they did a bunch of interesting, they asked a bunch of interesting questions and they found out a lot of things like, for example, four out of five Python developers use Python as their main language. So when you see people are using Python, mostly they're using
that as their main language. It's not like, well, I write and see, but I also have this script that like compiles it in Python or some weird thing like that, right? So the majority of people using Python use it as their main language. And that's, that's pretty interesting. But the thing that I want to point out is they said for, you know, what do you use Python for? And if you pick where Python is main
versus where Python is secondary, you get slightly different answers. But generally it says for data analysis and data science, 50% of the people use it for that and 49% of the people use it for web development. And then there's a whole bunch of other areas that like, you know, it's a multiple answer type of thing. So they've also checked those off. But in this survey, basically they said 50% of the people said data analysis, 49 web development. Does that surprise you guys? Not really.
I think it surprised me. I know there's a lot of data scientists. I feel like on a web development team, there's more participants than on a data analysis team. I feel like a data analysis is often done by a smaller group of people than like the 30 people that maintain the website or, you know, whatever,
right? Like that was my conception. I don't know if that's accurate, but apparently no. Either there's even more data scientists than I guess, or the there's larger teams of data scientists and smaller teams of web developers. So this is all interesting. Another thing that I thought was pretty interesting around this is they asked people to guess that ratio in this survey and they gave them like a five choices, four choices, something like that. And they said, what do you think the ratio is?
Five times the number of web developers to every data scientist, one to one, you know, maybe worse. I can't like 10 to one. I can't remember. But when they asked that question to the group, half of whom were data scientists and half of whom were web developers based on the original question, the responses they got was only one, only 10% of the people got it right. 20%. And,
most thought that it was five to one, five times web developers to one data scientist. So I think the general perception is there's more stuff happening on the web around Python than there are data scientists. But I think also what the most surprising thing to me when I saw that result was half of the
people that answered that question were data scientists, you know? And so to me, my interpretation is that means that a lot of the data scientists feel like they're not as big of a representative group of the community as they are. They feel like, Oh, we're like 20% of the community. But in fact, they're like half. And I don't know why that is, but I think it's interesting.
It is interesting. Well, there's definitely like a, if you look at the hype around different packages, for instance, there's more packages that are web related than there are data science related. Mostly I think because a lot of the data science tools have been bundled into these large groups of
tools, even though. Yeah. Yeah. So they have like a common set of things that they were like, data scientists like to all use sort of this, you know, a handful of different tool chains, whereas web developers like to go off and create their own asynchronous library. Yeah. Everyone's got their own library to match their new JavaScript library. Yeah. I wonder if the Anaconda distribution and things like that also play into that feeling,
right? You just install it and you just have the tools, right? You don't necessarily go and hunt for every little library as much. This sort of reminds me of who coined that term? Scott Hanselman, I think the concept of the dark matter developer. Yes. I was thinking that too. Yeah. Yeah. I think, I think you actually told me about it. So I guess the idea is that there's a large segment of the programming population or, you know, people who program for even for a living
sometimes that are just not visible in the community. They, you know, get their work done and it's just not, maybe they don't identify as a programmer or it's not just something, it's just not something that they do publicly or have like side projects and stuff like that. Nothing wrong with that. Yeah. And most people, they don't go to conferences or go on social media and go crazy about their job.
Like, like if I was a barista, nothing wrong with that, but like, I probably wouldn't go to barista conferences and go to barista meetups and like, you know, talk about being a barista on social media very much, right? Like it just, it just, yeah, you actually write some awesome stories if you did that. Make coffee for me. Hey, anyone out there listening, go for it. That's all you. I'm going to get that. I would listen to that. I think there's a lot, a huge number of companies that have
one data scientist or data analyst in their company. And so they feel outnumbered because they're the only one in their company or there's like the ratio is very much skewed within their company, but they're kind of all over the place. Yeah. Yeah. I totally agree. And I do think there's a little bit of this dark matter. stuff possibly happening here. Maybe you're not allowed to talk about your analysis that
you're doing. So you just don't, maybe you've backed into becoming this data scientist business sort of analyst type person. You started out as like a trader and then you, you kind of were the person that had enough code to like figure out how to do the overall reporting and analysis and whatnot. And you still don't think of yourself as a developer, but you effectively are. Right. So I don't know. I think there's a lot of interesting angles here.
Yeah. Yeah. Also I was just thinking like, you know, a lot more secrecy maybe in the, in those areas. Like if you're working on some state of the art, like machine learning thing, I don't know, self-driving cars or whatever, maybe you don't necessarily want to go out there and share with the world what you just did. Whereas in the web development work, I feel like we're sort of, you know, we figured most of it out. And so people are more willing, I think, to,
to share what they're working on. At least that's the impression that I got. And often it's, yeah, I agree. And it's on the web already. Yeah. Whereas if you're working on the Apple self-driving car, like you can't even acknowledge its existence, right? You're like working for the CIA. Yeah. I work for the state department. Okay. So anyway, if you want to dig into more of this, I had Eva Jelowalski and, Dmitry Filipov on,
they're the two folks who did the, this and they printed it in a Python. So on Talk Python 176. All right. Item number six, Brian, what do we got? We have a darkening of the community. This is a very dark topic. It is a little dark, but kind of in a good way. Black is taking over. Black is, there's not in a bad way. So the project black is a code
formatter that just sort of formats all your code for you. And, and, and it like PEP 8 and some of those flake eight and stuff tools were to tell you where places where you might want to look at your code and change it. Whereas black is, yeah, I'm not, I can tell you if you want me to tell you, but I'll just go ahead and change it. It's awesome. Sort of taking, there's a lot of projects around the web
or around, Python that are picking it up and, and using it as their standard. And, I think it's been definitely very interesting in 2018 watching this happen. I love that tool. So it's so good. Like, especially if you, if it's a community project, you're working on a project with several developers and it's just so easy to standardize everything and you don't have
to have the back and forth conversations of defining your own code style. And I mean, assuming you like the output that black produces, which I personally really like, and, it's, it's been super helpful for real Python. We have like an, materials repository, some sample projects. So like every single
folder in there was created by a different person, by different author. And so I went down that, that route of like, okay, we're going to define like a beautiful code style and then tell people how to format everything and make sure it's, it's sort of adhere to. And that's really, really hard to do. And so I just ran black on the whole thing, set it up on CI. So we'll check, it'll check the
formatting problem solved. And it took 15, 20 minutes to do that. So yeah. But what a lifesaver, such a great tool. So a couple of things about it. It's taken, it's one of the things that takes things like style and format and stuff like that, and takes those out of code reviews. And so code reviews can just be focused on algorithm and design and not on whether you use single quotes or versus double quotes or
something. The other thing is, black shows an example. It has part of it is a pre-commit hook so that you can add a black pre-commit hook to your code. So everybody just as they're, as they're checking it in, we'll have black run on their code. And, that highlighted the pre-commit tool for me. And I think pre-commit is being used more and more because of black using it. Yeah. I've heard that more than once. That's pretty awesome. You're like, wait,
what's a pre-commit hook? That thing exists in Git? Yeah, definitely. Nice. I like it. The slogan, like you can have it in any color you like as long as you want it in black. Yeah. I didn't know that that was from Henry Ford until somebody told me about Henry Ford saying that about the Model T. Does it have like a theme song or what's going on here? Oh, well, every time I use black, I hear Black Hole Sun from Soundgarden. So, I went ahead and just mentioned that in the
Black Hole Sun is Soundgarden was such a good band. They, I mean, they really came from like a special era of music. We had them, Pearl Jam, Alice in Chains, Dinosaur Jr. Like there's a lot of good stuff back there, but this is the only one that seems to be making it to a Python package. So way to go. All right, Dan, for item number five, I feel like for a really long time, it was like, well, pypi.python.org slash pypi is a little bit out of date and a little bit crummy, but we're working on it.
And it just, it seemed to be in this like stasis of like half improved forever, but not anymore, right? Yeah. The new, new pypi, new Python packaging index went live this year and what an amazing improvement. I mean, like, you know, throughout the whole project and I know you had a really, really great show where you interviewed, you know, a bunch of people working on that transition and launching the new and improved pypi. And it's just crazy, you know, like throughout the whole thing,
it's, it's been like rewritten from the ground up the way I understand it. It's new hosting infrastructure and it's much more friendly now for new contributors. Like beforehand, I think it was sort of like a home grown web framework because, well, it was that pypi, old pypi was created before web frameworks really were a thing in Python. Here's the socket. Let's start with that. Like,
wait, wait, wait, wait, what? Yeah. And so, yeah, I just love it. You know, like every time I'm searching for a package now, it just, it just looks so much better. It looks like, it looks good. You know, it's like, it's when you compare it with JavaScript, like the note JS world and NPM, I felt like, yeah, that was sort of the gold standard for a while for what a online package repository should look like and the features it should have.
And I feel like pypi is now playing in that league and it's super nice. I think it is super nice as well. And it's silly to say that it matters that it looks good because it's just pip install a thing. But I think it really communicates a message to people who come into Python or who are using Python, but especially people who are new and they don't, they're like, oh, what is this stuff? Oh, there's a package injection. You're like, whoa, that thing is like from the
90s. What is that? Right? Like that, I mean, that communicates something about how much the community cares for, for that kind of stuff. Right. Yeah. They also rolled that over with like everybody in the world using it and very little hiccups. Yeah. It definitely is. Really impressive. Yeah. It's super impressive. So a couple of interesting notes here when they rolled it out first, it was pypi.io and I thought, oh, they're just being part of this hip new.io.
You know, it's like, it's like the .ly of yesteryear, you know, it's like Libya all of a sudden became a super awesome place to have like your domain name because it had ly in the end. And so io, right, it's super popular now, but now it's pypi.org. And the story is the pypi.org was owned by somebody else and it took a long time for the PSF to get it from them. Whereas they could go and buy pypi.io. So
that was like an intermediate step in the whole rollout. Yeah. Some of the nasty details of web development. Exactly. And then the other one is, this is a pretty interesting example of a popular Pyramid web app, right? They, they started out in Flask and actually said, no, we'd rather go with Pyramid, which I thought was a pretty interesting choice being a fan of Pyramid myself. And then Brian finally marked down, right? Yeah. So they, the old PyPI was, did not support readme mark.md. It
would only was a readme dot, what is that? RSL or something or. Yeah. RST. Yeah. So I try to avoid restructured text as much as I can. No offense to everybody that loves it. Yeah. I don't hear you. It's like markdown, but it's broken. The stuff you put in there doesn't quite work. Just kidding. No comment. Just kidding. Just kidding. I'm just saying the markdown stuff doesn't always work in there. All right. Number four, item number four, Brian, what we got?
You touched on this a little bit before. Yeah. I think it's been fun to watch. It was unexpected. The rise of Python in the embedded world. We actually did a segment on this in, on Python Python bytes, but there's so many in partly from the maker movement and, and a lot of people trying to get into controlling their, not just their computers, but their world with, internet of things. And also just playing with, you know, adding lights to your bicycle and things like that with
microcontrollers and hardware hacking. Adafruit's been a big part of that, of, pushing it, but, but there's a lot of projects around Python and embedded. Micro Python is one of those. And micro Python's being used by both, by both hobbyists and, and serious, developers. Hobbyists are serious developers too, but, but I think some companies are using micro Python as well. Adafruit has its own version of this called a circuit Python. Actually, I don't know if it's
related to micro Python or not, but Adafruit has circuit Python. And then micro bit is, code, Python base for, the micro bit processor. And then we even have, mu, which is a, editor that directly interacts with, microcontrollers running Python. It's been pretty fun to watch this year. I really think the rights of IOT and Python is just taking root now. And micro Python is so
interesting to me. I mean, the fact that can take a Lambda expression and tie it to a hardware interrupt just like never ceases to amaze me, you know? Yeah, that's really cool. And so, and a lot of this is because somebody wants to learn electronics or something, but they also want to make it do things. So you have to learn a programming language and it used to be, we taught people C and like the barrier to entry for C is huge. Having a Python be able to do that is wonderful.
Yeah, it is. So you talk of the IOT stuff. I learned something from Matt Makai, recently on an episode, and I want to share this with you and just see what you think. So you guys click on this, this link here. So here's an IOT device using a Python. And the idea is the title of the article, this is on the Twilio blog. It's how I potty trained my kids using Twilio and an AWS IOT button. So, so it's a little button like those Amazon dash buttons. And he put it near his son's bed.
And apparently there are kids sharing the room and wanting to make sure he didn't wake up and make a lot of noise and wake up with the other kids. So he put this button that basically has like a poop emoji and you click it and it says, I got to go. And the parents would come in and like help out the kid while he was learning. And this is written using like the Python API on like AWS Lambda and stuff like that. So yeah, the rise of Python and IOT.
So Q4, 10 years. And his kids are like, dad, did you have to put that on the internet? This is awesome. I love this. Exactly. Yeah. This is great. Isn't this funny. So yeah, I mean, funny and like an amusing, but also like, actually that's kind of helpful. Like I can see how that would be helpful as a parent. Cause you definitely don't want your kids waking up and you have to be up for a few hours in the middle of the night.
So if an IOT button can help that, then get the IOT button. All right. Item number three and it's packed full of goodness here, you guys. So legacy Python's days are fading. What do you think? Well, I haven't used it for ages. So yeah, I haven't either. Dan is real Python running on like what? Two, six. Yeah. Not a lot of legacy Python for me anymore. Thankfully. That's awesome. And yeah, it's, I mean, it's, it's kind of crazy that it's, you know, it's a sort of crap
on us, the Python two end of life. And now we're not that far away anymore from it. So yeah, we're actually super close. And the official item that I'm linking to as this piece of news is Guido von Rossum's official statement. So I don't know, four years ago or something, it said, yeah, Python two, it'll be done in 2020, whatever. Right? Like that's like a broad range for a period where things stops a whole year. So we needed a more clear date. People are speculating. Maybe the date
of PyCon might be a cool one. Maybe the end, maybe the beginning. So he came out and said, let's not play games with semantics. The way I see it for the situation for Python two seven is that end of life is January 1st, 2020. And there will be no updates, not even source only security patches after that date period. That sounds like the end. Yeah. It's a good end though, because the, and some people might complain about the whole security
patches, but this is volunteer driven. So it's volunteers time. And I think that to let them work on new things is good. Yeah, absolutely. And it's not like people are surprised, right? It's not like, wait, last month they didn't, nobody told us there's going to be a new Python. Like it's been going on for 10 years. This will make it 12. It's time. Yeah. And there's a couple of loudmouths that two podcasts
that have been talking about it for a couple of years. So yeah, I think we're actually making a dent with this referring to Python two as legacy Python. I think it's, it's making a difference and not everybody loves it, but I know some people it's been working for. Do you guys know, you know, with the, no more official security patches, after the end of life date, I mean, obviously that this is a big like business opportunity for other companies to come in and
offer, you know, long-term Python two updates and stuff. Does anybody know who's, who's doing that right now? Like some Linux distributions, I would guess, but I don't really know what's going on in that space. Yeah. Possibly the big news that I just heard. So, you know, there's red hat enterprise Linux,
right? And they do a lot of that kind of stuff and the versions that they have live on for a long period of time, but they just announced that they're dropping Python two support from red hat enterprise Linux, which to me sounds like, okay, even the laggards are stepping forward. So I think that that's pretty big, but I suspect that there's still a lot of support on the backend.
There's been a few people who've like talked about forking Python two and just supporting it themselves as a single individual, but it seems, I'm sure there's going to be some of that, but like as a company, like, let's say, I don't know, your bank of America, you're like, dude, we don't need to upgrade. There's this guy. He forked it. We're good. You know what I mean? Like that just doesn't seem realistic. It's also not just the C run, the CPython runtime anymore. It's like so much more
than that. So Django no longer supports Python two, like the scientific stack, like NumPy and stuff are dropping support for Python two. So even if you still have the older stuff, you know, you're like, well, we can just keep running. It's not just about what's running your code. It's also what you can use in your code. And you know, that's going to start looking like it's just zooming off into the distance. So not good.
Well, I mean, I think Python two will actually live on forever in some situations where like, for instance, you got something that doesn't need a security patch because it doesn't touch the internet at all. And as long as it will run on whatever CPU you're sticking it on for some people will have like a huge program that they never touch because the requirements never change. And they use it every day,
but it doesn't need to be flexible. It doesn't need to be worked on that much. They'll probably upgrade when they have to change the program. So yeah, I'm wondering if that also means that Apple is finally moving to Python three and making that the default for macOS because the latest release, it's still with two, seven, 10, I think. Yeah, that would actually be really, really nice. You know, it's starting to shift in Ubuntu and just different Linux distributions, but also on macOS.
So yeah, that would be, that'd be super cool. This portion of Talk Python To Me is brought to you by us. Have you heard that Python is not good for concurrent programming problems? Whoever told you that is living in the past because it's prime time for Python's asynchronous features. With the widespread adoption of async methods and the async and await keywords, Python's ecosystem has a ton of new and exciting frameworks based on async and await.
That's why we created a course for anyone who wants to learn all of Python's async capabilities, async techniques and examples in Python. Just visit talkpython.fm/async and watch the intro video to see if this course is for you. It's only $49 and you own it forever. No subscriptions. And there are discounts for teams as well. Brian, you're talking about things that don't require security patches. I just saw a video from NASA talking about their trip to the moon and setting up a base there,
which is just awesome. So they had like all this sort of quick, different scenes of people doing stuff. And one of the scenes was them working, they had sublime open, like editing a bunch of Python code controlling like something about the moon mission, which was pretty cool. But I didn't catch whether it was two or three. I couldn't tell. That's pretty cool. Yeah. Someone else on Reddit did catch that the sublime was unregistered by the
way. Okay. So I guess the final thought here is we've also talked about this on Python bytes is that there's the Python clock.org, the Python to death clock. And it has been updated for this, this new end of life statement. And it reads one year, one month, two days. So if people are out there and they got to start upgrading, like the death clock is ticking. All right. I also got item number two and that I would categorize as the end of the innocence for PI PI and pip. So we've heard about
a couple of issues, one, like right before 2018, like late 2017. And then another one again, this year about people posting or taking over control and posting some form of evil PI PI package. And primarily they look like they're kind of either a research project, a prank, or some kind of like Bitcoin theft. So one of them installed like a clipboard monitor that would watch for a Bitcoin addresses. And as soon as it saw one, it would like ship that off to like the mothership or something to
that effect. But I don't know if anything super bad that's happened, but it, I guess until, you know, 2016, 2017, PI PI as a place was just, it's incredible to me. It's a place on the internet that was entirely unprotected and yet safe. I can imagine that. Yeah. If you had like a open FTP server and you just put it on the internet, like all sorts of badness would happen to that thing in a day. It would be full of all sorts of things you don't want it to be full of.
And here PI PI was going along and it was just, you know, the community sort of policed itself and took care of itself. Plus it used to be really hard to become a person that could push things to PI PI. Yeah. Maybe it correlates to the shipping of PI PI.org. Yeah. There seemed to be a lot of, like, not a lot, like I'm trying to be more specific, but I think npm again, in the JavaScript, like Node.js world, they had a big, they actually was, it was something similar. I think it started with a
typo squatting attack, but then it was actually a worm that would then go. So if you got the malicious package, nothing would really happen except that package as it was installing was looking for your, like if you were registered with the npm package repository and it would go in and patch the packages that you owned to spread that worm further. I don't know if it was some payload, but it was, you know, really exploding quickly. And I guess the same thing could happen with PI PI. Like,
I don't, I don't see why not. Yeah, it definitely could, but it hasn't. And let's, let's not give any people ideas. So the, yeah, exactly. Don't say that security by obscurity. Come on. Exactly. Right. Security by lack of creativity. Okay. So the two main problems were basically like you pointed out, Dan type of squatting, which is like, we're going to upload a package that takes a copy of Django and then slightly modifies it, set up.py and publish it under Django
or Django or whatever, like how you pronounce it without the J, right? So if people pip install and they miss the spell, they get your bad thing. Right. So that's one. And the other was to take advantage of folks who didn't understand why something wasn't available to them. So like, if you tried to use regular expressions and you say re dot, and there's like, it says, well, re is not defined. You're like, well, maybe I need to pip install re. No, you just need to import
it. Right. It's built in. So they were targeting the built-in libraries and I believe all the built-ins are not patched. And Brian, didn't we cover something on Python bytes that would help guard against typo squatting? It would like try to grab packages that also had misspellings around your package name. I think there was something like that. Yeah, I think so. I don't remember what it's called,
but I think that exists and people could find it. Oh yeah. Yeah. I remember listening to that episode and you guys were talking, I forgot what it's called too, but like it generates different variations of your own package names and try and like register all of those. Yeah, exactly. It's very work intensive.
Yeah. So it's interesting on what is the fix? And I don't know what the fix is. The PSF is trying to get more resources to support PyPI and the Python packaging infrastructure in general, but like largely that's been improve it. Do things like you talked about previously, Dan, on just shipping pypi.org. Right. But this validation, if you have so many releases of so many packages, how do you even
verify it? I don't know. It's tricky. It's not necessarily something that could be, that can be solved with a straightforward technology fix, right? Like essentially you're, it's a problem that all of these, let's call them online marketplaces for code repositories or whatever you want to call them.
You know, if you look at the Android app store, Apple app store, I mean the Apple app store, they have these, these like crazy intensive review processes, you know, considering that there are millions of, of, of apps in this app store and they're reviewing every single one. And I think there's even some manual steps involved there. And even there, we've had malicious apps that were trying to steal people's, you know,
private information and it's a really locked down platform. So I think it's really, it's a challenge that, that we're facing now because we're all connected over the internet and, you know, people are really relying on these, on these package platforms. So, it seems like it's something we'll, we'll have to try out a bunch of solutions and see what will work there.
I think we should get some of the, all of these new machine learning Python people to get involved in and take a look at it because conceptually the normal updates for a package are little tweaks on, on what it's doing. And if suddenly it starts, connecting to different servers all over the
world, or even just one where it used to not connect to anything, that would be a change. It would be a different behavior than it was ever doing before that, that should be something that a machine could catch. Yeah, actually, I think you're right. I mean, there's almost no scenario where pip install a thing should create an outbound connections besides to download the package. Or, you know, even a package that doesn't normally use any connections or use requests suddenly adds,
uh, you know, socket connecting or something. that's something to take, you know, at least we can filter it down to things people could take a look at and say, is something weird going on here? Well, it would actually be sweet to have like a CPython interpreter that is fully sandboxed that, you know, can't like, you could just run code that people send you and you can run it safely and it wouldn't be able to write to any folder on your disk or open sockets and whatnot. That would actually be
kind of sweet. I mean, it's all interpreted. So it's seems like that should be possible. I mean, it's going to take a lot of work, I guess, but I think it's possible as well. You know, Dan, you mentioned the online app store as maybe something. I wonder if there's something like that in terms of packaging that we could add. And what I'm thinking is like, if I go build an iOS app, I have to declare the things that's going to ask for, like it has to ask for access to contacts.
It has to ask for access to GPS. So, you know, you declare it needs the GPS feature. It needs these other features, right? So what about a package that declares, like I don't declare any network capabilities. So I don't talk on the network and things like, you know, I can depend upon packages that do, for example. On the flip side, none of these have caused a huge amount of damage. And I don't want to go to the point of overreacting so that everybody has to take off their shoes to travel.
Yes, I know. Things like that. I know. We don't want feel good security that doesn't actually make any difference, but makes life hard for everybody. Yeah. Yeah, I agree. That's a good point. I don't know what the answer is. I don't know what the fix is or if there is a fix, but I would definitely declare the end of innocence for PyPI. That's my declaration as item two for 2018. Dan, you have the honor of kicking off. What is the number one story in the Python space for 2018? What happened?
Oh, man. What a sad honor, though. Well, yeah, I know. It's not about the messenger, right? Well, yeah. I mean, Guido van Rossum stepped down as the BDFL, which is both sad and ironic because I guess BDFL stands for Benevolent Dictator for Life, which I think he still is now technically, but he's just going to be less involved with the stewardship and leadership of CPython. Yeah. He's a little more like the Queen of England now, in a sense, right?
Like he's still representing Python. He's still very active in the community, but he just said, I'm going to let other people deal with the decisions around it. And basically, went so far as saying, like, I'm not going to decide for you how you decide. Like it was even a meta non-decision. So he said, all right, you all have to figure out how to govern yourselves. And it looks like he's coming back in a little bit more in terms of participating.
Well, right. I mean, I think that's one of the reasons is because his involvement with Python was just the stuff that wasn't fun. And now he gets to be involved with some of the things that are fun, like speaking and mentoring and things like that. You know, I think that's a really interesting point. I'm not sure I would want his job the way it kind of looked a year ago, right? Just, you know, arguing over peps and stuff like that.
And you know that if you do stuff on the internet, that it's, there's definitely a very small minority people that are, you know, abrasive, unfriendly. It just sucks. But even just a few people like that will suck your energy for sure. Yeah. And we still don't know what's going to happen. So we'll, maybe that'll be the story for 2019 is what, what happens now.
I think you're right. I think 2018 is the year Keto step back. 2019 will be the year that whatever, whatever takes place going forward is going to be that year. So regarding the new governance structure for CPython, which obviously, you know, for any Python developer, that should be really dear to our heart because it's going to have such a big impact
on the future of, Python and the CPython project. And so the last update that I saw, basically we have a PEP 8,000 now, which is, a PEP that discusses the various new proposals that have been forward, that have been put forward regarding the Python language governance. And, I just recently saw, it was like two or three days ago, Victor Stinner's post on discuss.python.org, where he's comparing the seven Python governance peps. So I guess like actually like the different
proposals, they're separate peps, but they're also listed under PEP 8,000. And so there's different models that have been put forward. Like for example, leading the project with a trio of, three co-leaders. there could also be a community governance model where there's no central authority. So like a total 10 80 from, you know, a dictatorship to no central authority. I don't know how it would, how that would look like, you know, in concrete terms, but there's like different models that are
being discussed right now. And, I think it's going to be very interesting to see what the CPython team ends up with there. I agree. And it's, it's pretty important. Like even simple stuff like Lucas Lange, also creator black, what we spoke about earlier, he's doing the release management for CPython for the
next couple of releases. And he was thinking like, Hey, it would be great to release this yearly, but we can't even discuss having it released yearly instead of every 18 months because we don't even know how to decide whether we, how would we, who decides, how do we vote? We don't know. So, so many things like that are just like on hold until this gets figured out. So there's a lot of pressure to make it happen.
Definitely. Yeah. It's like, it's, it's a total bottleneck, I guess for, yeah, like you were saying, you know, just to put out a new release, you would have to, they would have to figure that out first. And so, I'm, I'm pretty sure that also means it's going to be a decision that's going to be made relatively quickly there so we can all move on.
Yeah, absolutely. One thing you did note though, is that Guido starting to come back and be more active in the community again, just not in this BDFL decision maker way, right? Yeah. And I thought that was kind of cool. you know, I don't know, it might be coincidence, but I don't think it was. So as I was putting together, you know, the recent, PyCoders weekly issue, I saw that there were a couple of interviews with, Guido out there or articles
that, you know, had like little sound bites from him. So he did, a really long, like video based interview on the MIT AI podcast. So it's actually on YouTube. It's more like a video based show, but I think it's also podcast, that I thought was really interesting. And, he also started blogging again. So he has this personal blog at, I think it's
neopythonic.blogspot.com or something like that. And he put out a new post there and it's, I think it's, it's, it was kind of nice to see some activity there again, because I'm, I'm sure it's also a sign that, you know, now he maybe feels like he has more, more time to do that or feels inspired to do things like that again. And I think that's a really, really positive sign and just for his personal wellbeing, I think.
I think it's a positive sign as well. And just like Brian and I were talking about earlier, a lot of his energy had previously gone into like debating PEP 572 and just dealing with all the, the decision-making and now he can focus back on, you know, what most people would want to focus on coding software, cool stuff like that. And it sounds like he's getting back into it. So that's awesome.
Yeah. Yeah. And I mean, you know, what a, what a crazy responsibility to be at the center, like at the focus of it all, you know, and, and to be, to be the one person that needs to make all of these decisions. And sure, you know, like he was delegating a lot of these decisions and everything, but the way the CPython governance was set up before it was like all focused on one person. And, um, yeah, I mean, it's tough to do that for several decades, right?
Yeah. I honestly, I was surprised that he had kept going in that role for that long. I mean, it's sad to see him step down, but at the same time that was 25 years. Like how many other people have a job that goes 25 years? That's high pressure, right? Yeah. It's incredible. I mean, thank you so much, Guido. Yeah, absolutely. It's Python's a special place and it has a lot to do with him. All right, guys, that's it. Those are our top 10 for 2018. It's been a fun year, hasn't it? Absolutely. Yeah.
Yeah. I think 2018 just has a more, more good stuff to come. So it doesn't seem like anything is slowing down then. And that's pretty awesome. Now, before we round out the show, you have to answer the two questions, both of you. So Brian, let's start with you. If you're going to write some Python code, maybe test a little something or other, what editor would you use? My turn. Right on. And Stan. Yeah. Good one. Dan?
I'm still a Sublime guy, although I have been experimenting with Visual Studio Code and it's pretty sweet actually. Yeah. I would say like the Sublime Visual Studio Code, there's probably a lot of bleed over between those and with all the activity around Visual Studio Code, it's got to be eating into the Sublime world. Yeah, definitely. Probably. I like what VS Code is. I like watching what VS Code is doing.
They're just not quite there with the testing yet. I'll take a look again when they get that up to speed. Yeah, I agree. I think they're doing really sweet stuff. Okay. Brian, notable PyPI package? One preferably that doesn't have any badness in it? pytest. pytest. Right on. Dan? Sweet. So I was going to say black again, but we talked about this at length. So I'm going to say pytest-ICDIF. So it's a pytest plugin that gives you better error messages for your pytest
assertions in some cases. So it'll basically do like a more extensive like diff on, let's say, when it's comparing two dictionaries and it'll tell, you know, those keys were added, those keys were removed, and it has like color highlighting. And I just found that this is like a really, really sweet little tool. And I started installing it in all of my projects that have tests. And I highly recommend it. That sounds great. I may have to go check this out. Cool.
All right. So a straight sweep for pytest there. That's great. All right, guys. Thanks so much for being on the show and being part of this. It's so many cool things to talk about. And I really appreciate it. Thanks for having us. Thanks. Yep. Bye. Bye-bye. This has been another episode of Talk Python To Me. Our guests in this episode were Brian Aukin and Dan Bader. And it's been brought to you by CloudBolt and us at Talk Python Training.
Spend your work time fulfilled. Write Python and Django code at CloudBolt, developing their state-of-the-art cloud management software in beautiful Portland, Oregon. Visit talkpython.fm/CloudBolt to join the team. Want to level up your Python? If you're just getting started, try my Python Jumpstart by building 10 apps course. Or if you're looking for something more advanced, check out our new
async course that digs into all the different types of async programming you can do in Python. And of course, if you're interested in more than one of these, be sure to check out our everything bundle. It's like a subscription that never expires. Be sure to subscribe to the show. Open your favorite podcatcher and search for Python. We should be right at the top. You can also find the iTunes feed at /itunes, the Google Play feed at /play, and the direct RSS feed at /rss
on talkpython.com. This is your host, Michael Kennedy. Thanks so much for listening. I really appreciate it. Now get out there and write some Python code. We'll see you next time.