Grant Sanderson (3Blue1Brown) - Past, Present, & Future of Mathematics

Grant, you know, the videos were really inspiring, like, you were the reason I'm like going into grad school. And there's this little bell in the back of my mind that's like, do I want that? And just train on a whole bunch of those. And the thing that takes, at most, 30 minutes of the teacher's time, maybe even 30 seconds, has these completely monumental rippling effects for the life of the student they were talking to that then sets them on this whole different trajectory?

Okay, today I have the pleasure of interviewing Grant Sanderson of the YouTube channel, 3Blue1Brown. You all know who Grant is. I'm really excited about this one. By the time that an AI model can get gold in the International Math Olympiad, is that just AGI, given the amount of creative problem solving and chain of thought required to do that?

I, to be honest, have no idea what people mean when they use the word AGI. I think if you ask 10 different people, like what they mean by, you're going to get 10 slightly different answers. And it seems like what people want to get at is a discrete change that I don't think actually exists where you've got, okay, AI is up to a certain point, they're not AGI. They might be really smart, but it's not AGI.

And then after some point, that's the benchmark when like, now they're, now it's generally intelligent. The reason that world model doesn't really fit is it feels a lot more continuous where, you know, GPT-4 feels general in the sense that you have one training algorithm that applies to a very, very large number of people. And then it's like, you know, it's a very, very large set of different kinds of tasks that someone might want to be able to do.

And that's cool. That's like an invention that people in the 60s might not have expected to be true for the nature of how artificial intelligence can be programmed. So it's generally intelligent, but maybe what people mean by, oh, it's not AGI is, you've got certain benchmarks where, you know, it's better than most people at some things, but it's not better at most people than others.

At this point, it's better than most people at math. You know, it's better than most people at solving AMC problems and like IMO problems is just not better than the best. And so maybe at the point when it's getting golds in the IMO, that's a sign that, okay, it's as good as the best. And we've ticked off another domain, but I don't know. Like, it's what you mean by AGI that you've enumerated all the possible domains that something could be good at.

And now it's better than humans at all of them. And enough that it could take over substantial fractions of, you know, human jobs or something. We're right now, it's impressive, but it's not going to be even 1% of GDP. But in my mind, if it's getting gold in IMO, I mean, having seen some of those problems from your channel, I'm thinking, wow, that's really coming after podcasters and video animators, I don't know.

I don't know. That feels orthogonal because getting a gold in the IMO feels a lot more like being really, really good at go or chess. Like those, those feel analogous. Well, it's super creative. Like, I think anyone, I don't, I don't know chess as well as the people are into it, but everything that I hear from them, the sort of moves that are made and choices, have all of the air of creativity.

I think as soon as they started generating artwork, then everyone else could appreciate, oh, there's, there's something that deserves to be called creative here. And the creative side of the math, you know, I don't know how it would look when people get them to be getting golds at the IMO.

But I imagine it's something that looks a little bit like how AlphaGo is trained where you have it like play with itself a whole bunch, you know, math lends itself to synthetic data in the ways that a lot of other domains don't, you could have it produce a lot of proofs in a proof checking language, like lean, for example, and just train on a whole bunch of those. And like, is this a valid proof? Is this not a valid proof? And then counterbalance that with English written versions of something.

And so I imagine what it looks like once you get something that it's solving these IMO level things, one of two things either it writes a very good proof that you feel like is unmotivated because anyone who reads math papers has this feeling that there are there two types, there's the ones where you morally understand why the result should be true.

And then there's the ones where you're like, I can follow the steps. Why would you have come up with that? I don't know, but I guess that shows that the result is true. And you're left wanting something a little bit more. And so you could imagine if it produces that on the goal to get a goal in the IMO, is that the same kind of ability as what is required to replace jobs, just like not really.

And the impediments between where it is now and replacing jobs feels like a whole different set of things like having a context window that is longer than you know some small things that you can make connections over long periods of time and build relationships and understand where someone's coming from.

And the actual problem solving part of it. I mean, it's a sign that it would be a more helpful tool, but in the same way that like Mathematica can help you solve math problems much more effectively. And tell me why it should be less amazed by it or maybe put it in a different context, but the reason I would be very impressed is that which chess or something, obviously this is not all the chess programs are doing.

But there's a level of research you can do to narrow down the possibilities. And more importantly, in the math example, it seems that with some of the examples you've listed on your channel, for example, the ability to solve the problem is so dependent on coming up with the right abstraction to think about it, coming up with ways of thinking about the problems that are not evident in the problem itself or any other problem in any other test.

That seems different from just a chess game where you don't have to like, what is the largest structure of this chess game in the same way as you do with the IMO problem. I think you should ask people who know a lot about go and chess, and I'd be curious to hear their opinions on it because I imagine what they would say is if you're going to be as good as go as alpha go as you're also not doing tree search, at least exclusively.

It's not dependent on something as both in that because you get this combinatorial explosion, which is why people thought that game would be so much harder for so much longer. There sort of has to be something like a higher level structure in their understanding, and then don't give me wrong, I would be super impressed and like anticipating very impressed when you get AIs that can solve these IMO problems because you're absolutely right. There's a level of creativity involved.

The only claim I'm making is that being able to do that feels distinct from the impediments between where we are now and the AIs take over all of our jobs or something. It seems like it's going to be another one of those boxes that's this historic moment analogous to chess and go more so than it's going to be analogous to the industrial revolution.

I'm surprised you wouldn't be more compelled. I am compelled. You don't think that skill of this problem is isomorphic to this completely different way of thinking about what's happening in the situation. And here's me going through the 50 steps to put all that together into this one proof. I'm surprised you don't think that's upstream of a lot of valuable tasks.

I think it's a similar level of how impressed I was with the stable diffusion type stuff where you ask it, give me a landscape of beautiful mountains but made out of quartz and gemstones or something.

And it gives you this thing which has all of the essence of a landscape but it's not literally a landscape and so you realize there's something something beyond the literal that's understood here. That's very impressive. And in the same way to solve one of these math problems that requires creativity.

You can't just go from the definitions. You're 100% right. You need this element of lateral thinking which is why we find so much joy in finding the solutions ourselves or even just seeing other people get those solutions. It's exactly the kind of joy that you get out of good artistic analogies and comparisons and mixing and matching. I'm very impressed by all of that.

I think it's in the same category and maybe I don't have the same opinions as a lot of other people with this hard line between there's pre-AGI and post-AGI. I just don't know what they mean by the word AGI and I don't think that you're going to have something that's this measurable discrete step. Much less that a math tournament is going to be an example of what that discrete step would look like. Interesting. Interesting. Okay. Applied mathematicians.

Where do we put them in society where they can have the biggest benefit? Like a lot of them go into computer science and IT. I'm sure there's been lots of benefits there. But you know what are our parts of society where you just have a whole bunch of mathematicians go in and they can make things a lot better. I don't know. Transportation or logistics or manufacturing. I feel else do you think they might be useful. That's such a good question. In some ways, I'm the worst person to ask about that.

But this isn't going to answer your question but instead it's going to like fan the flames of why I feel it's an important question. I have actually been thinking recently about if it's worth making an out of typical video that's specifically addressed in inspiring people that ask that. Especially students who are graduating. Because I think this thing happens where when you fall in love with math or some sort of technical field, by default in school, you study that.

And when you're studying that effectively, you're going through an apprenticeship to be an expert in that or a researcher in that. You know, the structure of studying physics in a university or math and university, even though they know that not all majors are going to go into the field, the people that you're gaining mentorship from are academics and our researches in the field. So it's hard not to effectively be apprenticing in that.

And I also have noticed that when I go and give talks at universities or things like this and students come up after and they're like saying hi, there's a lot of them like, grunt. You know, the videos were really inspiring like, you're the reason that I studied math or that you're the reason I'm like going into grad school. And there's this little bell in the back of my mind that's like, cool, cool. I'm amazed.

I don't know if I believe I was solely responsible for it, but like, cool to have that impact. Do I want that? Is this a good thing to get more people going into math PhDs? On the one hand, I unequivocally want more people to self identify as liking math. That's very good. But those who are doing that necessarily get shuffled into the traditional outlets like math academia.

Yeah, I think you highlight it very right. It's like math academia, finance and computer science, data science, something in their general are very common things to go to. And as a result, they almost certainly have an over allocation of talent. All three of those are valuable, right? I'm not saying like those are not valuable things to go into. But if you were playing God and like shifting around, where do you want people to go?

Again, I'm not answering your question. I'm just asking it in other words because I don't really know. I think you should probably talk to the people who made that shift of which there aren't a huge number. But like Eric Lander is maybe one good example. Jim Simons would maybe be another. Whereas people who were doing a very purely academic thing and then decided to shift to something very different.

Now, I have sort of had this thought that it's very beneficial to insert some forcing function that gets the pure mathematicians to spend some of their time in a non-pure math setting. NSF grants coming with a requirement that 10% of your time goes towards a collaboration with another department or something like that. The thought being these are really good problem solvers and a specific category of problems.

And to just distribute that talent elsewhere might be helpful. And when I run this by mathematicians, sometimes there's a mixed response with like, I don't know if we'd be all that useful. Like there's a sense that the aesthetic of what constitutes a good math problem is by its nature rooted in the purity of it, such that it's maybe a little elitist to assume that just because people are really, really good at solving that kind of problem.

But somehow their abilities are more generalizable than other people's abilities. Why ask about the applied mathematicians rather than saying, shouldn't the applied biologists go and work in logistics and things like that? Because they also have a set of problem solving abilities that's maybe generalizable. The back of my mind, I think, no, but the mathematicians are especially. It really is something general about math.

So I don't have the answers. I will say I'm actually very curious to hear from people for what they think the right answers are or from people who made that switch. Let's say they were a math major or something adjacent like computer science physics. And then they decided that they wanted to pour themselves into something not because that was the academic itch that scratched that they were scratching by being good at school and getting to appreciate that.

But because they step back and said, what impact do I want to make on the world? I'm hungry for more of those stories because I think it could be very compelling to convey those specifically to my audience who is probably on track to go into just the traditional math type fields. And maybe there's room to have a little bit of influence to disperse them more effectively. But I don't know. I don't know what more effectively looks like because at the end of the day, I'm a math YouTuber.

I'm not someone who has a career in logistics or manufacturing or all of these things in such a way that I can have an in tune feel for where is there a need for this specific kind of abstract problem solving. It might be useful to speculate on how an undergrad or somebody who was a young math was might even begin to contemplate here's where I can have an edge.

I'm remembering actually just occurred to me a former podcast guy at large do set. He was a game designer actually and he started learning about George's own which is this idea that you should tax land and only land. And so he got really interested in not only writing about those ideas, but also with well if you're going to tax land, you had to figure out what the value of land is.

I mean, how do you figure out the value of land? There's all these algorithms of how you do this optimally based on neighboring land and how to average across land and there's a lot of intricacies there. And so he now has a start off where he just contracts with cities to implement these algorithms to help them assess the value of their land, which makes property taxes much more feasible.

You know, just that's another example right where the motivation was more philosophical, but his specialty as a technical person helped him, you know, helped to make a contribution there. I think that's perfect. Probably the true answer is that you're not going to give a universal thing for any individual is going to be based on where their life circumstances connect them into something either because they had an interest in George's and for whatever reason.

But if someone, I don't know, their dad runs a paper mill and they're like connected to the family business in that way and realize they can like plug themselves in a little bit more efficiently.

You're going to have this wide diversity of the ways that people are applying themselves that does not take the form of general advice given from some podcasts somewhere, but instead takes the form of simply inviting people to like think critically about the question rather than following the momentum of what being good at school implies about your future.

We were talking about this before the interview started, but we have a much better grasp on reality based on our mathematical tools and I'm not talking about anything advanced literally just it would be able to count and. In the decimal system that even the Romans didn't have how likely do you think it is that something that significant would be enjoyed by our descendants and hundreds of thousands of years or do you think that that kind of basic numeracy level stuff.

Those kinds of thinking tools are basically all gone just so I understand the question right you're talking about how having a system for numbers changes the way that we think that then lends itself to a better understanding of the world like we can do commerce exactly that.

Or we can think in terms of orders of magnitude would have been hard to think we have the word orders of magnitude right in a way that is hard to write down much less think about if you're doing Roman numerals like is there something analogous to that for a descendants.

I mean fluency with a programming interface really can help understanding certain problems I think when people mess around in a notebook with something and when it feels like a really good tool set there's a way that that has the same sensation as adopting a nice notation in that you write something with a small number of symbols but then you discover a lot about the implication of that in the case of notation it's because the rules of algebra are very constrained.

And so when you write something you can go through an almost game like process to see how it reduces and expands and then see something that might be non trivial and in the case of programming of course the machine is doing the crunching and you might get a plot that reveals some extra data.

I think maybe out of phase where there's room for that to become a much more fluid process such that rather than having these small little bits of friction like you've got to set up the environment and you got like it in a notebook you've got to find the right way to do that.

And so there's a lot of work that you can do in a notebook you've got to find the right libraries that there's something that feels as fluid as when you are good at algebra and you're just at a whiteboard kind of new looking it out.

I think there's something to be said for the fact that there's still so much more value in if you and I were going to like go into some math topic right now right you you ask me something that's a terrible question for a podcast but I'm like let's actually dig into it. And so when you're looking to do that is still paper I think like I would break out some paper and we would scribble it out and so whenever it becomes the case that the right medium to do that.

Lens itself to simulation into programming and all that. That feels like it would get to the point where it shifts the way that you even think about stuff.

Yeah, that's really interesting. What's up with Miracle Years? So this is something that happens has happened throughout science and especially with mathematicians where they have a single year in which they wake up many if not most of the foreign discoveries that they happen the career you know Newton Einstein Gauss they all had these years it do have some explanation of what's going on.

What's your take? I think there's a bunch of possible explanations it can't just be youth because youth last 10 years not one year so it must have something to do with every 35 year old right now.

I'm just like, I hope there you know what I mean, 20 years. So yeah, it can just be that. I don't know there's a bunch of possible things you could say one is you're in a situation in life where you have nothing else going for you or you just like really free for that one year and then you become successful after that yours over based on what you did. Yeah, but what is your take? I don't know. So I agree that's probably multiple factors not one.

Yeah, one thing could be that the miracle year is the like exhalation and there's been many many years of inhalation where often let's see with you know the classic one is Einstein's where his make miracle year we're also some of the first papers like kind of springing on to the scene and I would guess that a lot of the ideas were not bumping around his head only in that year but it's like many many years of thinking about it and kind of coalescing.

And so you might be in a position where you can build up all of this potential energy and then for whatever reason there's one time in life that lends itself to actually releasing all of that. If I try to reflect on like my own history with what I'm doing now, I think I didn't appreciate early on how much potential energy I was working from simply from being a student and in college where there's just a bunch of ways of thinking about things or empathy with new learners or

just cool concepts right the basic concept behind a video that in fact it was many many years of like all of my time having learned math before I started putting out stuff online that I was able to eat into and then maybe was a little later where okay the well never runs dry there's always a long list of things that I want to cover but in some sense like I recognize that the well was at risk of running dry in a way that I never thought that it could and without being a little deliberate about

devoting some of my day not just to output and producing but to stepping back and like learning new things and touching something I never would have that doesn't happen by default. I don't know if this is all the also the case for the people who have genuine miracle years where they were like letting out all of the stuff and then it takes a decade to build up that same level of potential energy.

The other thing you have everything to gain and nothing to lose when you are young so even if it's not merely you there's a willingness to be creative and there's also none of the obligations that come from having found success before right you know there's there's certain academics who made an extremely deliberate effort not to let what what they call is it like the curse of success or.

The sometime for but I think maybe James Watson you know had this standard reply to invitations for you know talks and interviews and things like that it was basically like no to everyone because I just want to be a scientist it was much more articulate than that that's all these nine points but that was the gist of it and short of doing that I think it's very easy for someone to have a lot of other things that eat into their mind share and time and all of that that even if it's just 20 hours a week like actually that that really interrupts a creative flow.

Were you a student when you started the channel technically yeah the very very first video was made when I was a senior at Stanford basically I'd been like toying around with just a personal programming project into my last year of college that was the beginnings of what is now like the animation tool I work with I didn't

think for it to be a thing that I would use as a math YouTuber I didn't really know what a YouTuber was it was really just like a personal project and it was March of that year I think that I published the first ever video and so it was kind of

right at that transition point do you think you could you would have done it if you had let's say become I don't know what you're planning on doing after college I remember you say somewhere maybe a data scientist but data scientist and math PhD were the two like 50 50 contenders basically

is there a role in which he started doing that but then later on made man or do you think that that was only possible in a world where you had some time to kill in your senior year if the goal was to make math YouTube videos it would have been a wild thing to do to do it by like making man as the method for it because so strikingly inefficient to do it that way at the very least it probably would have built on top of an existing

framework if I had I mean there's so many things that I would tell my past self if I could go back in time even if the goal is to make that it's like certain design decisions that caused pain that could have been fixed earlier on but if the goal was to make videos there's just so many good tools for making videos I probably would have started with those or if I wanted to script things maybe I would have first learned after effects really

effectively and then learn the scripting languages around after effects that might have even been better for all I know I really don't know but I just kind of walked into it because I the initial project was to make something that could illustrate certain ideas and math especially when it came to visualizing functions as transformations mapping inputs to outputs as opposed to

graphing the video output was just a way of knowing that I had completed that personal project in some sense and then it turned out to be fun because I also really enjoy teaching and really enjoy tutoring and stuff. I don't know then again there's a lot of other people who make their own tools for math gifts and little illustrations and things which on the one hand feels very inefficient like there's I don't know if people come across a math gift on Wikipedia.

There's a very high probability comes from this one individual who just strangely prolific at producing these like creative comments visuals and he has his own like home baked thing for how he does it and then there's someone who came across on Twitter Matt Anderson who has these completely beautiful

math gifts and such and again it's a very home baked thing that is like built on top of shaders but he kind of has his own stuff there and maybe there's something to be said for the level of ownership that you feel once it is your own thing that just unlocks a sense of creativity and feeling okay I can just describe whatever I want because if I can't already do it I'll just change the tool to make it able to do that.

For all I know that level of creative freedom is necessary to like take on a wide variety of topics but your guess is as good as mine for those kind of fact tools yeah this is personally interesting to me because I also started the podcast in college and I just like was off track of anything I was playing under you

otherwise and it this is you know many many orders of magnitude away from through the one round I don't want the audience to you know cringe and unison but I just think it's interesting like these kinds of projects how often something that I don't know later on then to being successful is something that was started almost on a whim as a hobby you know when you're in college or something like that I will say there's a benefit to starting in a way that is low

stakes like you're not banking on it growing I I have no anticipation of much less an expectation of three blown brown growing I think the reason I kind of kept doing it was in the fork of life if I did the math PhD and all that I thought it might be a good idea to have a little bit of a footprint on the internet for math exposition I was thinking of it as like a very niche thing that you know maybe some math students and some

people are into math would like it but I could sort of show the stuff as a portfolio not as a as an audience you know size that was meaningful and I think I was surprised by what an appetite there was for the kind of things that I was making and in some ways maybe that's helpful because I see a lot of people who jump in the they jump in with the goal of being a youtuber so I think it's the most common

desired job among the youth as to be like a tiktok or a youtuber which think of that what you will but when you jump in with that as a goal you kind of aim for to large an audience and end up making the content which is best for no one because one you're probably not that good at making videos yet and so you're competing if it's a general

a generally applicable idea you're competing with like all of the other communicators out there. Whereas if you do something that's almost unreasonably niche and also you're not expecting it to blow up it's like one you're not going to be disappointed it's like outstanding when a thousand people view it as opposed to being disappointing and then to you might be making something that is the best possible version of that

content for the audience who watches it because no one else is making that for them because it's too narrow a target and that's sort of the beauty of the internet is that there's an incentive to do that and I don't know if this is the case with your podcast when you're starting off we're not thinking about oh how can I make this as big as

possible actually made it more in depth for those who were listening to it is just surprising to you that prehistoric humans don't seem to have had just basic arithmetic and numeracy to me with with a sort of I guess to us with the modern understanding that kind of stuff seems so universally useful and so fundamental that it's talking that it just doesn't come about naturally in the

course of interacting with the world do you think it's is that surprising that it's a concept of numbers and you're right it's so in our bones that it's hard to empathize with not having a numeracy a lot of its linked to if you think okay what's the first place that like people think about numbers most people like in their daily lives it's linked to commerce and money and such so maybe some

of the questions the same as saying is a surprising that like early humanity didn't have commerce it didn't deal with money maybe when you're below Dunbar's number in your communities like a tit for fact structure just makes a lot more sense and actually works well and it would just be obnoxious to like actually account for everything the other loosely related idea have you come across those studies where when anthropologists will interview tribes of people

that are removed enough from normal society that they don't have the level of numeracy that you're I do but they you know there's some notion of counting you have one coconut or nine coconuts like you have a sense of that that if you ask what number is halfway between one and nine those groups will answer three whereas you or I or people in our world were probably answer five and because we think on this very linear scale it's interesting

that evidently the like natural way to think about things is logarithmically which kind of makes sense like the social dynamics as you go from solitude to a group of ten people to a group of 100 people have roughly equal steps and increasing complexity more so than if you go from like one to 51 to 102 and I wonder if it's it's the case that by adding numeracy in some

way we've also like lost some numeracy or loss of intuition and others where now if you ask you know middle school teachers what's a difficult topic to teacher for students to understand like logarithms but that should be deep in our bones right so somehow got unlearned and the needs to be maybe it's in the formal sense that it's harder to re-learn it but there's there's maybe a sense of like numeracy and a

kind of quantitative thinking that humans naturally do have that is hard to appreciate when it's not expressed in the same language or in the same ways yeah I have seen the thing from Joseph Henryk where this still existing tribes were there in this kind of situation they can do numeracy in the arithmetic when it's in very concrete terms if you talk about

it's not a problem or something but that the abstract concept of a number is not is not available to them do you think the abstract concept of a number is useful to your life oh yeah like in what ways it's almost like asking how is the concept of the alphabet useful it it comes up so often that it's I mean just like how many lights do I set up for this interview right

is that the concept of an abstract number though because it's like two people two lights one to one course like did you leverage the abstraction of two as a object which is simultaneously a rational and a real and an integer I see you know is in the context of a group that has additive structure but also multiple like just there's light for you light for me right yeah I'm pretty sure the abstract idea of a number is important for all of us

but I don't think it's immediately obvious it's more that it shapes the way we think I'm not sure if it actually changes like the way we live if you assume you don't work in STEM right where like you literally are using it all the time yeah I'm trying to go through my day and think through what where am I using them I mean there's the obvious stuff like the commerce examples you mentioned where you go to a restaurant and you're

for what to pay or what to tip but that seems a very particular example is a really use numbers that infrequently I don't know yeah many people listening are probably screaming out of their head like much more apt examples but it's hard to say yeah yeah well when a mathematician is working on a problem what is the biggest mental constraint is it working memory is a processing speed

plants are limited by nitrogen usually like what is the equivalent of nitrogen for mathematician that's a fun question I mean so I'm not a research mathematician I shouldn't pretend like I am and so the right people to ask that question would be the research mathematicians I wonder if you're going to get consistent answers as with so many things there's not one maybe the number of available analogies to be able to draw

connections like the more exposure you've had to this bird field such that you could maybe see oh a problem solving approach that was used here might be useful here sometimes that's literally codified in the forms of connections between different fields as you know functores between categories or something like that but sometimes it's a lot more intuitive the idea that a general flavor of

you know someone's doing a combinatorics type question and they're like oh maybe generating functions are this useful like tool to bring to bear and then in some completely different context of studying like prime numbers they're like oh maybe it could take a generating function type approach maybe you have to massage it to make it work and the one of the reasons I say this is one of the tendencies that you've seen in math papers in let's say like the last 200 years the

typical number of authors is like much bigger now than before you people have this I think misconception that math is a field with lone geniuses who are off like coming up with great insights like a lone extra blackboard and the reality is it's a highly collaborative field I remember one of the first times that I was hewing from a math petitions like a young kid and I was in this math circles event and someone was asking this person like what surprise do about your

job that the first thing he said was how much travel was involved that he wasn't expecting that and it's because you know if you're studying some very specific niche field that the way that you make progress in that is by collaborating with other people like in that field or maybe adjacent to that field and there's only so many about them that probably aren't at your university so you travel a lot to work with them these days a lot of that I

think happens on zoom but conferences are still super important and these sorts of events that bring people all under one roof like MSRI is maybe an example of a place that's trying does that systematically and you could say that's a social thing you know get more ideas but I think it's maybe hitting on this idea that what you want is exposure to as many available analogies so the short answer to your question is like the nitrogen for

mathematician is the analogy this actually is an interesting question I wasn't planning on asking you but it just occurred to me is it surprising how new a lot of mathematics is even mathematics that is taught at the high school level whereas with physics or biology that's also new but you can tell a story where we didn't have the tools to look at the cell or to inspect an electron until very recently but we've had mathematicians for two 3,000 years who were doing you know pretty sophisticated

things even the ancient Greeks not depends on which we've associated but impressive things I don't know why linear algebra is so you given that fact I wouldn't have thought of math as being like new in that way if anything especially for talking at the high school level I remember there's always a sensation that it's frustrating that all of the things are actually a way more than a hundred years old in terms of you know

the names attached to the theorems that you're doing like none of them are remotely modern whereas in biology you know the understanding we have for like how proteins are formed is relatively much more modern and you might be just a couple generations away I it's a some extent there's a raw man power component to it

how many people did pure math for most of history for most of history no one no one was a pure mathematician they were like a mathematician plus something else or they were a physicist who you know they were a natural philosopher and so far as you're doing like natural philosophy one component of that is developing math but it's not the full extent of what you do you know even the ones who we think of is like very very

pure mathematicians in the sense that a lot of their most famous results are pure math like gouse actually a lot of the work a lot of his you know output was also centered on very practical problems and yeah and maybe like since then that's when you start to get an era of something more like pure mathematicians

and the raw number available that you have that man hours that are being put into developing new theorems is probably just got this huge spike as one of the population grows and then also the percentage of the population that has the economic freedom to do something as indulgent as academia like grows maybe it's pretty reasonable that most of it is much

much more recent so do my guess but I mean some of these things seem actually pretty modern like information theories less than a hundred years old that's true and it's you know pretty fundamental like theoretically you could have written that paper a long time ago that's a really good example and maybe like this is

a sign that the math that's developed is more in the service of the world that you live in and the adjacent problems that it's used to solve then we typically think of it on the one hand information theory such a good example

because it's so pure that you could have asked the question you could have defined the notion of a bit but evidently there wasn't a strong enough need to think in that way whereas when you're doing error correction or you're thinking about actual information channels over you know a wire and you're at Bell Labs that that's that's what prompts it another maybe really good example for that would be chaos theory you could easily ask why why is chaos theory so recent you could have written the the

Lorentz equations since differential equations existed why didn't anyone do that and understand that there was this sort of sensitivity to initial conditions and in that case it would maybe be the opposite where it's not that you need the existence of computers as a problem to solve or the problems that they

introduce other problems to solve but instead you need them to even discover the phenomenon in the first place like a lot of original concepts in case theory came from basically running simulations are doing things that required a massive amount of computation that simply

wouldn't be done by hand someone could ask the question but they wouldn't have observed the unexpected phenomenon and there even if it's questions that are as relevant to a pre computer world as to a post computer world like the nature of weather modeling or just the nature three body problem like all of that kind of stuff somehow without the right tools for thought it just didn't come into the mind and so maybe yeah maybe there's other things like that where those

questions were pieces of technology that start to fundamentally shape everyone's life will then invariably also shift like the mathematicians focus I actually reminds me the first day of Scott Ironson's quantum information class he said that what I'm about to describe to you could have been discovered by a mathematician before you know quantum physics existed if only they'd ask the question we're going to do

probabilities but we're only allowed to use unitaries and the rest of it is you know you could have just discovered quantum mechanics or quantum information from there. I mean the thing about math right especially if you're talking about pure axiomatized math the experience as a student as an undergrad is that you are going to a textbook and it starts with saying here's the axioms of this field and then we're going to deduce from those axioms like various different

lemons and theorems and proceed from that and with that is the framing you get the impression that you could have just come up with any axioms make up some pile of axioms deduce what follows from them and the space of

possible like math is unfathomably huge and so you need some process that calls down what are the useful things to maybe pursue and so one of the things that I think is also often missing in those pure math textbooks is the basically the motivating problem why is it that this was the set of axioms people found to be useful

and not something else you know the framework for quantum information theory it's like you married together linear but linear algebra and probability that's great but there's all sorts of other things where you could kind of try to cram them together and maybe get some sort of math out and the question becomes is it worth your time to do that you know not theory is something that emerged because I think it was Lord Kelvin had a theory that all of the

elements on the periodic table had structures which were related to a not like I'm not being if you have a closed loop in 3d space but maybe if you want to continuously deform it without it ever crossing itself you ask the question like could you get back to say an open loop or if you can't get back to an open

loop what are the set of all other loops in 3d space that could be deformed into that and you end up categorizing what all the different knots are and this was started with a completely incorrect theory for what's going on at the atomic level that gives Adams this very stable structure because I think he found with smoke rings like if you're somehow very dexterous with how you create these smoke rings you can get them to form

knots in 3d and they're very stable it's like weirdly stable in the it'll never you know cross over itself and so it has all those properties now that was irrelevant for understanding the periodic table but it was an interesting mathematical question and people kind of ran with it and in that case it was an arbitrary reason that someone thought to ask the question and then some people ran with it and frankly it's probably fewer people who run with it then

would if it turned out to be a more useful question so really you want to ask like what are the what are the things that prompt people to ask what turns out to be a mathematical question given that the space of what would be mathematical questions is so unfathomably huge that it's it's just impossible to explore it through a random walk wait

are you saying that Lord kelvin's apple getting Newton's story was that he was smoking a lot of pipe and categorizes the puffs yeah I can't remember if it was smoke rings out of his mouth I mean that's a natural place we should look in the edge of the book yeah okay so you you changed how you and other creators have changed how pedagogy happens via animated videos what would it take to do something similar for

video games text all these other mediums why hasn't there been a similar sort of broad scale adoption and transformation of how teaching happens there that's right understand the question you're saying where there's been a rise of explanatory videos yeah why is there not a similar rise in like pedagogical video games yeah and well one thing to understand I guess games are very hard to make

it takes a lot of resources for a given game and whenever people seem to try to do it with pedagogy as a motive like they're making some game whose goal is to teach something I don't play enough game so I can't really speak to it the way that well-versed game designers can it seems to be the case that they are not fun in the way that people would want them to be fun and then the ones that are actually most effective are not

you know as directly educational so there's clearly some of the one the one game that I actually have played because enough of my friends told me hey you should really do this it seems relevant to not explanation in the last like decade was the witness yeah have you played it I heard about it so as someone who doesn't play games and then did play it I it's fantastic it's absolutely well done in every possible way that you could want something to be well done

and critical on that is the nature of how problems are solved the reason people recommending it to me is because the feeling of playing the witness is a lot like the feeling of doing math it's nonverbal you come across these little puzzles with a simple mechanics of one puzzle inform you about the fundamental mechanics that become relevant to much much harder ones

such that if you do it with the right sequence you have the feeling of epiphany in ways that are like very self-satisfying and you come away feeling like man you should be able to do something like this for math and maybe you can and it's just that it's so hard to make a game at all that there's just not the rate of production that you would need to explore like get enough games out that one of them hits because with YouTube there's a lot of math videos on YouTube

it's okay that most of them suck right it's okay because hey you just need enough that when someone searches for the term that they want they get one that is good and scratches that itch or that you know they might get recommended something

that is bringing a question to their mind that they wouldn't have thought about but they become really interested once it's there whereas with video games I don't know you're also spending a lot more time as a user on each one rather than a five minute average experience it's like a many many hour average experience you ask the question on text I don't know if I accept the premise that there's not the same advances and innovation in the world of textual explanations

like math a gun is a really really good example of this that's like the textbook of the future so it's a website it's basically an interactive textbook the explanations are really good and in so far as it doesn't have more of an impact or more of a reach it's maybe just because people don't know about it or don't have an easy means of accessing something that recommends to them like the really good innovations happening in the world of textual explanations

in the way that YouTube has this recommending engine that tries its hardest to get more of these things in front of people and I'm quite sure that in the world of actual like written textbooks you I mean there's so many that I like so much that I think it would be a disservice to talk about that medium is not like making advances in terms of more and more thought put towards empathy to the learner and things like that

should the top point 1% of educators should they be exclusively on the internet because it seems like a way to you were just a college professor or a high school professor and you were teaching 50 kids a year or something given the greater scale available should more of them be trying to see if they can reach more people I think it's not a bad thing for more educators who are good at what they're doing to put their stuff online for sure

highly encourage that even if it's a simple is getting someone to put a camera in the back of the classroom I don't think it would be a good idea to get those people out of the classroom if anything I think one of the best things that I could do for my career would be to put myself into more classrooms and actually I'm quite determined at some point to like be a high school math teacher for some number of years

when there's such opportunity cost I guess it's something I would plan on like notably later as long as there's not other like life logistics that occupy a lot of mind share because everything I know about high school teaching is like it just kicks your ass for the first two years but I would say I one of the most valuable things that you can have if you're trying to explain stuff online is a sense of empathy for what the possible viewers are like that that are out there

the more distance that you put between yourself and them in terms of life circumstances I'm not a college student so I don't have the same empathy with what a college student is like certainly not a high school student so I've lost that empathy that distance just makes it more and more of an uphill battle to make the content good for them

and I think keeping people in regular touch with just what people in the classroom actively need is necessary for them to remain as good and as sharp as they are so yes get more of those top 0.1% to put their stuff online but like I would absolutely disagree with the idea of taking them out of their existing circumstances

maybe for a year or two so they don't lose that sharpness but then like put them right back in because one it makes them better at the online exposition but also the other thing I might disagree with is the idea that the reach is lower because okay yes it's a smaller number of people but you're with them for much much more time and you actually have the chance of influencing their trajectory through a social connection in a way that you just don't over YouTube

and I think you're using the word education in a way that I would maybe sub out for the word explanation like you want explanations to be online but like education the word education derives from the same root as the word deduce like to bring out and I really like that as a bit of etymology because it reminds you that the job of an educator is not to like take their knowledge and shove it into the heads of someone else

the job is to bring it out that's very very hard to do in a video and in fact even if you can kind of get it by asking intriguing questions for the most part the video is there to answer something once someone has a question and the teacher's job or the educator's job should be to provide the environment such that you're bringing out from your students as much as you can through in spirit and through projects

through little bits of mentorship and encouragement along the way that requires you know eye contact and being there in person and being at a true figure in their life rather than just an abstract voice behind a screen there's the thing to think of educators more as motivational speakers as in the actual job getting the content in your head is is maybe for the textbooks or for the YouTube

but now the job like why we have college classes or high school classes is we have somebody who approximates Tony Robbins to you know get you to do the thing that would be a subset of it but there's there's more than just motivational speech that goes into it right there's facilitation of projects or even coming up with what the project projects are or recognizing what a student is interested in so that you can try to tailor a question to their specific set of interests

or you can maybe act as the curator where hey there's a lot of online explanations for what a Poisson distribution is you know like which of these is the right one that I could serve and based on knowing you as a particular student what might resonate you might be in a better position to do that all of that goes beyond being a Tony Robbins out you know saying like oh you be the best person that you can be and all of that

and one one thing I might say is that any time that I'll chat with mathematicians and try to get a sense for how they got into a work out them started so so often it starts by saying well there's this one teacher and that teacher did something very small they like pulled them aside and just said hey you're really good at this have you considered studying more or they give them an interesting problem

and the thing that takes like at most 30 minutes of the teacher's time maybe even 30 seconds has these completely monumental rippling effects for the life of the student they were talking to that then sets this them on this whole different trajectory two examples of this come to mind actually one is this woman who was saying she had this moment she got pulled aside from the teacher and he just said like hey I think you're really good at math

he should stand consider being a math major which had been completely outside of her like preview at that time and that like changed the way she thought about it and then later she said she learned that he like did that for a large number of people

and he just pulled them and really think you're really good at that so that's a level of impact that you can have in person as a figure in their life in a way that you can't over a screen then another one which was very funny I was asking this guy why he went into the specific field that he did it was a seemingly arbitrary thing in my mind but I guess all pure math seems to be

and he said that he for his first year of grad school was sitting in this seminar and at the end of the seminar the professor who was this old professor he had never met him before they didn't have any kind of connection he seeks this guy out he comes up and he says you I have a problem for you a good research problem that I think I think might be a good place for you to start in the next couple months

and this guy was like okay and he like gets this research problem and he's been some months thinking about and he comes back and then it later came to light that the professor mistook him for someone else someone he was supposed to be mentoring and so he was just like the the stereotypical image of like a daughtering old math professor who's not very in tune with the people in his life

that was the actual situation but nevertheless that moment of accidentally giving someone a problem completely shifted the research path for him which if nothing else shows you the sensitivity to initial conditions that takes place when you are a student and how the educator is right on that nexus of sensitivity who can completely swing the fences one way or another for what you do

and for every one of those stories there's going to be an unfortunate counter balancing story about people who are demotivated from math I mean I think this was seventh grade there was this math class that I was in and I was one of the people who was like good at math and enjoyed it and would often like help the people in the class understand it and all that

so I had enough ego built up I guess to have like a strong shell around things and I guess for context there was I also really liked music and there was this like concert that had happened where I had like a certain solo or something earlier in the day or earlier in that week and there was a substitute teacher on one day so she didn't have like any of the context and she gave some lesson and then had it spend the second half of the class going over the homework for it

all of the other students in the class were very confused and I think I remember like they would kind of come to me and I would try to offer to help them and the substitute was going around the class in these circles and basically marking off a little star for how far down the homework people were just to get a sense are they progressing and that was kind of her way of measuring are they very far

and when she got to me I had done like none of them because I was spending my whole time trying to help all of the others and she like after having written a little star next to the same problem like three different times she said to me like you know sometimes music people just aren't math people and it keeps walking on

so now I was in the best possible circumstance to not let that hit hard because like one I had the moral high ground of like hey I've just been helping all these people like I understand it and I've been doing your job for you

this was my like little ecotistical seventh grade brain I knew that I knew the stuff and things like that even still even with all of that is like the armor that was put up I remember it was just this like shock to my system you know she says this thing and it just made me like strangely teary eyeed or something and I can only imagine if you're in a position where you're not confident in math and the thing that you know deep in your heart is actually you are kind of struggling with it just a little throw away comment like that could completely derail the whole system in terms of your relationship with the system

so it's another example to illustrate the sensitivity to initial conditions right where there you know I was I was in a robust position it wasn't as sensitive I was I was going to love math no matter what but you envision someone who's a little bit more on that teetering edge and the comment one way or

either saying you're good at this you should consider majoring in it or saying sometimes music people aren't math people which isn't even true or you know what I mean like the other thing about it that that like niggle that my brain when she said it yeah I all all of that is just so important for people's

development that when people talk about online education as being you know valuable or revolutionary or anything like that there's a part of me this sort of rolls my eyes because it just doesn't get at the truth that online explanations have nothing to do with all of that important stuff that's actually happening and at best it should be like in the service of helping that that side of things that's where the rubber meets the road

I had a telecom on the podcast and he obviously has marginal revolution university these YouTube videos where the explains economics and he had a similar answer to give I think I asked basically why aren't you just replacing or should we think of you as a substitute for all these economic teachers and in his mind as well he was more complement to the functions that happen in the class and to your point about the initial conditions I'm sure you remember the details of the story but I just

vaguely remember hearing this wasn't their case were a mathematician who went later ended up becoming famous he arrives to a lecture late see do you want to tell the story I don't remember it beat for beat but I think it was a statistics class and he was a grad student and he comes in it's late and there's two problems on the board that the professor had written he assumed that those two problems were homework and so he goes home and he like works on them and after a couple

weeks he goes to the professor's office and like turns in his homework and it's like I'm sorry I'm so late this one it just took me a lot longer than some of the professors like okay yeah and just like shuffles it away and then a couple days later when the prophet had the time to let go through and see them he realized that the student had fully answered these questions what the student didn't know is that they were not homework problems for another

chat board they were too unsolved problems in the field that the prophet put up as examples of what the field was striving for I don't remember what problems they were so that that would be more fun color to add to the

story but then at least as the anecdote told to me however many years ago goes the prophet and like finds the students housing and like knocks on the doors like do you realize that these are actually unsolved problems and then he gets to basically make those his thesis so that yeah that idea of just being given

something for completely random reasons and it shifts the course of what you do right or just you know something where if you know across or to solvable you just like you just keep going at it until you solve it or the four-minute mile right exactly yeah exactly that's a great example I another valuable experience at least one I had was taking air and since classes in

college and realizing I am like at least two standard deviations below him and that was actually really valuable experience for me not because it increased my confidence in I didn't have a moment or I was like oh wow I'm good at this but it was useful to know you know but podcasting is an easier thing to do right so then it's good to know that there is actual technical things out there where knowing that you can get really deep into

something and people are going to be like way above you having that sort of awareness do you think it's fair to have a mental model that has a static G factor type quality here such that your two single sender deviations below and that is forever the state of things or do you think that the right mental model is something that allows for flexibility on where contributions actually come from or where intuitions come from that you know through

many years of experience and certain kinds of problem solving maybe what seems like a flash of insight was actually like the residue of just years of thinking about certain kinds of puzzles that he had that you maybe didn't you can actually a story from that class actually I go for so he's giving some proof about like how one complete actually wasn't that he was giving a proof of that I forgot the name of the

method but it's a very important method in complexity theory that helped to prove the bounds of the complexity of different problems and he explains it and he says you know in 1999 I proved this myself but I realized that six months before somebody had already published a paper with this method and I realized I'm catching up to the frontier now you know but when I was a kid I was like doing

the trailer that's like 2000 years in regrets now and six months behind and then so later on in the day I'm like wait 1999 how old was Skydarensin in 1999 and he was 18 or 19 and he was basically proving proving from frontier results in complexity theory that were yeah so at that point you're like all right the air and some special animal here you are right he's probably a special animal

but it's just it's just probably good to know just like have that sort of upper constraint on your dining Kruger that you can this exists in the world. Maybe the thing that I would want to say is that whatever the scale is on which he's two standard deviations above you that might not be the one scale that matters and that like contributions to these fields don't always look like genius insights and that sometimes there's there's

there's there's food to be born from say becoming kind of an expert in two different things and then finding connections between them like the people who make contributions are not necessarily the the Skydarensins of the world still you are probably the true that there are like Von Neumann's another example of one of these right yeah yeah yeah okay how much does Gordel's Incablinist theorem practically matter is it is it something that comes up a lot or is it just

interesting thing to know about the bounds that isn't day to day applicable you've asked me another question where I'm not the best one to ask and I should throw that as a caveat to begin from what I understand it really doesn't come up I mean the the analogy to make the paradoxical fact that it's conveying the idea that you can't have an

axiom system that is both that will basically like prove all of the things that are true and which is also self-consistent the the contradiction that you construct out of that has the same feeling as the sentence this statement is a lie well you think about the statement if it's false then it must be true if it's true it must be false it's that same flavor and you might ask because the existence of that paradox

mean that it's hard to speak English it's so rare that you would come up with something that happens to have a bit of self reference in it there were one of the first times that there was something that came up that didn't feel quite as pathological in that way if the curious listener wants to go into it that search and would be Paris Harrington theorems where it's a little pathological but it wasn't the realist a question that came up that didn't seem like it was

deliberately constructed to be one of these self referential things where you know it it shows itself to be outside the bounds of whatever axiom system you are starting with and it so it was you know shown to be unresolvable in a certain sense but it it was asking a I want I don't want to say natural because a lot of these math questions aren't natural in the sense that most people would want but it was asking a question where you wouldn't expect that to be true so maybe

at the edges of theory there are sometimes when the paradoxes that are possible think it was a possibility theorem shows like do you can't wait in the impression I get is no no mathematician is thinking about it they're not actively worrying about it's not like oh god no like can I be sure that the stuff that I'm going to show you for all the practical problems of it's like you know the reman hypothesis or twin primes almost everyone's like no there's going to be an

answer like it may be there they turn out to be unresolvable and in one of these ways but like there's just a strong sense that that theorem came from a pathology in a way that natural questions that people actually care about don't that's really interesting that

something from the outside in and in popularizations is seems to be a very fundamental thing where people people have definitely heard about this right is not internally a good a good analogy here is in computer science the halting problem it's like you take a computer science

course one of the first things you learn is you know the proof of the halting problem and it's another one of those things where you don't really need to be able to prove that every single or you have that sort of program available that for yeah no comments no more coming

to you. Why are good explanations so hard to find despite how useful they are obviously there's many other than you as well there's many other cases of good explanations but generally it just seems like there aren't as many as there should be is it just a story of economics where it's nobody's incentive to be making really spend a lot of time making good explanations is it just a really hard skill that isn't correlated with being able to come up with a discovery itself what why are good

scares I think there's maybe two explanations the first less important one is going to be that there's a difference between knowing something and then remembering what it's like not to know it and the characters to go to explanation is that you're walking someone on a path from the feeling of not understanding up to the feeling of understanding earlier you're asking about societies that are that lack numeracy that's such a hard brain state to put

yourself in like what's it like to not even know numbers how would you start to explain what numbers are maybe you should go from a bunch of concrete examples but like the way that you think about numbers and adding things it's just you have to really unpack a lot before you even start there and I think at higher levels of abstraction that becomes even harder because it shapes the way that you think so much that remembering what

it's like not to understand it you're teaching some kid algebra and the premise of like a variable they like what is X you know like well it's it's not necessarily anything but it's what we're solving for it like yeah but what is it like trying to answer what is X is a weirdly hard thing because it is the premise that you're even starting from the more important one probably is that the best explanation depends heavily on the individual who's

learning and the perfect explanation for you often might be very different from the perfect explanation for someone else so there's a lot of very good domain specific explanations you know pull up in any textbook in like chapter 12 of it is probably explaining the content in there like quite

well assuming that you've read chapters 1 through 11 but if you're coming in from a cold start it's a little bit hard and so the real gold nugget is like how do you construct explanations which are as generally useful as possible as generally appealing as possible and that I think because you can't

get a word context it becomes this challenge and I think there's like tips and tricks along the way but because the people that are often making explanations have a specific enough audience it is this classroom of 30 people or it's this discipline of majors who are in their third year all the

explanations from the people who are professional explainers in some sense are so targeted that maybe it's the economic thing you're talking about there's not or at least until recently in history there hasn't been the need to or the incentive to come up with something that would be motivating and approachable and clear to an extremely wide variety of different backgrounds is the process of making her videos is

that mostly you okay given the scale you're reaching it seems that if it was possible you know just like a small increase in productivity would be worth like an entire production studio and it's interesting to me you're surprising that the transaction cost of having a production setup are high enough that it's better to literally do the mundane details yourself.

I mean this could honestly just be a personal flaw like I'm not good at pulling people in and then I've struggled to do this effectively in the past but part of it is that the month the seemingly mundane details are sometimes just how I even think about constructing it in the first place you know the first thing that a lot of YouTubers will do if they can hire is hire an editor and this will be because they film a lot of things and so a lot of the editing processes removing the stuff that was

filmed that shouldn't be in the video and just leaving the stuff that should be in the video and that's time consuming and it's kind of mundane and it's probably not that relevant to what the creator should be thinking about when you're not filming stuff like the editing process for me you know I start by laying out all of the animations and stuff that I'll want in a timeline and then once I record the voiceover the actual editing is like a day and it's not like I guess I could hire someone

and gain a day back of my life but the communication back and forth were saying what specifically I want like all of the little cuts that I'm making along the way are my way of even thinking about what I want the final piece to be such that it would be hard to put it into words.

It's similar for why I maybe find it quite hard to use like a co-pilot and some of these like LLM tools and coding where for the animation code it can be super great if you're learning some new library and it knows about that library that you don't but for my library that I know inside it

out if I'm just using it feels like oh this should be the most automatable thing ever you know it's just text like I should be the first YouTuber who can actually do this better because the substance behind each animation is text it's not like an

editing workflow in quite the same way but it doesn't work and I think it's because maybe it's just because you need a multimodal thing that actually understands the look of the output like the output isn't something that is consumable in text it's something about how it looks

but at a deeper level I can't even put into words what I want to put on the screen except to do so in code like that's just the way that I'm thinking about it and like if I were to try to put into English the thing that I want as a comment that then gets expanded

that task is actually harder than writing it in the code and if it's clunky to write in code that's a sign that I should like change the interface of the library such that it's less clunky to be expressive in the way that I want and it's in that same way where a lot of the

creative process that feels mundane those are just like the cogs of thoughts slowly turning in a way that if they weren't turning for that part they would have to be turning during the interface of communication with the collaborator. On the point of working with co-pilot where we can visualize the changes you wanted to make the paper for Microsoft Research, the sparks of HIP or how did it actually really interesting example where it was generating let's act and they generated some

output and they say change this so that the visual that comes up in the rendering is different this way and it was actually able to do that which was their evidence that it can understand the higher level visual abstraction somebody I guess I can't do that for madam.

There's a couple of reasons I might it might not be as fair comparison one would be the two versions of madam there was a split where there's a community version that is you know by the community for the community and then mine the interfaces are largely similar the rendering

are quite different but because of slight differences in that and you know it might have a tendency it learned from one or it's examples from one and it's intermixing them so stuff just doesn't quite run when there's discrepancy I may be shot myself in the

foot where all of my code for videos like I don't really comment it that much because it's like a one and done deal it's if it's you know there's also a move for the core library I could comment and document a lot better but like the way that I'm

making it feels much more like the editing flow if you were to look at the operation history of someone in after effects right like written down it's a little bit more like that where there's not a perfect description in English of the thing that I want to do and then the

execution of that it's just it's just the execution of that and you know it's not meant to be editable and I cite as much because I'm just in the flow of making the scene for the one video and you know maybe I could have given it a better chance to learn what it's

supposed to be happening by having a really well documented set of like this is the input this is the output this is the comment describing it in English but even then that wouldn't hit the problem on like I would have to articulate what the thing I want is

in the first place and the the language the program language is just the right mode of articulation in the first place this is something I wanted to I was really curious about ever since I learned about it I watched many of the summer of math exploration prize

videos and it was shocking to me how good the I mean these looks like entire production studios were dedicated to making them many of them and it was shocking to me that you could motivate and elicit this quality of contribution given the relatively modest

price pool which I was like five winners one thousand dollars each what is your explanation of just running prices like this why you were able to get such high quality contributions is it is a price pool even relevant is it just about your reputation and reach I do

wonder how relevant the price pool is I mean we've been thinking about this because you know we did it first in 2021 then we plan to continue doing it annually and I probably like was a mover and a shaker could raise much more if I wanted to like get a big

price pool there I don't think it would change the quality of the content because the impression I get is that people aren't fundamentally motivated by like winning some cash prize certainly they're not investing with it like expected value calculation

on what if they are that's a terrible terrible plan and anything like that might be a problem like let's say it was a hundred thousand dollar prize for each of the winners then it would be a real problem where someone who and you know people do like to

illusionally think that they're very likely going to be the winner they might like actually pour a lot of their own resources into it with the expectation of gaining it and then that's just a messy situation I don't want to be in with like why wasn't mine chosen as a winner because the whole event is not supposed to be about winners in fact maybe for listeners who don't know I should describe what you're talking about the summer of math exposition actually the history is a little bit funny

because it started with an intern application where I in 2021 wanted to have just like a couple interns to do a certain thing on my website basically and I put out a call for people to apply I got 2,500 applicants and somewhere in the application I mentioned that during the summer in addition to like the main task I wanted them to do you know I'd give freedom if they just wanted to do something relevant to math exposition online that was their own thing I'd be happy to

provide you know some mentorship or just give them the freedom to do that one day a week and give me a little pitch on what your what your idea would be and as I went through you know all of the applications which is a lot I felt so bad because so often the person would have a little pitch and like what they would want to make and in my mind I think cool you should make that you don't need me to like do that like just spend your summer making that why not and people were

clearly inspired by the thought of adding something and like I said earlier being a youtuber is the most common job aspiration among the youth these days and so is a consolation of sorts to those 99% that I had to reject for the internship I said oh but what we will do we're just going to like host this thing we'll call it the summer of math exposition where we'll give you a deadline I'll like promise to feature some people in a video maybe I'll choose five of

them to do that and if you feel like the thing that you were going to do like with me as your 20% project as an intern is something you're excited about make it a hundred percent project or 20% or whatever whatever freedom you have just do it anyway and like I can give you this little carrot in the form of featuring it in a video and give you a deadline which let's be honest is what actually makes the difference between people doing something and procrastinating on it

sometimes later there was I think it was brilliant to that year said they would be happy to put some cash prizes in so I said sure why not I don't think the cash prize is super important but it's nice I mean it shows that someone actually cared and put some real thought into doing something that wasn't just a made up gold star but they put some material behind saying that you were selected as a winner of this thing but all in all it was never supposed to be

about choosing winners it was just get more people to make stuff and if anything I'd actually I love it when I see stuff from existing educators and teachers where it's maybe not the youth who want to be YouTubers pouring their hearts and souls into it but it's the educator who built a lot of

intuition over the course of their career for what constitutes a clear explanation and they're just sharing it more broadly so to your question on what is it that caused there to be such high production quality in some of the entries there part of the answer might just be that like tooling is so

good now that individuals can actually make pretty incredible things sometimes I miss phrase device at production quality I just meant the whole composition as a whole yeah well there's a there's a selection filter too right like there were in that first year 1200 submissions I featured in you know

the winning video five of them so of course they're going they're necessarily unrepresentative of the norm by the very nature of who I was choosing but the fact that those were even in the pool yeah something that high quality was even the pool I think it hits a little bit to your miracle year point

where I think what might be happening is you have people with a ton of potential energy for something that they've kind of been thinking about making for a long time and the hope was to give people a little push here's a deadline here's a little prize here's a promise that maybe if you make it won't just go into the void but there's a chance that it gets exposed to more people which I absolutely played out and not for the reason that someone might expect where

I choose winners and I feature those winners and people watch them a huge amount of viewership happens before I even begin the process of looking at them and this was an accident too where in this first year we got 1200 submissions I

said expect judges who are reviewing it to spend at most 10 minutes on each piece so it could be longer but like don't rely on someone watching it for more but realistically when I'm reviewing something I want to watch the whole piece I absolutely do not have time to watch

it takes I've learned it takes me about two weeks of just full-time work to like watch a hundred of these pieces and give the kind of feedback that I want and to manage that problem of more than we could manually review we put together this pure review system that would basically have an algorithm

feed people pairs of videos and they would just say which one is better and then it would feed them another one and in the first two years we just used a tool that was common for hackathons that did this and what that did is one it gave us a partially ordered list of content by quality loosely we didn't need it to be perfect we just needed there to be a very high chance that the five most deserving videos were visible somewhere in that top 100 right so

there the algorithm doesn't have to be perfect but what happened I think is that by having this period when a ton of people are watching the same batch of videos so I think I've learned about the YouTube algorithm is in theory you would want to just use machine learning for everything right like you have some massive neural network around the input of it it's got five billion videos or however many exist and the output it decides what seven are best to recommend to

you that is completely computationally infeasible right I also think I think this is all public knowledge so what you have to do instead is use some sort of proxies as a first pass to nominate a video to even be fed into the machine learning driven algorithm so that you're only feeding in like a thousand or so or something in the order of thousands like nominees so the real difference that it can make if you've made a really good video between it getting to the

people who would like it and not getting there it's not the flaws in the algorithm the algorithms probably quite good it's the mismatch between the proxies being used to nominate stuff to see whether it's even in the running so one of the things used for nomination is understanding the co-watch graph where if you've watched video a and you've also watched video b and then I watch video a you're watching both of those gives a little link between them

or maybe you and a ton of other people watching both of them gives a little link between such that once I watch video a b is potentially nominated in that phase because you know it's recognized that there's a lot of co-watching

that's something that I'm sure still is like quite challenging at scale but is more plausible to do at scale than like running some massive neural network and so I think what might have happened is that by having a bunch of co-watching happening on this same pool of videos all you need is for like

some of them to have decent reach and get recommended right because then that kicks off it's like igniting a pile of kindling where then if others are good if they're going to give people good experiences they get not only nominated but then recommended which then like kicks back in the feedback loop there so that turns out to be I think as close to a guarantee as you can get of saying if you make something that's good it's a good piece that will satisfy

someone it they come away feeling like they learn something that they otherwise didn't know and it was well presented if you can get it into this peer-review process it will reach people it's not just going to be shouting into the void and in this case you know there was last year I think over a hundred videos where after the first two weeks they had more than 10,000 views which I know in the grand scheme people they're like huge numbers of views and millions out

there but like for a for a fresh channel talking about a niche mathematical topic to be able to put it out and like get 10,000 people to watch it I think is amazing right and the idea that that could happen for over a hundred people

I think is amazing so that had nothing to do with the prize pool right in that the the mode of might have been a hope of like actually getting some reach and having some sense of a guarantee of there being some reach ironically the reason to do the whole peer-review system in the first place is in

the service of selecting winners and so like without us something in the service of selecting winners if you just said hey we're having a watch fest where everyone watches each other's things somehow it wouldn't quite have the same like I

don't know pole that gets people into it so I think it still makes sense to have winners and to have some material behind those winners it doesn't have to be much though and if anything I think it might ruin it to make it too much I'll also say it's fifteen thousand dollars actually because we give

five hundred dollars to twenty different honorable mentions at least this year so still pretty modest in the scheme right how much money you can invest in the world I watch many of the honorable mentions as well because they were just topics that are interesting to me and I mean it's like the thing that the president of Chicago University said at one point where he said we could discard the people we admitted and select the next thousand for our class and

there'd be no difference it was that level of you know you're yeah that that level of peak yeah I really admire by the way not only the education that you provided directly with your videos which have reached millions of people but the fact that you're also setting up this way of getting more people

to contribute and maybe get to topics that you wouldn't have time to get to yourself so I like I really admire that you're doing that if you're self-teaching yourself a field that involves mathematics let's say it's physics or some other thing like that there's problems where you have to

understand how do I put this in terms of a derivative of an integral and from there it's can I solve this integral and I think that's what I want and what would you recommend to somebody who is teaching themselves let's say quantum mechanics and they I figured out how to put how to get

the right mathematical equation here is it important for their understanding to you to take go from there to getting it to the end result or can they just say well I can just abstract that out I understand the broader way to set up the problem in terms of the physics itself I think where a lot of

self learners shoot themselves in the foot is by skipping calculations by thinking that that's incidental to the core understanding but actually I think you do build a lot of intuition just by putting in the reps of certain calculations some of them maybe turn out not to be all that important and

in that case okay so be it but sometimes that's what maybe shapes your sense of where the substance of a result really came from I don't know it might be something you're like oh it's because of the square root that you get this decay and if you didn't like really go through the exercise you

would just instead coming away thinking like such and such decays but with other circumstances it doesn't decay and like not really understanding what was the core part of this you know high level result that is the thing you actually want to come out remembering like putting in the work with the

calculations is where you solidify all of those underlying intuitions and without the forcing function of homework people we just like don't do it so that I think that's one thing that I learned is a big difference post college versus during college is like post college it's very easy in learning

stuff to just accidentally skip that and then it doesn't sink in as well so I think when you're reading something having an notebook next to you just like having pen and paper pencil and paper whatever it is should be considered part of the actual reading process and if you are relying too much on

like reading kind of looking up and thinking in your head you know maybe that's going to get something but it's it's not going to be as highly leveraged as it could be well would be the will be the impact of more self teaching in terms of what kinds of personalities benefit most so there's obviously a difference in the kind of person benefits most in a situation where there's it's a college course and everybody has to do homework but maybe some people are better tuned

for the kind of work that's placed there versus all this stuff is available for you on YouTube and then textbooks for exercises and so on but you have to have the conscientiousness to actually go ahead and pursue it. I mean how do you see the distribution of who will benefit from the more modern way in which you can get whatever you want but you have to push

yourself to get it. There's a really good book that's actually kind of relevant to some of your early questions called Failure to Disrupt that goes over the history of educational technology trying to answer the question you have these repeated cycles of people saying such and such technology that almost always is getting more explanations to more people promises that it'll disrupt the existing university system or disrupt the existing school system and

just kind of never does. You know one of the things that it highlights is how stratifying these technologies will be in that they actually are very very good for those who are already motivated or kind of already on the top in some way and they end up struggling the most just for those who are performing more poorly and maybe it's because of confounding causation where the same thing that causes someone to not do poorly in the traditional system also means

that they're not going to engage as well with the the fluffer of tools available. I don't know if this answers your question but I would re-emphasize that what's probably most important to getting people to actually learn something is not the explanation or the quality of explanations available because since the

printing press that has been not literally true because maybe access to libraries and such as as universal as you would want but people had access to the explanation once they were motivated but instead it's going to be the social factors like are the five best friends that you have also interested

in this stuff and do they tend to push you up or they tend to pull you down when it comes to learning more things or do you ever a reason to it there's a job that you want to get or domain that you want to enter where you just have to

understand something is there a personal project that you're doing like the existence of compelling personal projects and encouraging friend groups probably does way way more than the average quality of explanation online ever could because once you get someone motivated one they're just they're

going to learn it and it maybe makes it a more fluid process if there's good explanations versus bad ones and it keeps you from having some people drop out of that process which is important but if you're not motivating them into it in the first place it doesn't matter if you have the

most world class explanations on every possible topic out there it's it's screaming into a void effectively and I don't know the best way to get more people to know things I have had a thought this is the kind of thing that could never be done in practice but instead it's something you would like write some kind of novel about where if you want the perfect school right something where you can insert some students and then you want them to get the best

education that you can what you need to do is let's say it's a high school you insert a lot of like really attractive high schooler plants as actors that you like get the students to develop crushes on the various plants right and then anything that you want to learn the plant has to express a certain interest in it they're like oh they're really interested in Charles Dickens or something right and they express this interest and then they you know

suggest that they would become more interested in whoever your target student is if they also read the Dickens with them they're also like interested in learning the physics like if you get that in a way you know you socially engineer the setting

the effectiveness that that would have to get students to actually learn stuff is probably so many miles above anything else that we could do that nothing like that in fact is gonna actually literally work but at least viewing that as this end point of okay this mode of interaction

would be hyper effective at education is there anything that kind of gets at that right and so okay the kind of things that get at that it would be being cognizant of your child's peer group or something which is something that parents I think very naturally do or okay it doesn't have to be a

romantic crush but it could be that there's there's respect for the teacher right it's someone that they genuinely respect and look up to such that when they say there's an edification to come from reading Dickens that that actually lands in a way and so taking that as a paragon and then letting everything also approximate that has I would emphasize nothing to do with the quality of online explanations that they're out there that at best just makes it such that

you know you can you can lubricate the process once someone's sufficiently interested you found a new replica use case yes I mean I'm not saying we should do it but think of how effective that would be okay final question this is something I should have followed up on earlier but your plans to become a high school teacher first of all years what are you playing on doing that and what do you hope to get out of that I would say no concrete plans I would say it would I wouldn't want to do it

in a period where like I also have young children and therefore it would make sense to maybe a lot of people will say this kind of thing but there's friends of mine who think when their child is in like you know high school that's that's when they would want to be a high school teacher I think two things I would want to get out of it one of them as I was emphasizing I think you just lose touch with what it's like not to know stuff or what it's like to be a student

and so maintaining that kind of connection so that I don't become dollar and dollar over time feels important the other I would like to live in a world where more people who are savvy with STEM spend some of their time teaching I just think that's one of the highest leverage ways that you can

think of to actually get more people to engage with math and so I would like to encourage people to do that and call that call to action some some notion of spending maybe not your whole career but a little bit of time in teaching there's not as fluid a system for doing that as say you know going through a tour of service in certain certain countries where everyone like spends two years in the military so shy of having a system like that for education

there's all these kind of ad hoc things where you know charter schools might have like an emergency credential system to like get a science teacher in but there's a you know teach for America is something out there there's enough ways that someone could spend a little bit of time that's probably not fully saturated at this point that you know the world would be better if more people did that and it would be hypocritical for me to suggest that and then not to

actually put my but my feet where my words are yeah well I think that's a great note to leave it on grant thanks so much for coming on the podcast and genuinely I mean you you're one of the people I really really admire but what you've done for the the landscape of math education is it I mean it's

really remarkable so this is a pleasure to talk to you thanks for saying that I had a lot of fun hey everybody I hope you enjoy that episode as always the most helpful thing you can do is to share the podcast then it to people you think might enjoy it put it in Twitter your group chats etc just blitz the world appreciate your listening I'll see you next time cheers