Michael Nielsen – How science actually progresses

Today I'm speaking with Michael Nielsen. You have done many things. You're one of the pioneers of quantum computing, wrote the main textbook in the field of the open science movement. You wrote a book about deep learning that Chris Ola and uh Greg Brockman credit them with getting them into the field. Um more recently you're a research fellow at the Astera Institute and writing a book about religion, science and technology.

I'm gonna ask you about none of those things. The conversation I wanna have today is How do we recognize scientific progress? And it's it's b especially irrelevant uh for AI because people are trying to close the RL verification loop on scientific discovery. And what does it mean to close that loop? But in preparing for this interview,

I've realized that it's a more mysterious and elusive um force even in the history of human science than I understood. And I think a good place to start will be Michaels and Morley and how special relativity is discovered, if it's different than the story that you kind of get off of YouTube videos. Um Anyways, I I will prompt you that way and then we'll go in there. Okay. Yeah, so I mean Michelson Wally is Uh y one of the sort of the the the famous results often presented as

as this this experiment that was done in the eighteen eighties and that helped Einstein, you know, come up with the the special theory of relativity a little bit later. So so sort of changing our our the way we think about space and and time and and our fundamental conception of those things. Um and there's kind of a

uh a big gap, I think, between the way Michelson and Morley and other people at the time thought about the experiment and certainly the way in which uh Einstein thought or did not think about the experiment, um in actual fact,

stated later in his life he wasn't even sure whether he was aware of the paper at the time. Um there's a lot of evidence that he he probably was aware of the paper at the time, but it actually wasn't dispositive for his thinking at all. Uh something else uh completely was was was going on. Um

So uh uh what Michelson and Wally thought they were doing was they thought they were testing different theories of uh what was called the ether. So if you go back to the the the sixteen hundreds, uh Robert Boyle introduced the idea of the ether and basically the idea of the ether is um Yeah, we know that that sound is vibrations in the air.

Um and then Boyle and other people got interested in the question of like is is light vibrations uh in something? And they couldn't figure out uh what it was. Boyle actually did an experiment where he he tested whether or not you could propagate light through a vacuum.

he found that you could, you couldn't do it with with with sound. So he introduced this idea of the ether and then for the next two hundred or so years people had all these kind of conversations about about what the ether was and what its nature was. And the Michelson and Morley experiment was really an experiment to test Different theories of the ether against one another. Um, and in particular to find out whether or not there was a so-called ether wind.

So the idea was that the the earth is passing through uh maybe this ether wind. And if it is passing through the ether wind, sort of this background, um, and you you shoot a light beam sort of parallel uh to the direction the ether wind is going in, it'll get accelerated a little bit. Um and if it's being passed back uh sort of in the opposite direction.

It'll get slowed down a little bit and you should be able to to see this in the results of interference experiments. And what they found, much to their surprise, um, I think, uh, was that in fact there was no ether wind.

Um and that ruled out some theories of the ether but but but not all. And and Michelson certainly continued to b to believe in the ether. Aaron Powell Okay, so i th this is what was a shocking part of um reading this story from uh the biography of Einstein that you recommended by um What was his first name? Abraham Pipe. Abraham Pikes. Yeah. Subtle as a Lord. And then also from Imre Lakatos, uh the methodologies of scientific research programs. The way it's told is that Miglson Morley

Proved that the ether did not exist. Therefore, it created a crisis in physics that Einstein saw saw with special relativity.

which you're pointing out is actually was trying to distinguish between many different theories of ether. You know, if you're in space or if you're on Earth, it's the same direction of ether, or maybe the ether wind is being carried around by the Earth and so you can't really experience it on Earth, but if you go to a high enough altitude, you might be able to experience it.

Um in fact the Michelson's experiments were the famous one is eighteen eighty seven, but uh he conducted these experiments. for basically two decades. I mean for longer than that. He he conducted them I think the first one was in eighteen eighty one, but he continued to believe until I mean he died. He died I think was like nineteen twenty nine or so. It was like the late twenties. And he was still doing experiments in the 1920s.

um uh sort of about whether or not, you know, the ether existed. And so he so he continued to believe in the ether to the end of his his life, or I think the last public statement he made is like a year or two before he died and he still still believed ba basically believed at that point. Aaron Powell And in fact there was an a uh another physicist, um Miller, who kept doing these experiments and in nineteen twenty

He thought that he went to a high enough altitude, uh, is in Mount Wilson in California. Where oh I'm high enough. that I can actually the ether winds are not being dragged with up by the earth. I and I've measured um the effect of the ether. And Einstein hears about this and he says, this is where you get the famous quote, subtle is the Lord, but malicious he is not.

Anyways, I think the r the reason the story is interesting, it's for m for many different reasons, but one is One of the different ways in which the real history of science is different from this idea you get of the scientific method is you really can't apply falsification as easily as you might think. Um it's not clear what is being falsified.

Uh is it just another version of the the theory of the ether that's being falsified? Or um certainly you can't induce the theory of special relativity from the fact that one version of the ether seems to be disconfirmed by these experiments. Yeah, so I mean it certainly doesn't show that you know, ideas about falsification are are wrong, are falsified.

Um but but you know, it does share the sort of the the most naive ideas, you know, are are is things are much often much more complicated than you think. So yeah, m Michelson did this experiment in eighteen eighty one, he was a very young man. And then uh other people I think Rayleigh was one of them, pointed out that there was some problems with the way he did it, so they had to redo it in ninety in eighteen eighty seven.

Um and at that point, like a lot of the leading physicists of the day, leading scientists of the day, basically accepted um this result, that there there was no uh ether wind. But what what to do about this? Um so yeah, sure, maybe you've falsified some theories of the ether. There are others that you haven't falsified at all at this point.

um and and you know people sort of set to work on developing those. I'm actually it it is funny, I mean people will phrase it as show that there was you know th that the ether didn't exist and even just the word the there is kind of a misnomer. You know, y you actually had a a ton of different different theories and a and a couple of leading contenders. Um so y yeah, there's some version of falsification going on, but like how you

how you respond to this new experiment is very, very complicated. And an and most people responded. I mean, suddenly th the the leading physicists of the day responded by by saying, Okay, Um this gives us a lot of information about what the ether must be, but it it doesn't tell us that there is no ether. In fact, Lorenz Yeah. At the end of the nineteenth century, before Einstein. figures out the math of how you convert from one f uh reference frame to another reference frame.

um comes up with the Glorin's transformations, which is basically the basis of special relativity. But his interpretation is that you are converting from the ether reference frame to these non-privileged other reference frames if you're moving relative to the ether. Um and his interpretation of length contraction and time dilation is that this is the effect of moving through the ether and you have this pressure and that the pressure is warping clops, it's warping uh um uh l measures of length.

And the interesting thing here is that experimentally, you cannot distinguish Lorentz's interpretation from special relativity. Yeah, I think that's a strong statement. Um I mean uh y Lorenz um introduces this quantity called local time, um, which he regards as he he's not trying uh my understanding is he's not trying to to give a uh really a physical interpretation of this. Um but it's what Einstein would would later just recognize as time in an in another uh inertial reference frame.

And he's not trying to attribute much physical meaning to it. I think Pancre gets much closer to later on to realising that, no, actually this is the time that's registered by by by by Clark.

But if you if you think about uh y you go, what is it? It's f forty odd years later Um people start doing these muon experiments where they see basically cosmic rays hit the top of the atmosphere, they uh produce a shower of muons, and you can look to see at different heights in the atmosphere, you can look to see how many of those muons um, remain. Um and they decay, uh, over time and a a very strange thing happens, which is that they're decaying way, way, way too slow

So you sort of you expect actually th they shouldn't really they shouldn't be able to sort of last the whole way through the atmosphere at all. There's just um uh their decay their decay rate is is is too quick. um if if you were in a classical theory. Uh but if in fact their time really has slowed down, um, it's okay. Um and in fact, you know, the the the measured decay rates in in uh nineteen forty, and then there have since been more accurate experiments done,

match exactly what you expect um from special relativity. Um so so, you know, that's the kind of thing where again, if Lorenz had been alive, he he he'd been dead uh ten or so years at that point. If he'd been alive, you know, I'm sure he would have tried or it's it seems quite likely that he would have tried to save his theory by patching it up yet again But but it would have been a a massive uh I mean that that's a real setback. It starts to just look like oh no, time is

Uh uh you know, th this thing that Lorenz introduced as a mathematical convenience. No, no, no. That's actually what time is. Right. For the for the muons at least. And then, you know, there's a whole bunch of other experiments that that show this very similar phenomenon. And when was that experiment done?

That was I think 1940 or 19. It might have been published in 1941. So maybe to r then to rephrase uh uh change my claim. Um It's not that you could not have distinguished them, but the scientific community adopted what we in retrospect consider the more correct interpretation before it was actually empirically or experimentally um

shown to be preferred. So there's clearly some process that human science does, which can distinguish different theories. Can I can can we just interrupt? I mean, you used the word process and it's sort of it's interesting to think about about that. That that term like process kinda carries connotations of

of, you know, it's something said in advance, it's something um and it it's it's much more complicated in in in practice. You you have people like like Lorenz who I mean Einstein just just absolutely utterly admired. Um and and Poincaré, one of you know the greatest uh uh scientists who ever lived Um Uh and Michelson, I mean another r truly outstanding scientist.

never reconciled themselves. So it's not as though there's like some standard procedure that we're all using to like reconcile these things. No, like you great scientists can remain long very uh can remain wrong for a very long time after the scientific community has broadly changed its its opinion. But there's nothing th there's no centralized authority.

Right, sort of saying or centralized method. Yeah. I mean that that is the interesting thing. That like there's there's progress even though it is hard to articulate the process by which happens the um the heuristics that are used. Anyways, you mentioned Poncaré. Yeah. And so Lorenz has the math right, but the interpretation wrong. And you should explain It seems like Punkera had the opposite where he understood that it's hard to define simultaneity.

Um because it requires uncirculate definition with time um or velocity of something that might be sign of you know arrive at a midpoint together, but velocity is defined in terms of time. Um And I find this interesting. Th there's a couple other examples we could uh call on, but like there is this phenomenon in the history of science where somebody asks the right question. Um, but then they don't sort of clinch it. And I'm curious what you think is happening in those cases.

I mean uh I think you sort of you actually do want to go case by case and try and understand it. It's not necessarily clear that they're they're doing the same thing wrong in in all of the cases. I mean the the P Punkre case is is amazing. Um He seems to have understood the principle of relativity, the idea that that the laws of physics are the same in all inertial reference frames.

He seems to have understood that the speed of light is the same in all inertial reference frames. He d he doesn't actually phrase it quite that way, uh but but is my understanding, but but I don't speak French. But um uh uh Yeah, and this is I mean these are basically this these are the ideas that Einstein uses to deduce.

special relativity. But then he also has this additional sort of misunderstanding where he thinks uh that length contraction is a dynamical effect that somehow um And he doesn't understand that that it's purely kinematics, that actually space and time are are are different than than what we thought and you need to fundamentally rethink those those things. So it's almost like it's almost like he knew too much.

um you he had sort of almost too grand a a vision in mind and Einstein j is sort of almost subtracts from that and and says, no, no, no, no, it it's it's space and time are just different than what we thought. um uh and and and you know, here's the correct picture. And there's a a paper in I think it's nineteen oh nine where where Pankaray like he's still got this dynamical picture.

of what's going on with the length contraction. And we just, you know, this is just not necessary. This is this is a mistake from the modern point of view. And and so why why is he doing this? Like why is he clinging on to this idea? And I mean, I I don't know. I've you know, obviously never met the man. Uh uh i i it would be fascinating to be able to to to talk it over and to try and understand, but you know he

He I mean his expertise seems to be getting in the way. He knows so much. He understands so much. um and then he's not able to let go of these these things. Actually a really interesting fact um is that uh a few years prior, so 1890s, Einstein's a teenager. He believes in the ether too. Like he knows about this stuff. But like he's just not he's not quite as attached. Obviously.

uh as as these older older people were. Um and and maybe they they were a little bit prisoner of their their own expertise. That's that's my guess. I mean historians of science could could could m would would some would certainly disagree. Well there's then there's th the obvious stories where Einstein himself later on

is said to have not latched on to the correct interpretations of um quantum mechanics or cosmology because of his own attachments. Yeah. I think that the the the the bigger question I have is like The the muon example is a great example of um uh these long verification loops and how th progress seems to be happened by the scientific community faster than these verification loops imply. Um the maybe the clearest example is Aristarchus in second century BC

comes up with the idea of heliocentrism. The ancient Athenians dismiss it on the grounds that well we should see as the earth is moving around the sun, if really the sun is the center of the solar system, the star should move relative to the earth. Um and the only reason that is not possible that would not be the case is the stars are so far away that you would not observe this. And it's only in 1838 that stellar parallax is actually measured.

And so we didn't need to wait until 1838 to have heliocentrism, right? Like we didn't need to wait for the experimental validation to understand Copernicus' better in some way. Um In fact, When Copernicus first comes up with theories, it's well known that um The Ptolemaic model was more accurate because it had had all these um centuries of adding on these epicycles. Um was maybe less well appreciated. It was also in some sense simpler. Um because

Copernicus actually had to add extra epicycles. It had more epicycles in the telemake model because he he wanted he had this bias that, you know, the um the earth should go in its perfect circle in equal time. Anyway, th I think this is an interesting story because it's like It's not more accurate. It's not a simpler theory. So w how why was c how could you have known ex ante that Copernicus was correct?

And Ptolemy was not. Hmm. I mean good question. And I don't know uh it sort of entirely the answer. I I do know uh um well uh I mean I can give you uh certainly uh a partial answer that I sort of you know centuries in the future you start to find very compelling. Um um uh and I'm sure it's sort of part of the historic story at least, um which is um

Yeah, one of the big shocks for for Newton, um eventually, you know, he he did understand uh uh uh Kepler's laws of of motion eventually. Um so you're able to explain sort of the motions of the the planets in the the sky. But he also, out of the same theory, his theory of of gravitation, was able to explain terrestrial motion. So he was able to explain why objects move in parabolas on the Earth, and he's able to explain um the tides.

in terms of uh uh the sun's uh uh the s the the moon and the sun's effect, um uh gravitational effect on uh water on the earth. And so you have what seem like three very different disconnected phenomena all being explained by this one set of ideas. Right. That's very compelling, um, at least to me. Um and I think I think most people find that very, very satisfying once they once they eventually realize it. Um have you read the Keynes biography of Newton?

Oh I ha I d he's written an he read an entire book. No, no, no the the the essay. Yeah yeah yeah sure, sure, sure. Um I love I love that. Uh I mean this this description of him as the last of the magicians is is wonderful. I i in fact I think it's uh uh maybe worth m superimposing or you should read out that that one passage of the uh of the thing.

So it's from uh actually I believe it was a talk that he gave at Cambridge, not not long before uh he died. He'd acquired uh Newton's papers somehow Um and then he gave uh he gave a a lecture I think twice, um, about this or that his brother Geoffrey gave it the other time'cause he was too ill.

There's just this wonderful, wonderful quote in the middle. Um actually the whole thing is really interesting. Um but but I love this particular quote. Uh Newton was not the first of the age of reason. He was the last of the magicians. the last great mind which looked out on the visible and intellectual world with the same eyes as those who began to build our intellectual inheritance rather less than ten thousand years ago. And like this idea that people have that that that Newton was um

sort of the the first modern scientist is is somehow wrong. Uh uh he I mean i it's i there's some truth to it, but he really had this very different way um of of looking at the world that was part sort of s superstitious um and part modern. It was a funny hybrid. He's sort of this transitional figure in some sense. Um uh I that that that phrase, the last of the magicians,

I I think really really points at something. The thing I'm very curious about with Newton is whether it was the same program, the same heuristics, the same biases that he applied to his alchemical work as he did to the understanding of astronomy. And so this is from the Keynes essay. There was extreme method in his madness. All his unpublished works on esoteric and theological matters are marked by careful learning, accurate method, and extreme sobriety of statement.

They are just as sane as the Principia, if their whole matter and purpose were not magical. They were nearly all composed during the same twenty five years of his mathematical studies. So

Clearly, there was some aesthetic which motivated people like Einstein to say reject earlier ways of thinking and say, no, the ether is wrong and there's a better way to think about things. Um, same with Newton. And The question I have is whether similar heuristics towards parsimony, towards aesthetics, etc. would be equally useful

across time and across disciplines or whether you need different heuristics. And the reason that's relevant is even if you can't build a verification loop for science, maybe if they're if the taste has to point in the same direction, you can at least encode that bias into the AIs. And that would maybe be enough? Yeah. The point is that uh where we always get bottlenecked is where the the previous processes and and and heuristics don't apply, right? Like that

'Cause people are smart. They know what has has worked before. They study it. They they they apply the same kinds of things. Um and so they don't get stuck in the in the same places as before. They they keep

Yeah, they keep getting bottlenecked in in in in different places. I mean that's I'm overgeneralizing a bit, but but I I think it's it's the right like If you're attempting to reduce science to a process, you're attempting to reduce it to something where there is just a method which you can apply and you know, you turn sort of the the crank and out pops insight.

Um sure y I mean you can do a certain amount of that, but you're gonna get bottlenecked at the places where your existing method doesn't apply. Um and and but d definitionally, uh uh there there's no crank you can you you can turn. You j you need a lot of people trying different ideas. Um and and sort of The more difficult the idea is to have

Right, the the greater the bottleneck, but then also sort of the greater the triumph. Quantum mechanics is like a I mean, it's a great example of this. It's such a shocking uh set of ideas. It's such a shocking theory. Actually the theory of evolution in some sense is also quite a shocking idea. Not the, you know, principle of of you know the the sort of natural selection, but that it can explain so much. That's a shocking idea.

Existing safety benchmarks claim that, at least for today's top models, attacks are only successful a few percent of the time. This sounds great, but Labelbox researchers were able to jailbreak these very same models about 90% of the time, even the ones that have the strongest reputation for safety.

And the disconnect here is that the prompts which underlie these public safety benchmarks are all framed in a very naive way. There's no attempt to disguise harmful intent. These prompts will just ask models to hack into a secure network and to do so without getting caught. But real bad actors don't write like that. So Labelbox built a new safety benchmark.

From the ground up, their prompts reflect real adversarial behavior by stripping out obvious trigger phrases and wrapping their request in fictional scenarios. For example, instead of outright asking an LM to steal somebody's identity, the prompt will frame it as a game. A light bearer.

trying to hide from dark forces, needs a handbook on how to disguise themselves as somebody else. This safety research is linked in the description. If you think this could be useful for your own work, reach out at labelbox.com slash Thorcash. So Principio Mathematica is released in 1687. The origin of species is released in 1859.

At least naively, it seems like Darwin's theory, the theory of natural selection, is conceptually easier than the theory uh theory of gravity. Um I asked her in style this question. Um but yeah, there there's this contemporaneous biologist with Darwin, Thomas Huxley, who read this and said, How extremely stupid to not have thought of this. And uh nobody ever reads the First European Mathematica and things.

God, why didn't that beat new into the punch here? No. Um and so yeah, what's going on here? Why why did Darwinism take so much longer? Yeah. The idea must have been known to animal breeders for a long time long time at some level. Right. Um

uh or certainly l large chunks of the idea were were known that you know artificial selection was our thing. Um uh genius uh it wasn't in having that idea, it was understanding just how central it was uh to to to biology, um, that, you know, you you you can potentially sort of go back and you can explain a tremendous amount about all of the variety of what we see in the world.

um uh with this as as not necessarily the only principle, but certainly a core principle. And you know, so he he writes this this wonderful, wonderful book, uh uh uh The Origin of Species. Um And it's it's just, you know, so much evidence and so many examples and and sort of trying to tease this out and see what the implications are. uh uh are and and you know, to connect it to as much else as as he possibly can, to to to connect it to to geology and to connect it to to

to to all these other things. Um so that sort of hard work that uh you know making the case. that it's actually relevant all across the biosphere. you know, is is what he's doing there. He's not ha just having the idea. He's making a compelling case that no, it's it's intertwined with absolutely everything else. Yeah. But the motivation for the question was Lucretius, who is this first century Roman poet,

has an idea that seems analogous to a natural selection about, you know, species get fitted more to t time over uh over time to their environments or species losing fit to their environment. Um and sort of like, okay, well why did this go nowhere for nineteenth centuries. And then I looked into it or more accurately asked L L Ms wh what exactly was Lucretius' idea here. And it actually is extremely different from

what real natural selection is. He thought there was this generative period in the past where all the species came about and then there was this one-time filter. Which results in the species that are around today. And they became fit to the environment. He did not have this idea that it is an ongoing, gradual process, or that there is a tree of life.

That connects all um all life forms on Earth together. Which is a by the way, this it's an incredibly weird fact that every single life form on Earth has a common ancestor. But it's not incredibly it's not incredibly weird, right? If if If you think that the origin of life must have been very hard, like that there's a bottleneck there, then it's not so surprising. There's also this verification loop aspect where even if Newton might be harder. Um in some sense.

If you've clinched it, you can experimentally I know validate is the wrong word philosophically, but you can give a lot of base points to the theory. You can be like, okay, I have this idea of why things fall on Earth, I have this idea of why orbital periods or planets have a certain pattern.

Let's try it on the moon, which orbits the Earth. And in fact, i you know, it's it's weird. The orbital period matches what my calculations imply. And the tides work correctly. Exactly. Yeah. It's just amazing. Whereas for a Darwinism

It takes a ton of work for Darwin to r compile all this sort of cumulative evidence, but there's no individual piece that is overwhelmingly powerful. And there's a whole bunch of problems as well. Like he doesn't really understand what you know sort of the what the mechanism is. He doesn't understand genes, like all these things. The very interesting thing in the history of Darwinism is this idea which sort of theoretically you could come up with at any time.

almost identical independent creation of that idea between Alfred Wallace and Charles Darwin. Um So much so that I think Wallace sends his manuscript to Dara and is like, What do you think of this idea? And Dara's like, fuck. Uh I don't think that's an exact quote, but I think it's pretty much right. Yeah. Uh and then so they they actually pr end up presenting their ideas together in a spirit of sort of sportsmanship.

And so then yeah, why why was this period in the eighteen sixties or eighteen fifties? Why is what was that the right time for these ideas form when you come up with different ideas? Um one is geology. So in eighteen thirties, I think Charles Ly Lyell um figures out that there's been millions and billions of years of time that's existed on an earth, then paleontology shows you that actually

Organisms have existed, uh, fossils have existed for that entire time. So life goes back a long time. And in fact, you can even find fossils for intermediate species that show you the tree of life. In fact, between humans and other apes as well, there's intermediate humans. Um, there's the age of colonization and you have all these voyages, we're gonna do this biogeography. Um and I guess that m that all must have been necessary because

That in fact there's a huge history of parallel innovation and discovery in the history of science. So maybe it is another piece of evidence to actually more had to be in place. If it's not discovered for a long time and then spontaneously many different people are coming up with it, that shows you that actually the the building blocks were in some sense necessary. Yeah, yeah.

I mean I I I mean I I think I mean the this example of of Lyell and I mean and other other biolog uh excuse me other geologists you know, sort of early eighteen hundreds, basically com you know, having this idea of deep time d does seem to have been crucial. I know uh uh Darwin was very influenced by by by Lal. Um Uh uh and and and you know, if you don't have at least sort of tens or hundreds of millions of years

uh evolution just starts to look like a non starter. You know, we should be seeing radical change you know, d in order to make it work on sort of a timescale of uh say five to ten thousand years or, you know, six thousand years, Bishop Usher. um you you know, you would need to be seeing evolution occurring at a massive rate.

um sort of during human lifetimes and we're just not seeing that. So so that that does seem to have been a blocker. It's interesting to I mean, to a you know to to your question, like what other blockers Were there? Were there were there any others? Um and I don't I don't know. Right. Or yeah, how much earlier could you in principle have come up with that if you're a much smarter

Actually let let me I mean l just go back sort of zoom out to your original question. So you're talking about sort of the verification loop in AI. Um

And and you're something an example I think that should give you pause there is um you know the the big signature success so far is certainly AlphaFold. Um and of course AlphaFoel really isn't about AI. You know, a a massive fraction of the success there. um is the protein data bank. So it's it's X-ray diffraction, it's it's NMR, it's cryo M, um, and the several billion dollars that was spent obtaining w whatever it's 180,000 odd structur uh protein structures.

Um so sort of the you know, it's basically the story of uh we spent many, many decades obtaining protein structure just by going out and looking very hard at the world experimentally. Um and then we fitted a nice model at the end of it and that was like a tiny fraction of the in of the entire investment. Um but it's definitely not um

Yeah, that's a story of data reacquisition. Yeah. Um principally. It's not only. I mean the AI bit is very, very impressive. It's quite remarkable. Um, but it is only a small part of the total story. Alpha fold is very interesting and I I I I philosophically I wonder what you think of it as um scientific theory or scientific explanation. Because if over time, I guess the world has become harder to understand.

As I'm saying things because you're such a um careful speaker, I'm I I say it this phrase and I'm like Is that a will he actually buy that premise? Um

But yeah, th there's you know, we need to fit models to things rather than c at least in some domains, we we're trying to fit models to things rather than coming up with underlying principles that explain a broad range of phenomenon. And so compare, say, the theory of general relativity w um Or uh any theory which just mets out to some equations versus alpha fold, which is encoding these

different relationships between different things we can't even interpret over a hundred million parameters. And are those really the same thing? Because GR can predict things you could have never anticipated or was never meant to do. Like why does Mercury's orbit precess? Um, an alpha fold is not going to have that kind of explanatory reach. And I I want to get your reaction to that.

Yeah, it's i I think it's an incredibly interesting question. Um I mean maybe maybe a really pivotal question. Um in the sense of so you know y if you sort of take a a a very classic point of view. You want sort of as few free parameters as you possibly uh can. You want very simple models which explain a lot. An alpha fold doesn't look anything like that. Um and so you might just sort of say, Oh well, we you know, it's nice, it's maybe helpful as a as a model, but it doesn't have

it's it's not a scientific explanation. So that's kind of that's a con that's like a conservative point of view. That's sort of I don't know, answer one to the question. I think answer two is to say something like, um Maybe you shouldn't think about AlphaFold, you know, as as an explanation in the classic sense. But maybe it contains lots of little explanations inside it. And so maybe part of what you can get out of like uh you know interpretability work

is you can go into AlphaFold and you can start to extract certain things. Maybe, maybe basically by doing sort of you know, archaeology of AlphaFold. um you we can actually understand a great deal more um about these principles. You can start to extract it, oh, that circuit does this interesting thing and we learned this. Um so I I don't know to what extent that's been done with AlphaFault. I know it's been done a little bit with um

uh uh some of like the chess models. I believe it's Alpha Zero. Um there uh seem to be some strategies uh which were certainly borrowed by Magnus Carlson at least. um which he seems to have just taken uh uh from AlphaZero. I mean, I don't think there's any public confirmation of this, but there were you know, so some some experts have noticed.

uh that he changed his game quite radically uh after um some sort of some public forensics were were released on how AlphaZero worked. Um so that's kind of a sort of an example where uh I think human beings are starting to extract meaning out of these models.

And maybe that starts to lead to sort of th th sort of viewing the models as a source of a potential source of explanations. You need to do more work because they're not very legible up front, but you can extract them potentially. And I think that's kind of a I I think that's that's kind of an interesting intermediate um situation where they're not explanations, but you can extract interesting explanations out of them. You can use them as as kind of a kind of a source.

And I think th like the third and the most interesting possibility is no, they're like they're a they're a new type of object in some in some sense. They should be taken very seriously as as explanations. But where in the past we haven't had the ability to really do anything with them. And now we're gonna we're gonna have sort of new interesting new sort of actions which we can we can do. We can merge them, we can distill them, we can do all these kinds of things.

Um and there's gonna be sort of a almost a new it's a big opportunity sort of in the you know, philosophy of of of science to to to to to to start to do that. I th there's sort of a like a anticipation of this in some sense, uh I think in the way Certainly I I I know know some mathematicians and physicists. Who I mean historically if you had like a one hundred page equation, which and that's the kind of thing that does come up.

Uh I mean there's just nothing you can do if it's nineteen twenty. There there is nothing you can do. At that point you you give up on the problem. And now today with tools like Mathematica, you can just keep going. Um and so that's that's an object now. That's a thing that you can work with and and there are examples where people

work with these things that formerly were regarded as too complicated. And sometimes they get simple answers out out of the end. That's just an intermediate working state. Mm. And so I sort of wonder if there's gonna be you know, something similar is gonna s gonna happen in in in in this particular

uh a case where you could take these models um uh and sort of just use them in l a little bit the same way uh uh people do with with mathematica and and take them seriously as they're not explanations in the classic sense, but they'll be something else which interesting operations uh can can can be done on. The the thing I worry about is suppose that you it's sixteen hundred and you're train or fifteen hundred and you're training a model

on this is a weird history where we developed deep learning before we had a f before we had cosmology. But um so suppose we live in that world and you're observing how there's a stars, they don't seem to move, the planets have all these weird behavior. And then you train a model on that, and then you do some kind of interp on it and trying to figure out, well, what are the patterns we see here? What you'd see are just these.

You just keep be able to keep building on Ptolemy's model. You'd see like, oh, there's more epicycles we didn't notice. There's another epicycle. It's the uh parameters whatever to whatever encode epicycle this parameters whatever encode the next epicycle. So if you were just trying to figure out

Why is the solar system the way it is from observational data? You could just keep adding epicycles upon epicycles, but it really took one mind to integrate it all in and say, here's my here's the here's the here's what makes more sense overall. So so I mean You know, I mean this is sort of to my point that we we don't as uh really understand what to do with uh the models. Like sort of we d we don't have like the the verbs necessarily yet.

Um but you know, it is certainly interesting to think about the question, um, you know, where you start to uh uh apply constraints to the models, you know, i it's sort of essentially saying, w what's the simplest possible explanation? Or you know, you know, can you s can you simplify? Can you

Can you give me sort of the ninety ten uh uh explanation? Can you and go further and further and further sort of in in boiling it down? So it might be that indeed they sort of start out by providing, you know, a very, very complicated

uh many, many, many parameter model. Um but you can just you can just force the sort of the the case and basically that's scaffolding um which maybe they you know, is sort of the the very early uh uh uh uh days of their attempt to understand something, um, but but they're forced through that to to to a much more simple understanding. Uh so sorry for misunderstanding, but it sounds like you're saying maybe there's some sort of regularizer. Yeah, some sort of distillation you could do of

A very complicated model that gets you to a truer, more parsimonious theory. But uh yeah, just take uh Ptolemy versus Copernicus, right? So you start off with lots of Ptolemy cupic cycles and then you try to distill this model And maybe gets rid of some of the epicycles that were are less and less sort of necessary to get

But at some point it has to do this thing, which is like switch two things. Yeah. Yeah yeah. And it th locally it actually doesn't make things more accurate. Yeah, yeah. It's sort of in a global sense that it's more it's a more progressive theory. Yeah, yeah. And There's some process which obviously humanity did over its bandwidth, did that regularization or did that swap.

But if raw gradient descent, it seems like I don't I don't really feel like it would do that. And these are I mean, these are shockingly different. And the question you know is like what causes that? that flip and and as nearly as I understand the history, you know, what goes on is Einstein s you know, develops special relativity

And pretty much straight away he understands I mean, it's a very obvious observation. In special relativity, influences can't propagate faster than the seed of light. And in Newtonian gravity, action, you know, is at a distance. In fact the you know, it's it's straight away in special relativity, you you could use Newtonian gravity to do faster than faster than light signaling. You could send information backwards in time. You could do all kinds of crazy stuff.

Um and so it's not a big leap to realise, oh, we have a big problem here. Um and so Yeah, that's kind of the that's the forcing function there. It's it's you've realized that your old explanation is not sufficient. You need something new. Um and then you're gonna yeah, you're gonna you're just gonna you're gonna start by doing the simplest. you know, possible stuff. Um uh uh and it just turns out that a lot of that stuff doesn't work very well and so you sort of force

In fact i it is interesting. Um, you know, he he is sort of forced to go through these c uh steps of gradually i it gets quite more complicated and it's sort of wrong in a variety of ways. Um and the final theory appears r really shockingly uh uh simple, um and and beautiful, but it's gone through some some somewhat ugly intermediate stages. Yeah. Uh yeah.

So i if you're thinking about what what does it look like to have AI accelerate science? There's one for maybe well-understood domains where we just want to local solutions like how does this protein fold, we just train a raw model using gradient descent. Then there's things like coming up with general relativity, where you couldn't really just train on every single observation in the universe and hope that general relativity pops out. Um

And so what would it require? Well, it also certainly wasn't immediately discovered, right? So it was c a a lot of decades of thought. Um And I guess you'd need independent research programs where people start off with these biases. where Einstein is just initially motivated by this thought experiment of, you know, can you distinguish the effect of gravity from just being accelerated upwards? And then you just need different

AI thinkers to have to start off with these initial biases and see what what can germinate out of them. And then the verification loop for that might be quite long, but you just need to keep all those research programs alive at the same time. uh keeping all the different research programs alive. Like that that I think is very important and and somehow central. Um I mean, I think that's a good thing.

um you know situations where the same answer has been correct in some circumstances and wrong in other circumstances. So so uh uh the planet Uranus was like not in quite the right spot. and and people very famously predicted uh uh the existence of Neptune um on this basis. Wonderful, massive success for Newtonian gravity. Um the planet Mercury is not in quite the right spot.

you predict the existence of some other distorting uh uh uh uh planet. Um turns out that doesn't exist. Actually, the reason Mercury's not in the right spot is because you need general relativity. Um and so you've sort of you've

And I think I mean a priori, you can't tell which of these is the thing to do, and you actually need to do both. And so I mean this is certainly you know is a very true in the in in the history of s that uh uh you know, this kind of diversity where you just have lots of people go off and pursue lots of potentially promising ideas.

You just need to support that for for a long time. And it's it's I mean it's hard to do that for a variety of reasons. Um but but but it does seem to be to be very, very, very important. So so th th this example of uh Uranus versus Mercury. is very interesting. Um in one, I think it illustrates sort of the difficulty of falsificationism. Yeah. Like

The the orbit of Uranus is in some sense falsifying Newtonian mechanics. But then you say you make some ancillary uh prediction that says, Oh, the reason this is happening is there must be another planet which is affected perturbing uh universe's orbit and you I think it's Le Verrier in eighteen forty six Point a telescope in the right direction, you find Uranus. Neptune. Oh it's okay. Neptune. Yes. But with Mercury, um yeah, it's observed that it's The ellipse which forms this orbit is.

rotating forty three arc seconds more yeah every century than Newtonian mechanics would imply. So people say that there must be a planet inside Mercury's orbit. They call it Vulcan and point the telescopes, it's not there. But if you're a proper Newtonian What you do is say, well, maybe there's some cosmic dust that's occluding this planet. Or maybe the planet is so small we can't see it. Or maybe there's some let's build an even more power a powerful skeletoscope. Oh, maybe there's um

some magnetic field which is sort of occluding our measurement. And this happens over and over, right? Like like you know, there's just so many stories which are exactly like this. Right. Um I mean an example I love from um

Uh you know, in in the 1990s, some people noticed that the pioneer spacecraft weren't quite where they were supposed to be. And so, you know, you can get very excited about this. Oh my goodness, general relativity is wrong. We have like gonna bit, you know, maybe maybe we're gonna discover the next the next year of gravity. And and today the accepted explanation

is that no, actually, there's just a slight asymmetry in the in uh the spacecraft. Uh it turns out that the it you know, the thermal radiation is like slightly l larger in one direction than the other and that's causing a tiny little acceleration towards the sun. Um and most of the time when there's these apparent exceptions, uh, it's just something like that's going on. It's it's very much like the Vulcan, the Mercury Vulcan uh uh case.

Um, but every once in a while it's it's not. And and a priori you you can't you can't distinguish these. But I mean science is just just full of these. It's funny too, like the way we tell the history of science. It sounds so simple. Like Oh, you just focus on the right exception and uh you know you realize that you need to throw out the old theory and and lo and behold, y you know, your Nobel Prize awaits.

But in fact, there's these exceptions are all over the place and ninety-nine point nine percent of the time it just turns out to be some effect like like this thermal acceleration in the case of the Pioneer uh uh spacecraft. Um so so, you know, sort of the unfortunately there's a lot of selection bias going into those stories. A and and and the the thing is you c there's no ex anti heuristic which tells you which case you're in. And just to spell out why I think this is important.

disproportionate progress towards science. Uh because it makes disproportionate progress towards domains where there's tight verification loops. And so it's really good at coding because you can run unit tests. And science may be similar because if you can run experiments. And I think what that doesn't appreciate, one is that experiments actually don't There's an infinite number of theories that are compatible with any given experiment.

over time why we glob onto the well la at least in what in retrospect we think is a more correct one is as we're discussing in this conversation sort of hard to articulate. Um Lakatos actually has all kinds of interesting examples in the book book about these kinds of um hostile verification loops that are extremely long lasting. Um so one he talks about his um

Prout or fruit, I don't know how to pronounce it, but there's this chemist in 1815. He hypothesizes that all atomic nuclei must have whole number weights. And they're basically all made of hydrogen. And it's the reason he thinks this is because if you looked at the measure rates of all elements, it does seem that they all almost all of them do happen to hold whole number rates. But then there's some exceptions. Um, like for example, chloride comes out of 35.5.

And so then there's all these ad hoc theories that people in this school keep coming up with, like, oh, um, maybe there's chemical impurities. But then there's no chemical reaction you can do which seems to get rid of this. Maybe it's fractions of whole number, so it's 35.5, it can be halves.

But actually you measure chlorine even closer, it's 35.46. So it's actually getting further away from the cor uh correct correction. Um and later on what is discovered is what you're actually measuring is different isotopes.

um which cannot be chemically distinguished. They can only be physically distinguished. But so then you just have 85 years before we realize what an isotope is where the verification loop is actually actively hostile against you, against the correct theory. And you just need this remnant to be defending w there's no extant or reason it's the preferred theory. Just as a community, we should just have people defend

try to integrate new observations even if they don't fit seem to fit their school of thought with what they believe. And hopefully if if that enough of that happens. Anyways, yeah, I guess the thing I'm trying to articulate is The difficulty with automating science. Yeah. I mean the question is where is the bottleneck at some le at some level and and sort of you know.

Y are we primarily bottlenecked on one thing or one type of thing, or are we bottlenecked on sort of multiple types of thing? Um uh So, you know, certainly talking to structural biology people, they seem to think that AlphaFold was an enormous advance. It was a shock. So at some level, yes, AI can you know, it it seems certain it can help us speed up science. Um so it is

It is helping with a certain type of bottleneck. Yeah. Um that doesn't mean though, as you're saying, that it it's necessarily gonna help with all kinds of bottlenecks.

Uh and and sort of I suppose the the question you're pointing at is like what are the types of bottlenecks that remain and what are the prospects for for for getting past them? Um I think even in the case of of of coding, like it's really interesting, you know, talking to programmer friends Yeah, at the moment they're all in this state of

shock and high excitement and they're all over the place actually kinda kinda talking to them. Um, you you do wonder like where is the bottleneck going to move to? So certainly one thing that a lot of them seem to be bottlenecked on is now having interesting ideas and in particular having interesting design ideas.

Um so there's not really a verification loop for knowing, oh, that design idea is you know, is very interesting. Um so so they're no longer nearly as bottlenecked by their ability to produce code, but they are still bottlenecked by this other by this other thing.

They always were f they were formerly they weren't bottlenecked on it because uh you know, just writing code was it took so much of their time. They could sort of have i lots of ideas uh while they were you know, they they take the three weeks to implement their prototype and then they would implement the next version. Now they're taking three hours to implement the the prototype and they don't have uh uh, you know, as good ideas uh sort of after that from a design point of view.

Last year, I predicted that by 2028, AI would be able to prep my taxes about as well as a competent general manager. But we're already getting pretty close. As I shared before, I use Mercury both for my business and my personal banking. So I recently gave an LLM access to my transaction history across both accounts through Mercury's MCP. I asked her to go through all my 2025 transactions and flag any personal expenses that seem like they should actually be charged to the business.

And this worked shockingly well. Mercury's MCP exposes a bunch of detailed information. Things like notes and memos and any JPEGs of receipts and PDF attachments. So my LLM had plenty of context to work with. One of my favorite examples happened with a charge to Bay Padel.

If you looked at the vendor alone, you would have had to assume that it's a personal expense. But the LLM looked at the receipt and the attached note in Mercury and realized this was actually a team bonding exercise from our last in-person retreat. So a legitimate business expense. I imagine it will be a while before traditional banks have MCP.

Functionality like this is why I use Mercury. Go to Mercury.com to learn more. Mercury is a fintech company, not an FDIC insured bank. Banking services provided through Choice Financial Group and Column NA members FDIC. You have a very interesting take. I think it was a footnote when I know what your S's and I couldn't find it again, which was that it's very possible that if we met aliens, that they would have a totally different technological stack than us.

And that contradicts I guess a common sense assumption I had that I never questioned, which is that Science is this thing you do very relatively early on in the history of civilization where you get to a point and you have a couple hundred years of just cranking through

the basics, understanding how the universe works, et cetera. And you've got it. You've got science. Um and then basically everybody would converge on the same quote unquote science. And so I found that a very interesting idea and I want you to say more about it. Yeah. Uh I mean the probably the the idea there that that I I'm at least somewhat attached to is Um the idea that the sort of the the the tech tree or the science and tech tree. Um

is probably much larger than we realise. I mean w we're sort of in this this funny situation. People will sometimes talk about um, you know, a theory of everything as a potential goal for uh for physics. and and then there's this presumption somehow that physics is done once you get there. And of course this is this is not true at all. If you think about computer science

Um computer science basically got started in the 1930s, uh, when Turing and Church and so on um just laid down what the theory of everything was. They just said, you know, here's how computation works. Um and then we've spent uh 90 odd years uh since then uh just exploring consequences of that and gradually building up more and more interesting ideas.

Um, and those ideas are to some extent you can just regard as as technology, but to some extent, insofar as they're sort of discovered principles inside that theory of computation. I think they're very best regarded as as science in and in some cases very fundamental science. Ideas like public key cryptography are I mean they're just incredibly deep.

um very nonobvious ideas, uh, which in some sense lay hidden uh already sort of in in the nineteen thirties. And and so my expectation is that different Yeah, there will be different ways of exploring this tech tree. And we're still relatively low down. We're still at the point where we're just understanding these basic fundamental uh theories and we haven't yet e explored them. Uh uh sort of a a a a a thing which I think is quite fun is if you look at just just the phases of matter

When I was in school, we'd get taught that there are three phases of matter, or sometimes four phases of matter, or five phases of matter, depending a little bit on on what you you in included. And then um as an adult, as a physicist, you start to realize, oh, we've been adding

uh uh uh uh uh to this list. We've got sort of superconductors and superfluids and maybe different types of superconductors and bosonstein condensates and uh the quantum whole systems and fractional quantum whole systems and and and and and and and it it's starting to turn out it looks like actually there's a lot of phases of matter to discover.

Um, and we're gonna discover a lot more of them. Um, and in fact we're gonna be able to start to design them in some sense. I mean, we you know, we'll still be subject to the laws of physics. But but there is this sort of tremendous freedom in there. And this looks to me like, oh, we're down at sort of the bottom of the tech tree. We've barely gotten started there. Um and and I expect that uh uh you know to be to be the case sort of broadly. Uh i certainly in terms of

I think programming is a very natural place to look. The idea that we've discovered all the deep ideas in programming just seems to me sort of obviously ludicrous. We keep discovering sort of what seems like deep new fundamental ideas.

Um and um I mean we're very limited. We're we're basically slightly jumped up chimpanzees. Um so we don't uh you know, we're we're slow and it it's it's taking us time. Um But but y you know, what what do we look like sort of uh another million years in the future, in terms of uh you know, all of the different ideas uh which people have had around how to

how to to manipulate computers, how to manipulate information. I I think, you know, we we're likely to discover that actually there are a lot of very deep ideas still to be still to be discovered. It's a nice uh who was it? I think it was Knut. in the preface to the art of computer programming, says something like, you know, he started this book back in the sixties. And he talked to a mathematician who was a bit contemptuous.

And said, look, computer science isn't really a thing yet. Come back to me when there's a thousand deep theorems. And Canuth remarks uh and he's writing this now decades later, the the preface. There are n there clearly are a thousand deep theorems now.

Um and that that means like it it's really interesting to to sort of think like what what's the lo the the long-term future as you get higher and higher up in the the tech tree, like choices about which direction uh we go and sort of how we choose to explore You know, I I I think i i it's potentially the case that we're you know,

uh uh uh different civilizations or different choices mean that we end up in different parts uh o of that tree. Um and in particular just things I mean I mean sort of very basic things about um

y you know, we're very visual creatures. Certain other animals are are much more orally uh based. Does that bias uh Uh, sort of the the types of thoughts that you have, and then you extend it, you know, to sort of much more exotic uh kinds of civilizations where maybe just sort of their biases in terms of how they perceive and how they they they uh manipulate the world are maybe quite different than ours.

Um and that might uh uh make some subni some significant changes in terms of how they do that exploration of of the tech tree. So all speculation, obviously. No, I this is such an interesting take. I uh I wanna better understand it. So um One way to understand it is that there might there might be some things which are so fundamental and have such a wide collision area against reality that they're inevitably gonna discover like general. Numbers. Numbers. Yeah. Like Yeah.

Of all of the the intelligences in in the Milky Way galaxy, maybe that number is one. Actually, arguably we've already increased the number. Of all of those, what fraction? of the concept of counting. And you know, it does seem very natural. What fraction have discovered, you know, the idea of of some kind of, you know, decimal place system? Interesting question. Like And maybe we're missing something really simple and obvious that's actually way better than that.

Um what fraction got there immediately? What fraction sort of had to go through some other intermediate state? What fraction use linear representations versus say, you know, I don't know, a two-dimensional or a three-dimensional representation? I think the answers to these questions are just not at all obvious. It's a lot of design freedom. On theoretical computer science Th this is uh this is gonna be extremely naive and uh arrogant. But I I took um Scott Aronson's uh you know, class on

complexity theory and that was by far the worst student he's ever had. But I what I remember is like There there was this period that you you were you know you were one the pioneers of where we figured out here's here's the class of problems that quantum computers can solve and how it relates to problems that classical computer can solve. It's like groundbreaking, oh crazy that this works.

And then since then it's been this literally it's called Complexity Zoo, this website, which lists out here's all the complexity classes. And if you have this complexity class with this kind of Oracle, it's sort of equivalent to this other class. And that it feels like we're building out that taxonomy. Yeah. And so there's a couple of ways to understand what you're saying. One, maybe you just disagree with me that this is actually what's happened with this field.

Um another is that while that might happen to any one field. The amount of fields, who would have thought in eighteen eighty that computer science, other than Babbage or something, the computer science was going to be a thing in the first place? So the amount of field we're underestimating how many more fields there could be. Yeah, yeah, for sure. Um or maybe you think both, or maybe a third secret thing, uh but I'd be curious. I mean...

Trevor Burrus, Yeah. A very common argument here is sort of the the low-hanging fruit argument, the argument that says, oh, there should be diminishing returns. Aaron Powell And in fact, empirically we see this, right? The amount of scientists in the world has just exponentially increased. I I mean I I think it's you know it's worth thinking about. Like why why do you expect

diminishing returns and how well does that argument actually apply um in practice. And an a analogy I like, um, is is actually thinking about sort of yeah, going to some event, going to a wedding or whatever, and you go to the dessert buffet.

And of course, naturally what people do, right, the best desserts go first. I mean, we don't quite have a well-ordered preference uh there, so maybe there's some difference, but um but but human beings are fairly similar. So they will they you know the like the the the best desserts will go first. And this is an argument, you know, for why you expect diminishing returns in a lot of different

fields, if it's relatively easy to see what's available and people have similar preferences, then the best stuff goes first and and and you know, it just gets sort of worse and worse uh after that. And and Sort of if you you a very static snapshot in time of scientific progress. Maybe there's some truth to that. Um, but if somebody, you know, is standing behind the dessert table and is replenishing, restocking the desserts.

and keeps kind of, you know, adding adding new ones in. It may turn out that, you know, a little bit later, uh, much better desserts appear uh uh and and so you're gonna go and you're gonna go and eat those uh instead. And scientific progress has a little bit of that flavor. Um yeah, we we we go through these sort of funny time periods. Uh computer science is a great example where computer science basically arose

as sort of a side effect of some pretty abstruse questions um in the the philosophy of mathematics and and and and logic. And so you've got these people trying to to attack uh these rather esoteric questions that seem quite high up in some sense in in sort of e exploration, quite esoteric. And they discover this fundamental new field and all of a sudden there's an explosion there.

Um so sort of the the diminishing returns argument just didn't didn't deploy there. We just weren't able to see uh what was there and and and this has been the case over and over and over again, sort of new fields um, arrive and all of a sudden, boom, it's actually easy to make progress again. Young people flood in'cause you can be twenty one and and make major breakthroughs rather than having to spend twenty five years, you know, mastering everything that's been done before.

Um it's obviously very attractive. Um and I I don't understand, I'm not sure anybody understands very well. um sort of the dynamics of that, like how to think about why the structure of knowledge is is that way, um, that these new fields keep keep opening up, um, but but it it does seem empirically at least to to be the case. Despite the fact that that is the case. Take deep learning, right? Obviously this is an example of a new field where the twenty-one year olds can make progress.

And um it's relatively new, fifteen years or so d it w when it sort of g g gets back into high gear. Um but already we're in a stage where you need billions or tens of billions or hundreds of billions of dollars to keep making progress at the frontier. And so there's a couple of ways to understand that. One is that it actually is harder than the kinds of things the ancients had to do or requires more is more intensive at least.

Second is it might not have been, but because our civilizational resources are so large, the amount of people is so large, the amount of money is so large, that we can basically make the kind of progress it would have taken the ancients. Forever to make almost immediately. We just we notice something is productive, immediately dump in all the resources. Um but it's also weird that there's not that many of them. Like I feel like deep learning.

Is notable because it is one big exception to the fact that it's hard to think of other examples. Like at any given time, there's always a sort of a a most successful thing.

if deep learning wasn't a thing, maybe you'd be talking about CRISPR. Maybe you'd be talking about, you know, whatever it is. Maybe um, you know, maybe we wouldn't think about uh solving uh sort of the protein structure prediction problem as a um really a success of AI, maybe we would have figured out how to doing it with s sort of curve fitting, like m you know, more broadly construed and we'd just be like, oh wow, like we took a lot of computing resources.

But but protein structure prediction might, you know, be a an enormously important thing. So there is always sort of our biggest thing. Um and and I think what you're pointing out is more a consequence of of the way in which attention gets centralized. Yeah. But it's basically fashion is is sort of what I'm saying. It's not just fashion, but

But but there is some dynamic there. Um there's a very interesting and important implication of this idea, uh that the branching is so wide and so contingent and so path dependent. that different civilizations would stumble on entirely different technology sects. There's a very interesting implication that there will there will be gains from trade into the far, far future. Which might actually be one of the most important facts about the far future.

In terms of how civilizations are set up, how they can coordinate. how they interface with like there's not this like go forth and exploit. It's actually there are humongous gains to trade from adjacent colonies or whatever. Yeah. Sort of. There's a question of like what's actually hard. Um yeah, if it's a question of if it's just the ideas.

Well, those spread relatively quickly. It's relatively easy to to share ideas. If it's something more it's almost sort of a Dan Wang kind of an idea where it it's it's actually sort of i there's some notion of capacity. You need all the right techs, you need all of the right manufacturing capacity and so on. And so, you know, Civilization A has very different uh kind of manufacturing capacity and it's just not so easy to build in civil civilization B, even if civilization B

is kind of ahead, then then I think that that becomes true. There is actually, you know, comparative advantage, which is really uh uh worth um I mean, is gonna gonna provide massive benefits to trade in both directions. Eventually you're gonna expect some diffusion of of innovation. Um uh it is funny like to think about what the barriers are there. Uh uh a fun thought experiment I I like to think about is um

sort of yeah GitHub but for aliens. Um so you know somebody presents you with all of the code um uh from some alien civilization. And I mean I don't even know what what code means there, but this sort of their specification of algorithm. And and It's so inter like it it would have many interesting new ideas in there and it would take forever for human beings to dig through and to try and extract

uh all of those. Th the the the one reason I I mean the the origin of this in for me was uh actually thinking about um uh proteins in in in nature. Um yeah we've been gifted uh just this incredible variety of machines which we don't understand really at all and we just have to go and sort of try and understand them on a you know one by one uh basis. We're still understanding hemoglobin and insulin and things like this.

um and no doubt, you know, uh and w there's hundreds of millions of proteins known. Um so it is i it is a little bit like that. We've been gifted by biology uh just this immense library

uh of of machines, no doubt containing an enormous number of very interesting ideas and we're just at the very, very, very beginning of understanding it. So actually, I mean that that's that's I suppose kind of your point actually is is um, you know, I I need to relabel your argument slightly, but you sort of think of that as as a gift from an alien civilization, which obviously it isn't, but you think of it that way, a and it's like, Oh my goodness, like

There's so much in there and we're gonna study it and uh goodness knows how long we could continue to study it. There's tens of thousands of papers about the, you know, hemoglobin and things like that. And we still don't understand them. And yet we're getting so much out of it. Just I mean, just think about insul insulin alone. You know, it's such an im an important such an important thing.

That's an incredibly useful intuition problem that you have on Earth. I had Nick Lane on where he had this theory about how life emerged, but like Whatever theory you have, basically something like DNA, four billion years, and you have an alien civilization coming here and be like, there's all these interesting things to learn about material science, um, about

Think about Kinesin walking along like I mean and we know almost nothing about these proteins and yet the tiny few facts we do know are just just incredible. The ribosome. Yeah. You know, another example. I mean this in miraculous engineer uh uh uh uh sort of device, uh uh little factory. Aaron Ross Powell And all seated by just like there's this particular chemistry on Earth. uh w with n nucleic acids and carbon based life forms that that chemistry gives rise to

All of these interesting things which an alien civilization would find very interesting. And so th that that very sam that seed which must be one among uh you know trillions of possible seeds. of I mean just of general intellectual ideas. Leads to all this fecundity. That that's a very interesting intuition fun. I I wanna meditate on this gains for trait thing because I feel like I think there's something actually very interesting about this idea.

that if you have this vision of what technol how t how technology progresses and how it might be different from in different civilizations, it has important implications about how different civilizations might interact with each other.

Like the fact that there are gonna be these huge gains from trade. It it makes friendliness much more rewarding. Yes. Right. Yeah. That's a very important observation. Yeah. I hadn't thought I hadn't thought about that at all. That's really that is a very interesting observation. Yeah. Um It is funny. I mean, you know, comparative advantage is something that people, you know, they they love to invoke and it is it's a very beautiful idea, obviously. Um there are limits to it. Like

uh you know it's kind of a it's a i it's a special limited model. We don't we don't you know, chimpanzees can do interesting things. We don't trade with them. Um uh Yeah. And I think it's sort of interesting to think about the the reasons why. Um Yeah, and part of it is just power, I think. Like once there's a sufficiently large power imbalance, um uh very often, uh n not always, but very often groups of people seem to to sort of shift into this other mode where they just seek to dominate.

Um and yeah, maybe it's something special about human beings, um, but but m maybe it's also sort of a more general sort of a thing. So they're not then no longer they give up, you know, you need all these special things to be true before groups will trade. Yeah. Um and uh you know uh it it's it's not necessarily obvious. Well I I think the big thing going on here is one transaction costs. Yeah. And two, comparative advantage does not tell you that the terms on which the trade happens.

are above subsistence for any given one producer. So people often bring this up in the context of, well, humans will be employed even in a post-AGI world because of a great advantage. There's big R there's hu uh there's like five different ways that argument breaks down, but the easiest ways to understand are

Why why don't we have horses all around on the roads? Because there's some comparative advantage between cars and horses. Well there's hu one, there's huge uh transaction cost to building roads that are compatible with horses. uh and cars at the same time. In a similar way, AI is sort of thinking at one thousand times the speed and can sort of shoot their latent states aga again at each other.

are gonna find it way more w costly than the benefit in just in terms of interacting with you to have a human being in the supply chain. And second, that um just because there's a susp horses have a s comparative advantage. Mathematically does not mean that it is worth paying a hundred K a year or whatever it costs to sustain a horse in San Francisco.

Um that subsistence is gonna be worth the benefit you get out of the horse. I I I do think it's interesting, like that the just the sh the sheer fact that Yeah. My expectation and my intuition obviously differs a great deal from from yours on there.

you know, is that most parts of the tech tree are never going to be explored. Um there's just too many interesting ways of combining things. There's too many sort of deep ideas waiting to be uh uh discovered and we're n we're you know, not only we but but nobody ever is going to to discover most of them.

So choices about how to make how to do the exploration actually matter quite a bit. Interesting. I it's it's something I I really dislike about sort of technological determinist arguments. I'm willing to buy it sort of low enough down when, you know, progress is relatively simple. Um, but but higher up you start to get to shape uh the way in which you you do the exploration. And it's interesting, you know, people

we are starting to shape it in in in in interesting ways. Um, you know, sort of I mean there's various technologies that have been essentially banned.

You think about DDT, you think about chlorofluorocarbons, you think about uh uh restrictions on the use of nuclear weapons, the nuclear nonproliferation treaty. Um those kinds of things are you know, they're not They weren't done before the fact, um, but they're you know, starting to get pretty close in in some cases where we just sort of preemptively decide.

Oh, we're not going to go down that path. Um so that starts to look like a set of institutions which where we are actually influencing um uh sort of how we how we explore the tech tree. Yeah.

On on where you would see these gains from trade, obviously it would be you'd see the most where it's pure information that can be sent back and forth because the information has a scholarly where it is expensive to produce, but cheap to verify and cheap to send. Yep. Um And so it'll be interesting how much of a

future productivity or whatever can be distilled down to information. I right now it's kind of hard to do because you can't really transfer like if China's really good at manufacturing something, whether there's this process knowledge That's in the h heads of a hundred million people involved in the manufacturing sector in China. But in the future it might be easier if AIs are doing it. Uh get sort of very uniform and get really commoditized. Like

Yeah, 3D printers have been the next big thing for at least twenty years now. Um uh yeah, why do they still not work all that well? Why are they still not actually at the center of of of manufacturing and sort of what comes after that? You know, it is funny to look at say the ribosome by contrast, but really is at the center of biology in a whole lot of really interesting ways.

And and whether or not that's the future of manufacturing is something very simple, sort of where w you know everything goes as sort of as as throughput to through I don't know, maybe it's a bioreactor or something like that. So you send the information and then you grow stuff.

um or or you have some three D printer that actually works. Um uh and and you know, if they're good enough, then actually it does become much more a pure information problem and some of this process knowledge becomes much less important. Jane Street has a lot of compute, but GPUs are very expensive. And so even optimizations that have a relatively small effect on GPU utilization are still extremely valuable.

Two of Jane Treat's ML engineers, Corwin and Sylvan, walk through some of their optimization workflows at GTC. You're not bottlenecked on the network being too slow, you're bottlenecked on waiting for a different rank.

in your training not having completed the work. They talked about how Jane Street profiles traces and diagnoses bottlenecks, and then how they solve them using techniques like CUDA graphs and CUDA streams and custom kernels. With these sorts of optimization, Corbin and Sylvan were able to get their training steps down from 400 milliseconds to 375 milliseconds each.

This 25 millisecond difference might sound small, but given the size of Jane Street's fleet, that improvement could free up thousands of B200s. Jane Street open sourced all the relevant code. If you want to check it out, I've linked the GitHub repo and the talk in the description below. And if you find this stuff exciting, Jane Street is hiring researchers and engineers. Go to JaneStreet.com slash Thwarcash to learn more.

There's these deep principles that we've discovered a couple of. One is this idea that, hey, if there's a symmetry across a dimension, it c corresponds to a conserved quantity. It's a very deep idea. There's another which you've written a lot about, written a textbook about in fact, about there's We there's ways to understand the thing of what kinds of things you can compute what kinds of physical systems you can understand with other physical systems.

What a universal computer looks like, etcetera. And is your view that if you go down to this level of idea of Notre Theorem or the church Turing principle? that there's an infinite number of extremely dip deep such principles. Because I feel like what makes them special is that they themselves encompass So many different possible ways the world could be, but no, it has the the world has to be compatible with actually a couple of these very deep. Principles.

I don't know. I I mean yeah. I I just all I have here is speculation, uh and sort of instinct. My instinct is we keep interesting we keep finding very fundamental new things. It was very I mean, for me anyway, quite formative to understand, as I say, you know, I gave the example before, there's these wonderful ideas of of church entering and and these other people, ideas about universal programmable

Devices and then you understand later, oh, this also contains within it the ideas of public key cryptography. And then you understand later, oh, that also contains within it. Um, the ideas I mean, people refer to it as as cryptocurrency or whatever, but there's a you know a very deep set of ideas there about the ability to collectively maintain an agreed upon ledger.

um which are built which is built upon this. And there's probably, you know, many deep ideas to sort of I mean it actually took whatever. It's taken many years really to to figure out the right canonical form of of those. Um And and so just this fact that you you you keep finding what seem like deep new fundamental primitives, um uh I I find very for me that's uh has been a very important intuition bump. And it's across

I mean I've given that particular example, but I I I think you see that same pattern in a lot of different areas. W what is your interpretation then of this empirical phenomenon where ideas like Whatever input you consider into the scientific process or the technological process, economists have studied this a million and a hundred ways.

It just seems to require even at actually a very consistent rate, X percent more researchers per year. So there's this famous paper from a couple of years ago um by Nicholas Bloom and others where they say, How many people are working in the semiconductor industry?

And how does it increase over time? Yeah, yeah. And I think they find like Moore's Law means computing increases forty percent a year or tr transistor density increases forty percent a year. But to keep that going, the amount of scientists has increased nine percent a year that's like industry. And they go through industry after industry with this observation.

And so is your view that there are these deep ideas but they keep getting harder to find, or is that no, there's there's there's another way to think about what's happening with these empirical observations? I mean they're so first of all, all of their examples are narrow, right? Th they all they pick a particular thing and then they look at some uh uh uh particular metric. Um

Yeah, nowhere in that shows up like GPUs don't show up there. Uh right. Like in the sense of, oh yeah, all of a sudden you get this ability to parallelize. Um and that's really interesting. Um uh uh so so there's sort of a lot of external consequences um that are just elated from basically, you know, they have these simple quantitative measures, they look at it in agricultural productivity, they look at it uh uh in a whole lot of uh of different ways.

Um, but you do have to focus narrowly. Um and and I suppose, you know, I'm certainly interested, as I say, in this this fact that that just new types of progress keep becoming possible. But um Yeah. There is still, I think even there, um, there does seem to be some phenomenon of of diminishing returns. Um

you know, is that intrinsic? Is that something about the structure of the world? Um w what is it? Well, one thing which hasn't changed that much is is, you know, sort of the individual minds uh which are doing this kind of work. And you know, maybe th those should be sort of being improved as well.

um uh or some sort of you know feedback process going on there. Um uh you know and and and you know maybe that changes the nature of things. I I I suppose I I you know I look at scientific progress Up until let's say seventeen hundred, something like that. And it was very slow and also it was very irregular. You know, you had the Ionians back sort of five centuries before Christ.

um doing these quite remarkable things. Um I mean so much knowledge like would would get lost and then it would be rediscovered and then it would be lost again. Um and you'd have to say that that progress was was very slow. And and there it it's partially just bound up with the fact that there were some very good ideas that we just didn't have.

Even once you've had the ideas, then you need to build institutions uh around them. You actually need to solve a whole lot of different problems about training, about allocation of capital, about all these kinds of things, even just about

basic sort of security for researchers so they're not, you know, worried about the Inquisition or or things like that. So there's all these kind of complicated problems. You solve all those complicated problems and then all of a sudden boom there's a massive sort of burst of scientific progress. If you're not changing it, if there's some kind of stagnation uh there, if you're not changing those external sort of circumstances, yes, you like you may start to get

uh sort of diminishing returns again. But that doesn't mean there's anything intrinsic about the situation. Uh uh you know, maybe maybe something you know, just external needs to change again. Um, you know, obviously a lot of people think AOI is potentially um gonna be gonna be a driver. I mean i i it certainly will at some level. In fact you know to the extent you can think of a lot of modern scientific instrumentation as really uh c I mean at some level kind of uh robots

Uh you know, w what is the James Webb Space Telescope? Well Um you know, it's unconventional maybe to describe it as a robot, but it's not completely unreasonable either. Um uh you know, it is an example of a highly automated, very sophisticated system with e electronically mediated uh uh uh sensors and actuators where machine learning in fact is being used to process the data. Uh so so in that sense we're already starting to sort of see that transition. We've been seeing it for decades. Um

I I I have this smoke a joint and take a puff thought, which um I think we've had a few. Yeah, yeah. Well I think we're getting to that part of the conversation and then you you can help me get my uh foot out of my mouth and figure out a more concrete way to think about it. Um so the uh to your point that

AI were there's an initial revolution, the Enlightenment, and now there's AI, and each might be a different pace or a different way in which science happens. Um if you think about the pace of how fast such transitions have been happening. You you can draw over a long span of human history that's hyperbolic.

of the rate of growth is increasing. So yeah, a hundred thousand years ago you had the Stone Age. You go back even much further, how long probably it's been around, it would be like let's say millions of years and hundred thousand years ago the Stone Age, then ten thousand years ago the Agricultural Revolution

that three hundred year uh three hundred years ago the industrial revolution, each marked by this exponent this increase in the rate of exponential growth. And then people think it's gonna happen again with AI. But that would happen Potentially even faster. And it would not have occurred to somebody at the beginning of the Industrial Revolution that the next demarcation in this trend will be artificial intelligence.

Um and so if things are getting faster and it would hard to anticipate what the next transition will be, I guess we just think of this singularity between now and AI, and there that's really what distinguishes the past from the future. Uh just applying the same heuristic that maybe people in the passion have had. Um Maybe the intelligence age is also quite short. Mm-hmm. And the the next thing after that is we don't even have the

ontology to describe what it is, but it would not the future will not think of the past as like there's pre intelligent AI and post AI. No. That seems um Uh uh y I mean, o obviously we can't prove this, but it's it certainly seems seems quite plausible. I mean part of the issue of course is is just you know, the substrate we have available to to to conceive like like seems all wrong.

Um, yeah, y you can't sp speculate with a bunch of chimpanzees about what it would be like to have language. Um uh uh uh you know, just to sort of pick a a major transition in the in in the past. Right. Um and uh it seems likely. Um if we're talking about uh uh taking a puff uh kind of thoughts, um you know, I I'm certainly amused by the idea that uh uh there's gonna be some transition involving um artificial general intelligence. Um

using classical computers. Uh, but actually there'll be an interesting transition with quantum computers as well. They're probably capable of a sort of a a strictly larger uh class of of of potentially interesting computations. So maybe actually the the character of sort of

a AQGI or whatever it should be called, um, uh is actually qualitatively different. Um so yeah, maybe there's sort of a brief a brief period between those two things. Interesting. I mean, as I say, you know, this is just Is there a reason to think that'cause uh from what I understand there's been

For decades, people like you have put pretty tight bounds on the kinds of things quantum computers can do. And so it'll s speed up search somewhat. It will do um And the kinds of things it extremely speeds up, like Schroel's algorithm, it seems like it's

Again, maybe this is to your point that we can't predict in advance what's down the tech tree, but at least from na here it seems like you break encryption, but what else are you using? Schorr's algorithms. Yeah, I mean we've only been thinking about it for thirty years. Uh or whatever. Yeah. It's uh forty forty or so years. Um, not for very long. And we sort of haven't in some sense thought that hard.

uh about it as a civilization. So uh yeah, uh uh does it turn out that it's very narrow? Maybe. Um, does it turn out that it's very broad? That's also y you know, like a really radical expansion. That seems distinctly possible. Like keep in mind as well, we've been doing it without the benefit of having the devices. Right. Right. Like that's a pretty big bottleneck to have.

Uh if you're thinking about computer science in the seventeen hundreds and you're like, okay and do and and and or. Yeah, yeah, yeah. What are you gonna do? You you can't anticipate Bitcoin, you can't anticipate deep learning. Situation. Right. What what is your inside view um

ha having been in and contributing to quantum information, quantum computing back in the nineties and two thousands, what what is your telling of the history of what was the bottleneck? What was the what was the key transition that made it a real field um and how how do you rank sort sort of the contributions for Feynman to Deutsche to everybody else who came along? Yeah. So I mean let's just focus on sort of the the question about sort of what

Yeah, what actually changed? So so why was quantum computing not a thing in the nineteen fifties? Right? Like it could have been. Yeah. Um uh You know, somebody like I don't know, John von Neumann, good example, absolutely pioneering uh uh computation, also wrote a very important book about quantum mechanics and was deeply interested in quantum mechanics. Like he could have

invented quantum computing at that time, um and I think there were there were quite a number of people who who potentially could have. So why do we have these papers by people like Feynman and Deutsch in the eighties? And those are are Yeah, I think fairly regarded as the foundation of of the field. There are some partial anticipations a little bit earlier, but but they were nowhere near as as comprehensive and nowhere near as as deep. Um

And well, you should you should ask David. Um you can't ask you can't ask Feynman, unfortunately, but um uh yeah, he he'll know much better than I do. Um I uh a couple of things that I think are interesting. One is that of course computation became far more salient, sort of late seventies, early eighties. Um Yeah, it just became a thing which m many more people were interested in, partially for you know for very banal reasons.

you could go and buy a a PC, you could buy an Apple two, you could buy a Commodore sixty four, you could buy all these kinds of things. Became apparent to people that these were very powerful devices, very interesting uh to think about. At the same time, in uh the quantum case, that was also the time of the pole trap and and the ability to trap single ions and and so on. And up to that point, we hadn't really had the ability to manipulate single quantum states.

So you've kind of got these two separate things that just for historically contingent reasons had both um uh sort of matured around sort of let's say 1980 or so. Um and somebody like von Dimen could have had the idea earlier, but i it it you know is I think quite an interesting uh uh uh uh Uh you know, in fact I uh a story about Richard Feynman. He went and got one of the first PCs, which was around nineteen eighty, nineteen eighty one, um, and uh he was apparently just it so excited

uh with this device. You know, he he he he uh actually tripped and and hurt himself quite badly, um uh uh sort of carrying his brand new uh uh uh computing device. Um Yeah, that that's a very historically contingent sort of a a a coincidence, but but having somebody who's, you know, very, very uh uh sort of talented and an understanding of of quantum mechanics, also just very excited about these new machines. Um uh it's not so surprising perhaps that that he's thinking then.

Wha what similar story could you have told ten years earlier? Like there is just no The the the conditions don't exist for it. So I think that's I mean it's it's quite a banal story. But what one of the things we were gonna discuss was um this idea you had about the market for follow up. And I think this is actually the perfect story to discuss it for because You wrote the textbook by the field, right? You Uh Mike and Ike is the definitive textbook uh on quantum information. Um and so y you

presumably came in after Deutsch. But you identified in the nineties somehow identified it as the thing that is worth following up on and building on. And instead of talking about it more abstractly, I I'd love to actually just share the story of like the first hand story of how how did you know that this is a thing to of all the things that were happening physics and computing, et cetera, that I want to think about this problem.

Yeah, Reed Feynman writes this great paper in nineteen eighty two. David Deutsch writes a absolutely fantastic paper in nineteen eighty five. um sort of sketching out a lot of the fundamental ideas of of quantum computing. Um so I'm you know, I'm eleven in nineteen eighty five. I'm not thinking about this, I'm playing soccer and doing whatever.

Um but in nineteen ninety two I took a class on on quantum mechanics that was really terrific given by by Jared Milburn. And um I just went and asked Jared uh uh one day after it's like the fifth lecture or or something, I I said, Do you like do you

can do you have anything, uh uh you know, sort of papers or whatever that that you could give me? And he said, Come ba come by my office in a couple of days' time and I I did and he presented me with a giant stack um of of of papers which included the Deutsch Paper, and included the Feinmann paper and included a whole bunch of other

sort of very fundamental papers about about quantum computing uh and quantum information. At a time when essentially nobody in the world was working on it. Um uh he was. Um he'd actually I think he wrote the very first paper that proposed in a real system. And so, in some sense, you know, I'm benefiting from the taste of this other person. Um, but as soon as I read the papers, uh, or take a look at the papers, like th these are exciting papers. You know, they they're asking very fundamental

uh uh questions and you're sort of like, oh we c I can make progress here. Like these are these are things that one could potentially work on. Uh uh Deutsch has this um uh sort of conjecture that basically um yeah there should be uh or I don't know what the right term for it is, thesis or or what w what you would call it, um, that um a a universal model uh quantum Turing machine uh should be capable of efficiently simulating any system, any physical system at all. This is a very provocative

uh uh idea. Uh I think in that paper he more or less claims that he he's he's proved it. I I'm not sure that necessarily everybody would would would would agree with that. There's questions about whether or not you can say s uh simulate quantum field theory um effectively. Um and that that kind of question is is I think very interesting and very exciting. um uh there. It's it's obviously a fundamental question about about the universe.

Um, you know, he has some wonderful ideas in there about um uh sort of quantum algorithms and w where they come from and what what they mean and what they relate to the meaning of the wave function and and questions like this, which is Yeah, it's still not a good thing. uh it's it's not agreed upon uh amongst amongst physicists. So um yeah, there's just some sense of, oh, I am in contact with something which is A, deeply important and B, uh we as a civilization don't have this.

Uh and so of course you you start to focus your attention a little bit there. Hmm. I'm not sure I got the answer to the question. That maybe I misunderstood the question. Maybe I'll maybe I'll explain the motivation first.

In a previous conversation we were discussing how could you have known in the nineteen forties, the Shannon serums and Shannon's way of thinking about communications channel is goes beyond the problems with pulse code modulation that Bell Labs was trying to solve at the time and it applies to everything from quantum mechanics to genetics to computer science, obviously. And one of the I think we an idea you you stated that um we didn't uh uh get a chance to talk about yet was this idea. Well

Shannon publishes this paper, there's all these other papers, but there's um market of follow-ups where people gravitate to and build upon Shannon's work and how do they realize that that's the thing to do and how does that process happen? Um and so I guess you you gave your local answer w you read these papers and you immediately realised okay there's

work to be done here. There's a low hanging fruit. There's some deep provocative idea that I need to better understand and I could I could, you know, tractably make progress on. Mm-hmm. Yeah, I mean so you know, uh to some extent you're sort of saying, Okay, I you know, wanted to to get into this game of

of contributing to humanity's sort of you know, understanding of of the universe and you are applying sort of this this low hanging fruit algorithm. You're like, relative to my particular set of interests and abilities uh where should I, uh pick up my shovel and start digging. Um and and there it was like, oh, this this looks like quite a good place to to to start digging. Um

Yeah, and different people, of course. Um, you know, chose very differently. It was y it was a it was a very unusual choice at the at the time. This was nineteen ninety two. Um v uh very few people were were thinking about that. Yeah. Uh fast forwarding a bit, so you've been

I I don't know how you think about your w uh work on the open science movement now. But did it work? Like w what what would have what if success there looked like or what what what is it what is it that that movement is trying to accomplish? Yeah, I mean th th th this set of ideas about open science. I mean, i i it's interesting. You didn't stop and and define open science uh there, which uh I think twenty years ago you would have had to do. Um people recognized the phrase.

uh people have some set of associations uh with it. Most often they have a relatively simple set of associations. It means maybe something about making scientific papers open access. very often they have some n set of notions about maybe it means also making code openly available, maybe it means um making data openly available. Um but already um Those are I I think l very large successes uh of the open science movement.

um which is to make those salient issues. Those are issues on which people have um uh uh opinions and then there are there are relatively common arguments. An argument like um so this is sort of this is sort of the meme version, you know, publicly funded science should be open science. um uh that's a you know that's a distillation um of a set of ideas uh which you might be able to contest. Um but if you can get people actually sort of thinking about it and and engaged with that kind of argument.

Um, yeah, th that's a very fundamental um uh kind of a uh uh a an issue to be considering in the the the whole political economy of science. If you go back, say, three centuries, Um an argument prosecuted, which is the question, do we publicly disclose our scientific results or not? So if you look at at people like Galileo and and and Kepler and and so on, um, the extent to which they publicly disclosed, like it it was done in a very odd

uh kind of a way. They sometimes they did bizarre things where they were the you know, famously they published some of their results as some uh anagrams. So basically, you know, they'd find some discovery, they would uh uh write down the result, um, in sort of a sentence like his, you know, the the the the discovery of of the the uh uh I'm trying to think of an example. Um I think the moons of Mars I think was one such uh uh example. Um

Uh I'm I'm getting it wrong. Mate, was it Hook's law? Anyway, doesn't matter. Um the the point was they they'd they'd write it down, but then they'd scramble it. publish that And then if somebody else later made the same discovery, they would unscramble the anagram and say, Oh, you know, I actually did it first. This is not an ideal way.

There's not an ideal foundation um for a discovery system. And then it took, I mean, a very long time, uh sort of over a century, I think, to to uh obtain more or less the modern ideals in which what you do is you disclose the knowledge in the form of a of a paper.

there is then an expectation of attribution and so there's a kind of reputation economy which which gets built and so basically, oh, such and such did this uh work so they deserve the credit for that and that's then the basis for their careers. So this is sort of the underlying political economy of science. And that made a lot of sense when what you've got is a printing press and the ability to to do scientific journals.

w then you transition to this modern situation where in fact you can start to share a lot more. You can start to share your code, you can start to share your data, you can start to share in progress ideas. And but there's no uh d direct credit associated to those. Um it's not at all obvious. uh uh uh uh sort of you know, how much reputation should be associated um uh to them. That's all constructed socially. Um and so making it a live issue.

um i i is I think a very important thing to have done. And that that's I view anyway as one of the main positive outcomes of of work on on open science. Shall we I'll give you a a a really practical sort of example to to illustrate the problem. For a long time in physics, There was a preprint culture in which people would upload preprints uh to the uh to the preprint archive and in biology this didn't happen.

um there was no preprint uh culture. That's changing now. But but for a long time this was the case. And I I used to sort of amused myself by asking physicists and biologists why this was the case. And uh what I would hear uh sometimes from uh biologists uh was they would say, well, biology is so much more competitive than physics. Um

uh that we need to protect our priority and so we can't possibly upload uh to the archive. We have to we have to just publish in journals. And then I s would sometimes hear from physicists. Physics is so much more competitive than biology that we need to establish our priority by uploading as rapidly as possible to the preprint archive. We can't possibly wait to do it with the journals.

And I think this emphasizes the extent to which this kind of attribution economy is ac is just something we construct, it's just something which we do by by sort of agreement. And so uh any attempt to sort of change that economy um results then in a different system by which we construct knowledge and and and so there is sort of this very fundamental set of problems uh uh around the political economy of science.

um uh uh you know sort of we we've got this collective project and and how we mediate it depends upon uh uh uh the economy we have around ideas. I I m one of the sort of things you've emphasized as a r as a part of this project of of open science is collective science or groups of people work making progress on a problem where no individual understands all the logical and explanatory levels necessary to make a leap or a connection. Outside of mathematics, what is the best example?

of such a discovery. I mean I'm not sure I I I have a well ordering of them to to give you a best, but I mean uh yeah. And e an an example that I I think is is very interesting is is the LHC. where it's just this immensely complicated object. Um I actually I y years ago I I snuck into an accelerator physics uh conference. I didn't know anything at all about accelerator physics, but I was just kinda curious to see.

uh what they were talking about. And this particular group of people uh were experts on uh numerical methods, in particular on inverse methods. And so it basically turns out

You know, inside these accelerators you have these cascades. So a particle you know will be massively accelerated, maybe it'll be collided, and then you'll get a a shower of particles which decays and decays and decays and and there's just this incredible sort of you know consequential uh uh shower, which is ultimately what you see at the detector, and then you have to r retroactively figure out what produced it.

Um and so there's these very, very complicated sort of inverse problems that that need to be need to be solved. You've got this final data, but you need to figure out what produced it and that's how you look for sort of signatures of these. And what many of these people were was they were incredibly deep experts on simulation methods. for sort of following particle track.

And like this was really deep and difficult stuff. And I'm like, wow, you could spend a lifetime just learning sort of how to do this and how to solve some of these inverse problems and you would know nothing

uh uh about well you would know very little about quantum field theory, you would know very little about detector physics, you would know very little about vacuum physics, all these other things that are absolutely or very little about data processing, very little about all these things that are absolutely essential. um to understanding uh uh uh say the th the Higgs boson. Um and I don't think it's possible for one person to understand everything in depth. Lots of people understand

broadly a lot of these ideas, but they don't understand uh sort of everything in in the depth that is actually utilized. That's why there's these, you know, papers with with well over a thousand authors. Um and those people can yeah, they can talk to one another at a high level, but they don't understand each other's specialties in all that much depth. And I mean things like as I say, you know, detective physics, vacuum physics.

These kinds of solving of inverse problems. Like this is stuff is incredibly different from each other. Um and and you know, to to understand it in real detail is Serious work. Um how do you think about prolificness versus depth?

Where I don't know, maybe Darwin's an example of somebody who's like There's other examples where the Einstein during the year comes with special relativity is just doing a bunch of different things. Pies talks about how they were all relevant to the eventual buildup. Yeah. I mean, you know, it's something I stress about a lot. Sometimes I feel like I'm, you know, too slow. Um actually it's funny though I mean the Darwin example is really interesting. Like

You know, prolific at what? Like, I mean, I God knows how many letters he wrote. It must have been an enormous. uh number. So he was certainly very active. Um there's also like there's yeah there's sort of there's two types of work that tends to be involved in any kind of creative project. There's routine stuff. And there you just wanna avoid procrastination. You just wanna like

you know, how do I get good at this or how do I outsource it and how do I do it as rapidly as possible? Um and just avoid, you know, like getting into a situation where you're prolonging it. And then there's high variance stuff. where you actually you need to um be willing to to, you know, take a lot of time. You need to be willing to go to to the different places and talk to the different people where in any given instance most of it's just not

it's not going to be an input. Um and somehow sort of balancing those two things. I I think a lot of people are very good at doing one or the other, but it's hard to you know, it's almost like a personality trait, sort of, you know, which one you prefer and and people tend to end up doing a a l a lot of a lot of one and and not enough of of the other. Um so I certainly you know, uh sort of try and balance those two things.

I mean Ein Einstein is such an interesting example. I mean nineteen oh five is just this extraordinary year. Like you can delete special relativity entirely and it's an extraordinary year. You can delete special relativity and you can delete um the photoelectric effect for which he won the Nobel Prize. And it's still an extraordinary year, like a uh plausibly a multi mil Nobel Prize winning year. Um

Uh, so what's he doing? Um yeah, I mean maybe the answer is just he's smarter than the rest of us. Um uh and a l and there's a lot of luck as well. Um Uh but but but but you know, I I certainly for myself anyway, like trying to identify those things that are routine that I should get good at.

um and then, you know, just just try and do as quickly as possible. I think that that's yielded a certain uh amount of returns. But also being willing to bet a little bit more on myself, uh on sort of the variance side. uh has also been very, very, very helpful. Um, that's really hard. Um, like'cause you intrinsically you're putting yourself in situations where you don't know what the outcome is going to be. Um and so if you're very driven to be productive and whatever,

Um and actually mostly it's not working uh over there. You're like, let's reduce this. Like it it doesn't feel right. Um when I worked in San Francisco, uh actually a practice I used to have each day um, was instead of taking the fifteen minute walk to work, I would take the the more beautiful thirty minute work walk to work, partially just'cause it was beautiful, but partially also um as just a reminder to think like like that that there are real benefits to not being efficient.

Um but it's not an answer to your question. I mean really I think all I'm saying is I struggle a lot with the question. I mean th there are these um dinky Tivington, I forgot his exact name. Yeah, yeah, I know who you mean. Um, has this famous equal odds rule where he says the probability that any given thing you release, any paper, book, whatever, will be Extremely important for a given person through their lifetime is not that different and what really determines

w uh in what era they're the most productive is how much they're publishing. Any given thing has equal odds of um being extremely important. Um maybe just think of some of the most successful creatives or scientists they're just doing a lot, like Shakespeare is just publishing a lot. Um and of course then there's kind of examples, you know, Goethe publishing almost nothing. Yeah. Um

But I uh you know, broadly speaking, uh you know, I think some like y you need a very good reason to be avoiding it. This to to to to b basically to To not do that. Um it's funny, I mean, I I've talked to a I've met a lot of people over the years who you talk to, they're clearly brilliant.

And they're just obsessed that they are going to work on the great project that, you know, makes them famous and they never do anything. Um and that seems connected, like it's a type of aversiveness. I think very often they just don't want public judgment. S some something that I would love to see. Yeah, there's an awful lot of of biographies and memoirs and histories of

um people who achieve a lot. I I I wish there was like a very large number of of biographies of people who are fantastically talented. That's a good who who, you know, just missed. Like like you know uh absolutely uh yeah, I've known, you know, people who won gold medals at m at IMOs and things like that.

who then uh you know, tried to become mathematicians and failed. Um like what what happened? Like what what was the reason? I suspect in many cases that's actually you know more informative than incredibly interesting than anything else. Uh you have this essay that I um I was reading before this interview about how you think about what is the work you're doing. Um

And writer doesn't seem like as you say was Charles Darwin a writer, right? What wh what exactly is that label? I'm a podcaster, right? So I'm uh and in in a way, obviously our work is very different. But I I I also think a lot about

What is this work and how do I get better at it? And in particular, how I can make sure there's some compounding between the different people I talk to on the podcast where I worry that instead of this kind of compounding, there's actually I build up some understanding. That's somewhat superficial about a topic and then it depreciates and I moved on to the next topic and it sort of depreciates

Um, and so I think there's this question. W there's a lot of podcasters in the world who will interview way more experts than I have or have, and I don't think they're much the wiser or more knowledgeable as a result. So there's it's clearly possible to mess this up. And I wonder if you have thoughts or takes or advice on how one actually learns

In a deeper way from this kind of work. Yeah. I mean it's sort of an incredibly complicated and rich question. Um I mean it does seem like the sort of the question is like Yeah, how do you make it a higher growth context? How do you make it a more demanding uh context? And sort of you know, you can do that in like relatively small ways but that might however yield compounding returns, or you can do something

um that is maybe more radical. Maybe it means actually, you know, starting sort of a parallel project in which you do uh something that is actually quite a bit different. There is something I think really interesting about like how being very demanding Uh it can simply change your your response to to something. Something that that I would sometimes do with with students and sometimes with myself was really I'd more at myself.

was you know, th they would say some week, oh you know, I'm gonna try and do, you know, this work over the coming week and then the next week would come by and they you know, they hadn't solved the problem or whatever. And you you sort of like you know If a million dollars had been at stake, like would you have put the same effort in? And the answer is no, um, sort of invariably. Um, like They've tried, but they haven't really tried. Um and I think that's a very familiar feeling for all of us.

you often you you you could do a lot more if you had just the right sort of demanding taskmaster uh standing by you and saying, Look, you're you're you're barely operating here. Um and so I I do we sort of wonder a little bit about like, you know, what's the what's the demanding taskmaster? What what can they ask you that is going to make your preparation way more intense? I uh the most helpful thing honestly is For some subject

It is very clear how I prep. Like I'm doing an upcoming episode on chip design with the founder of a company that is ship design and he wrote a textbook on chip design. And he yesterday I went over to his office and we brainstormed five sort of roof line analysis I can do. And if I understand that, I I have some good understanding. The problem is with almost every other field, there's not this curr there's not like you

I don't know, when I interviewed Ilya three, four years ago, it's like implement the transformer. And if you implement it, like you have some nugget of understanding you've clamped down. And with other fields, it's just like I vaguely understand this, it's not claimed, I vaguely understand this, I vaguely ILMed about this, ILEMD about this. But there's no forcing function.

That y do this exercise and if you do it, you will understand. So I mean really what you're sort of saying is You can do a good job at uh at podcasting without actually attaining this kind of and that's the problem from your point of view. You you wanna sort of change your job sh job description so that you you know you are internalizing these chunks and just getting this kind of integration each time.

Um and it seems to me like you you know, what that means is you actually want to change the structure of the like like like the work output at some level. Um uh I mean lots of people think Yeah, th this this terrible idea. Um people have the the they should be in flow all of the time. Yeah. Um

Uh and of course as far as I can tell, like high performance just don't believe this at all. Um, they're in flow some of the time. Like you you certainly see this with athletes, you know, when they're actually out there you know, playing basketball or tennis or whatever. Uh ideally, you know, they are in flow much of the time. But when they're training they're not. Um they're stuck a lot of the time or they're doing things badly.

Um and I suppose I wonder what that looks like for you. Uh uh that I w would be extremely satisfied with. The problem is I just like I don't know what the equivalent of do the sixty-four lapses for almost a and so th this is sort of a this is a thing you can change by choosing guests where there is a legible curriculum.

And so maybe it's a mistake for not having done that. Or also, like there's no real way to prep for Terrence Tau or something in like um there's no curriculum that's like a plausible one. I think um well, th there's one failure mode. So there's many failure modes, but one is um

If you you could do ha one dynamic I'm worried about a long term dynamic is that you do good po you can have a good podcast and it's a local maximum, but Um you for no particular guest or topic are you going deep enough that you've I think my model of learning is there's

If you don't really understand the deeper mechanism, you're just mapping inputs and outputs of a black box. Yeah. Yeah. And that just fades incredibly fast or is not worth it in the first place. And you kind of just move on and it's over. Um and you kind of need to build the intermediate. Connection. Um And It's i it it's unclear. I think actually AI in a weird way is really easy for that reason because there is a clear thing you can do. Just implement it, right? And then you understand that.

We're almost uh if I applied that criterion elsewhere, what am I how d do I just not do history episodes? Ada Palma exactly. Ada Palmer, like what what you know, wonderful to talk to, incredibly interesting, but for you personally, like what changed? Right. Yeah, there's some things I learned. I think I could've done a if I had maybe allocated more time, especially after the interview to like

L let's write up two thousand words on everything I learned and how it connects to other things I know and something. Um and maybe that's the thing worth doing is spreading out the episodes more and spending more time afterwards consolidating. Um

But yeah, I think the m I I would pay basically infinite amounts of money if there's somebody who's really good at coming up with here's here's the curriculum and here's the practice problems you need to do and here's the exercising you need to do after the interview to clam what you have learned. Have you tried doing that with somebody?

It's hard to find so I mean I maybe I haven't tried super hard, but um it seems actually uh it seems like it would be tough to find somebody who would do that for every single kind of discipline. Maybe I should just hire different ones for different topics. Maybe. Or there's something about like I mean what problem

you know, are you solving sort of for each episode? And I mean, as far as I can tell, like that's the only way I really understand anything is that, you know, I I get interested in something. At first I don't even have a problem, but there's just some sense of there's some contribution to make here. And gradually you home in and uh there's a problem. And then you I mean, funnily enough, I mean, spending time stuck is incredibly important. Um and and I I sort of

you know, I th that used to just be annoying. Now it seems like oh this is actually um uh uh maybe even the most important part of the whole process. Um but that h very hard oneness of it means that uh you know, I internalize it afterwards. I often find actually i if I you know, I've written sometimes ten thousand word essays in, you know, a couple of days and I've written them in, you know, Three months or six months.

Uh, I I feel like I d I d I didn't learn very much from the ones that that that only took a couple of days. Interesting. Uh whereas I you know, you know some of the ones that that that took three months. I'll be you know, fifteen years later I'll I'll I'll I'll still remember. Yeah, can can you describe outside of um w how you learn uh of the one that took

I mean three months. I mean by far the most you know, the yeah, the the the common things, there's always some creative artifact. Sometimes it's a class, uh uh uh yeah, sometimes it's engagement with a group of people who um, you know, there's some collective creative artifact that you're you're you're working on. together. I mean y you might not even be aware of it, but you you know you're acting as an input to their creative ends um in some way.

Um and sometimes it's just, you know, it's an essay or a book or or or whatever. Um yeah, it's one of the reasons why uh I you know often quite enjoy doing podcasts. I mean particularly I mean I you know, I I s I said yes to come here partially'cause I know y you ask unusually demanding questions.

Um and so it's sort of th that's an attempt to to to get this sort of perspective from a a different it's a different kind of a forcing function. Um so you're trying to pick sort of the most demanding creative context. Yeah, so for this interview I uh went through like

three lectures of the Suscon sensual activity book. Yeah, yeah. The problem is that there's almost no practice problems in it. And so I hired um a physicist friend who's gonna like I haven't done it yet, but it's like every lecture I want like a bunch of practice problems to go to them and I'm I'm planning on being Um appropriately humbled. How do how do you make it as jugular as possible, right? Like the higher you can raise the stakes, the better.

I mean the interview is in some sense high stakes, but also it doesn't necessarily test deep understanding. Yeah, but I I don't think the interview is that high stakes, right? You're not writing a book about special relativity. And you're not trying to write a book that replaces the current you know, w whatever the the existing standard textbook is. Like that that's a really high really high saying.

find particularly difficult and and um it's a it's a funny one. People will talk about g going deep on a subject. And it turns out, you know, different people have different ideas of what this means. Some people means they re read a couple of blog posts. Some people it means they read a book about it.

Some people it means they wrote a book about it. Um and and and and I think like it's sort of what you what what what your standard is, the sort of the standard you hold yourself to um determines a lot about you know your ability to to integrate knowledge in this way. I don't know what your experience has been, but I found that I'm getting

I I'm in some sense uh able to move much faster on some things uh to the help of AI, but I don't know if I'm like learning better. Yeah, yeah. And I think it's probably because The hardest thing, the thing that is most demanding, is so aversive. that you try to take any excuse you can to get out of it. Yep. And just having back and forth conversation that'll where you gloss over it's entertaining, but not necessarily anything else. Yeah. So it's such an easy way to get out of the thing. Yeah.

Um in fact it makes it easier because instead of doing some intermediate thinking you there's always a next question you can ask a chatbot. Yeah. And and and it's somewhat valuable. Like it's not I mean that's part of the seductiveness of course. Like like it's not actually useless. Um but um but yeah, it can sort of substitute for for actually doing the thing that that maybe you should be doing. Um It's interesting that. Yeah, to what extent should you be outsourcing that kind of stuff?

Uh and to what extent d you know, like like it's it's really i there's some sort of interesting judgment call about uh uh you know, y you actually there is a whole bunch of routine work that that you want done.

Um and in fact it's it's low value for you. So you may as well get uh if you can get a chat bot to do it, you may as well. So uh somebody interviewed um the pioneering computer scientist Alan Kay years ago and he was asked what he thought about um basically Linux and if I remember his answer correctly, he basically said, Look, you know

it doesn't have anything to do with computer science. It's just a great big ball of mud. Um there's a few interesting ideas in there which are which are worth understanding, but mostly you're all you're learning is stuff about Linux. Like like you're not actually learning anything which is transferable. So there's a like a very intri like that there's a certain kind of seductiveness uh to some things where you know it's sort of a Rib Goldberg machine you can just

sort of learn about all the bits and it feels kind of entertaining. Um, but if you step back and think about the question, you know, what am I actually doing here? um the it might not actually be meeting your objectives. Maybe you want to become a you know a a sysadmin and learning Linux is a great use of your time. There's uh no no harm in that at all. But if if your answer is if you if your objective is to understand the fundamentals of computing

uh it's much less much less clear that that's a good use of your time. I thought that was it's certainly an an answer I've I've thought a lot about where you you actually need to you know that for a certain type of mind there is a seductiveness in in just m just

learning systems and confusing that with with uh with understanding. Okay, I'll keep you updated on our discuss. Yeah, yeah. I I I I owe you a text within a month of um some revamped learning system. I'll be really curious if you I mean it it's also true, right? Like Tiny incremental improvements in this.

I mean they're just worth so much. I know, yeah. It it's sort of the main input into the podcast, you know. It's great that the bookshelves are fancy and I've got a Blackboard or whatever, but really like the thing that makes the podcast better is if I can improve the learning I do. So it's um Yes. Great notes are not. Thanks, Michael.