Sebastian Mallaby: The Infinity Machine

We're gonna be talking about this phenomenal book. Okay. by Sebastian Mallaby called The Infinity Machine Demas Hassabis Deep Mind and the Quest for Superintelligence. Uh it's an extraordinary book. Um and because of knowing some of the people in and the whole topic, it really hit me with how rich and informative and um just Page Turner and just a r a really riveting book. So

Uh, I know this is your sixth book. Sebastian, welcome. Sebastian is this is v the Paul Volker, Senior Fellow at the Council for F uh Foreign Relations, and he is a journalist and has been um, very prolific throughout his career at many different places, including the Financial Times, the Washington Post, The Economist, and others. And so welcome. It's great to have you. Great to be with you.

So we're going to get right into this book because the first thing I wanted to touch on was the title itself, The Infinity Machine, because a lot of people might not realize exactly what is meant by this. And uh I think uh if I try to piece it together because you really zoom in on some critical work that annotated uh and influenced Demonis. uh and Deep Mind. And it seemed to me there were two specific uh works that really had an impact. One was uh Claude Shannon,

in nineteen forty eight information theory that the fundamental unit of reality is information, the reality is is processing disinformation. And the other extraordinary book that won a Pulitzer Prize by Douglas Hofstad or um Godel Escherbach that um he basic he basically in this book, of course it was many things, but some of it was whatever human braids can do, computers should be able to do.

uh and dismissing anything magical about human consciousness, uh discovering patterns in a near infinity of data. So would you is that an apt way to to try to interpret the title of the book? Because obviously it's a very provocative one. Well, yeah, I mean, it's actually a bit more specific than that. And you're right that both Shannon and the book God of Lesher Bach were very important for Dennis.

But I think with the Infinity Machine title, what I'm getting at is that, you know, traditional computer systems were essentially based on uh deductive logic. uh and you know, is some way by kind of the Bertram Russ Russell project of trying to create a mathematical system.

uh that would encompass you know all possible truths that you could derive from a set of um basic um you know grand premises. And um deduction, uh it turns out is a very incomplete way of thinking about what intelligence does, what knowledge consists of. Um, because a lot of the time uh we are inducing um knowledge. We are not deducting it. So we're looking at lots of examples and we are inducing a conclusion from these examples. Now, the point about Gerd Lashabach was partly what you said.

which is that what uh what the human brain can do, machines will be able to do one day. But it was also that Goethel, mathematician in the title, was famous for his repudiation of Bertram Russell for showing the so called incompleteness theorem Uh which established that, you know, it's impossible to deduce everything. Um if you take the example of, you know, categorizing.

Yeah, w wha how do we understand why a carving knife, big, long and sharp, is in the same category as a butter knife, short, blunt, Um, and totally different looking. Well, they have the same function. But then there are other categories where you know you have a big dog and a small dog. Um they look very different, but they're still both dogs. Why? Because they mate together and they can be fertile. Um so how do you explain deductively

that um, you know, you categorize dogs in one way according to reproduction, you categorize uh nines according to function. I mean there's just lots of things you you you can't simply deduce. They're much more arbitrary than that. And when you um learn through induction, um you you need a lot of examples to induce accurately. Um and you know, if I look at ten New Yorkers, I will conclude

That all Americans drink coffee in the morning. Of course that's not true. If I looked at a bigger sample, I would understand it wasn't true. Yeah. So good induction requires an infinity of data. hence the title The Infinity Machine. An AI is really a system that can master induction. Yeah, that well, that's a great summary. I'm glad you clarified it. You even took it to another level. My uh trying to understand why you selected that as the title and uh quite apropos hearing your side of it. Now

Uh, this the Deep Mine company is pretty extraordinary when you go back'cause we're talking about two thousand ten. You know, we're they're talking about, you know, s almost sixteen years ago. Uh I mean Dennis will be Dennis will be fifty in July. So y he he's a young chap, thirty four back then, or thirty three, and he with uh, of course, um, Shane Leg and Mustafa they form this company and they have a charter to have um superintelligence achieved.

by twenty thirty at the latest, I guess. And what was also f striking is they had agents in the business plan. Now only recently have people aw a welcome to the idea that we have all these agents to execute autonomously. So they were so far ahead.

And what's really striking to me is that this was I mean, d this takes prescience to another level, I I I have to say. So Tell us about this formation of Deep Mind at twenty ten, because it at the time they must have been seen as, you know, kooky people, but it they it if you look back, they were they were so far ahead of everybody else.

Yeah. I mean that's totally right. In a way that's why I chose Demis as my central protagonist for a book about AI. You know, one could have chosen Sam Oldman or somebody else, but no, Demis really was so early that through his life

you can tell the story of the making of modern AI and he's just the perfect he he's fascinating himself, as perhaps we'll talk about that, but just the span and the early nature of his conviction because In twenty ten, as you say, you know, it was still the AI winter really. I mean, um, the image net breakthrough in terms of computer vision, um, and that Katz paper. This was late twenty twelve. So Demis is forming Deep Mind two years before that.

And he's been thinking about AI, you know, another fifteen years before that. Um so i it it is amazing. And and as it you're right, that the business plan, it it took me enormous amounts of you know, negotiation, charm and general patience. to extract the business plan from Demis. But finally he gave it to me. Uh and when you read it it's it is unbelievably pressant. I mean he really did say that extrapolating the way that computer power was growing

he expected um you know superhuman intelligence to arrive around twenty thirteen. Uh amazing, right? And and uh you you're right about agents too. Yeah.

Yeah, I w I was stunned. I just I mean, I knew he started this company, but I didn't know but the depth of the plan, you know, and so now let's get more into Demis because as I read this book I feel like what this this human was made for the mission in so many respects because Uh for one, he got into chess very early as a precursor to games and uh competition. And uh of course he was already doing that, sitting on books, uh, to be able to compete against older

uh competitors, but it brought out this fierce competition which was later characterized um by Jeff Hidden. He's the only person um perhaps more competitive than me and Uh, of course when he was fourteen, or was it twelve, when he he he lost a match, changed every oh, twelve, I think. And he and he said the only way I could know if I

push myself to the point just before death because his father said you gotta try your best and he gave it always a hundred percent, not ninety nine percent as Shane Legg, his co founder pointed out. So he had this d d this un um satiable drive, the ch the games, the fact that he then went on to games using uh GPUs, which really enabled deep learning. uh the fact that then he had products from the games when he was working at Bullfrog

so he could deal with products later. The fact that he was a computer scientist at Cambridge and then later on neuroscience, um, learning about the brain, the hippocampus specifically, Uh that he had this ultra charisma that he could uh c uh pull jetty mind. uh, that he learned from failure of from Elixir. The company started with um David Silver because of the fact that he was h far ahead of his time.

Um, and by the way, one other thing that wasn't in the book, which I learned from a conference when I was with Dennis in uh London uh November of twenty twenty-four, he was in a panel with four other uh Nobel laureates. And uh one of them was Jennifer Dowdna and um there were a couple others. Anyway, he said he wanted to be c he knew he wanted to be a Nobel laureate when he was a young child. And all the others looked at him like that.

You know, a and then and then you have him connected with Google later, which has not an infinite data set, but, you know, lots of data from Gmail and the internet and searches and Bye. This was a whole story built around this one person that had every piece covered to do this. I is that am I processing this wrong? No, I think that's right. I mean, actually Peter Thiel, uh, who wrote the first check into Deep Mind, um, in the series A round.

uh in twenty ten, said to me, you know, he often thinks of, you know, there's no such thing as sort of a general purpose entrepreneur. People generally have uh one company that's inside them screaming to get out. And so Demis, the mission for which he was born uh you know, was was building AGI because he had all those qualities you've just described and he was just the perfect person to do it.

Yeah, so that's what's so different about the other leaders in this field. Not only was he way ahead of them in time, but the the portfolio of experience, the personality. the special charisma, um, you know, all these features just

And then of course when he got forced to have to deal with products, I don't think that's something he would have picked'cause at heart he s he really seems like a scientist. And what also strikes me is, you know, he was so proud To publish papers in nature, have covers in nature. Um, which is the opposite of the other companies that are happy to put out their white papers and blogs and just post them on their website.

Uh, they don't really care. And so it's the difference between the scientists and the people that say, you know, we're so big we could just publish we don't need a journal. We don't need peer review. We could just publish our our own stuff. And that's like the mind of a scientist versus a different kind of leader in in in AI today.

There's also you know, part of the charm of the story is that Demius Susabis, of course, he born in North London, uh built his company in London in King's Cross, um, not a Silicon Valley person. Uh even though he got funding from Silicon Valley, both from Peter Thiel and from Elon Musk. and later from Google when he sold to Google. Um but, you know, one of the differences it seems to be in the style is that Demis cares a lot about narrative and communication. He does

care about his reputation and he he believes you have to communicate publicly about yourself in order to build um the space, the resources to go after AI. So he he's not innocent.

about the importance of public relations. But his style of public relations, as as you say, publish in a peer reviewed journal. That's what he thinks is good. Because then you can really press people. You can say, Hey, you can't question this. This is peer reviewed, you know Uh whereas you know, Sam Altman's idea is that he has a lot of followers on Twitter. Yeah. It's a kind of different style.

Yeah. He's how it's even more people questioning him after the New Yorker piece by Ronan Farrow that appeared. So I I think what we have now is we have this setup of a unique uh leader who has brought to with him these co founders with other talents, of course, and then is capable of bringing many other great minds to join him and basically uh follow his lead and

Uh, and he's on this mission. Um, w one of the most extraordinary things I had no idea, Sebastian, was in March twenty sixteen. The basically the day when uh Lee Seedall, the go champion of the world, was o overtaken by uh um the uh deep mind um algorithms, if you will. Uh the same day, if not within hours, he decides, Well, that's not enough. I'm starting Alpha Full. So Alpha Go, Alpha Full, the same you know, the turning point of the same day, is that is that about right?

It's unbelievable. You know, he the he's out there in Seoul with David Silver, his friend and and the reinforcement learning scientist who was behind AlphaGo. And um it's day four of this um tournament. And um by winning uh I guess it was three t the third three to one, um, they knew that they were gonna win the best of five. So that the the the tournament was over, effectively.

And as they walked down the street in Seoul, there was a camera team behind them that picked up on a microphone this entirely sort of, you know, it's an open mic moment. It wasn't intended to be for quotation or whatever. But but, you know, Demis was saying to Dave, his friend, look, we've done um AlphaGo, now we've got to move on to the next thing. And I think, you know, protein folding is what we should go after next. Now we can really do protein folding.

So, you know, as as Dave laconically said to me, um, you know, when demis Does an extraordinary achievement like solving go, he waits about thirty two seconds before announcing his next ambition. Uh It's so extraordinary. I mean, you know, the the mo the move thirty seven wasn't enough to have basically the machine come up with something that humans couldn't

And then, you know, right right there turned to what was gonna be the quest for Alphafold. And as you know, uh, I think for those who are listening, Alphafold went through stages where ultimately Alpha Fold two was what was recognized for predicting two hundred million proteins from uh you know amino acid sequences, the folding and the features of the protein which otherwise would take I mean I I'm at Script Research where we had so many crystallographers on our faculty.

And they might spend two years, three years, to crystallize a protein to be able to understand its folding. And here you could do it in seconds. through Alpha Fold two, now Alpha Fold three with the protein interactions. I mean, this is the singular biggest contribution of AI today. And that's, you know, something that Demis, John Jumper, and the whole big team, of course, can be proud of.

And what was really interesting, and now I know why, we're gonna get into this, um, when I was sitting next to John Jumper at the same conference uh in London And I said to him, Well, you couldn't have done Alpha Fold II without the transformer model, right? And he he didn't like that. It's an anti-transformer model thing. So before we get to that.

Um and I I'll qualify that because it wasn't quite as uh an antibody response. But there's as you discussed and we you introduced this pretty early, but I want to get back to it because it's really a central Part of the book. So previously there was a symbolic AI and it was rules based and it was rigid deduction. Okay. It only could go so far. And then came along the Toronto and subsequently Alberta tribes.

The Toronto tribe, deep learning. This is Jeffrey Hin, uh, with Ilya and Jan and Yoshio and others. And this is really about perception. It's about these neural networks that can classify And of course that's really where images took off with Faye Fe Lee's Image Net. And as you said, that really got going in two thousand twelve, not when Deep Mind was formed. And so that deep learning era, which we're still in in some respects, of course, that really took AI out of the winter into the hot zone.

Reinforcement learning. Which again, the the fact that that's the game story, that the games can learn by reward. Uh, and so this is perfect to start there and then just work your way up to Alpha Fold, right? Um, but this is really using agents and trial and error and um it's uh learning uh acting acting in an environment. So in the book you really make a big

um I think dichotomy between the deep learning and the reinforcement learning and how the best of both worlds when you confuse it. Can you tell us more about that? Sure. I mean, the first Deep Mind breakthrough was this system that could play Atari games uh like Breakout and Pong and and so forth. And this combined um both deep learning and reinforcement learning. And it was the first successful

combination of the two, uh in one system. And it was revolutionary because um, you know, as you indicated by using the word tribe, uh there hadn't been much communication between the reinforcement learning tribe, uh based mostly in Edmonton, um, in Canada, and then the deep learners who were you know, Toronto was one place but I guess there was Joshua Benjouri in Montreal and so forth.

Uh and and so th they really didn't communicate much, but there was one graduate student who had studied uh at a graduate level in both places, both Edmonton and in Toronto, and this was Vlad Nee. And Vlad Ny, a sort of Ukrainian Canadian, um, was persuaded to join Deep Mind early on.

And he brought with him this ambition to combine the two sides of his graduate studies, to put them into one system. And he teamed up with other people who had studied in Switzerland where this divide between the two schools, reinforcement learning and deep learning, was less uh pronounced. And so DeepMind resolved to make the two work together. And that's what unlocked these Atari game-playing agents. You needed the deep learning.

Partly just to give the system a perceptual tool to actually see the Atari uh board, the screen, understand the pixels on the screen. Uh but you needed the reinforcement learning to give the system the ability to kind of understand a game, to plan, to develop a strategy. Um and that's where the reinforcement learning came in. And Through combining the two, um, Deep Mind put itself in a league of its own.

Yeah, and and that of course was instrumental. That is, you could take the protein data bank, which had been worked on for many years, and y you know, the the brilliance of d using that, which we don't have many data sets like that. uh to basically um uh bring deep learning and reinforcement learning uh together. But going back to when I challenged John Jumper about well

could you know, for you to go into high gear with these two hundred million proteins, you you had to have had transformer models. And he really wasn't keen on that. And that gets me to the next thing, which is You know, you uh uh in the book, you know, through these chapters you're really starting to appreciate that Demis is, you know, s an extraordinary f uh force, uh a character. Now that that's those are not enough words. Uh

uh maybe not messianic, maybe not uh enders, but you know, in out there. He's in a very class of of his own as a b as a individual. But then you see the big myth. the fact that after all this tremendous momentum leading up uh eventually to the Nobel Prize in chemistry in twenty twenty four, uh that in twenty seventeen the the the group of seven at Google, the company he's part of

they come up with this transformer model and attention is all you need. And they basically set the stage for Ilya at OpenAI. To you know, he falls out of his chair or whatever he jumps out of his chair. Yeah. And so now Open AI is more. You know, they they have now the way forward. You're gonna they're gonna be, you know, GPT two, three, four, and on and on, and they're gonna just get um hyperscale.

With data, and they're gonna build the infinity machine. Uh I mean they didn't call it that, but all of a sudden, you know, they invented uh we're gonna get AGI. Well, they're like, uh what, seventeen years late here, um you know uh or seven at least seven years late, sorry. Uh after what uh Deep Mind had initiated. But

then what I just cannot fathom, although you bring up some possibilities in the book. You bring up, well, maybe because there was so much angst between Google and deep mind b you know, about being separated, or maybe it's because it was angst between Demis and Mustafa or or, you know, and the f about this pride that we developed, you know, the whole deep learning, reinforcement learning. We don't need anything else.

But there was uh basically a dissing of the importance of transformer architecture and generative AI. Can you help explain that? Because this was a catastrophic miss at the time. Mm mm mm. Well, um, you know, I talked to Temis a lot about this because I had the good fortune of pitching him on the idea of this project, um giving me hours and hours of his time. uh in November twenty twenty two. And of course on the last day of that month uh is when Chat GPT came out.

And so I was embedded with Deep Mind right when this whole topic of AI went from the fringe to the mainstream, uh and right when generative AI had its breakout moment. So in in our discussions and we would meet up in a p pub in North London, uh, near his house and there'd be a dusty staircase at the back that people didn't know about. Go up the staircase, there was this empty room we would have to ourselves and sit there for two hours.

and debate stuff and and so I had plenty of chances to ask him about why he had this big myth. And the most interesting part of the answer is essentially his neuroscience PhD had left him with this big conviction, um, that, you know, intelligence is first of all the result of different components of the brain. and different ways of coming at information and processing it. Uh it's not just one thing. Uh and also that in neuroscience this field action and perception.

uh which posits that um to really be intelligent you need to be embodied in the world. You need to have a physical model of the world, to experience gravity, to experience what it is to drop a glass and have it shatter on the ground. Um without those things you won't

really be grounded. Uh and this word grounded came a lot in up came up a lot in in my conversations with Demis. And so when he saw the early efforts at um OpenAI to apply transformer models to language model To to to a language model. You know, his his idea was sort of, you know, good luck with that. You know, you're not gonna get to full intelligence. just through language. Language is too limited. Not ground.

Not grounded, not grounded. And also he said to me at one point, you know, it's it's it's a question of how big you think human experience might be. Could you really um encompass all there is in human experience. just by downloading all the language on the internet? I didn't think so. Well, turns out he was wrong. You can get at least, I don't know, ninety percent of human experience by downloading everything on the internet. And language is more grounded than he realized in the sense that

First of all, it's written by humans who are grounded. Secondly, these systems get uh reinforcement learning from human feedback, so they get some grounding through the feedback after they've been trained. Um and so you get a very good approximation of a grounded intelligence. just by training on language data. And and that was the key mistake. You know, neuroscience had led him in the right direction early on with games. it it misled him when it came to language models.

Yeah, well of course he figured that out. Um and as I think everyone who's used AI now that Generative AI was gonna be another very, very big uh factor in the future as AI moves forward and Uh and then of course it isn't just the large language models and large reasoning models and the agentic and you know on and on. And um we'll see uh other models undoubtedly in the years ahead that uh move beyond what we have today. But

What really was interesting is this set up an extraordinary competition. You had um the Sam Altman and Greg Brockman and the one side and you had um you know, Anthropic because Dario and his sister had exited uh and of course uh many others. But, you know, principally you had Google and DeepMind and Google really had all these assets. And then you had these two startups that are now, you know, trying to have um initial public offerings in the hundreds of billions of crazy stuff, right?

Um, and here Google's kinda caught with their pants down that they're way behind in especially with this chat GPT moment, end of t as you said, end of twenty November twenty two. But amazingly enough it wasn't that long before Gemini three eventually evolved and caught up and even transcended. And now we have this basically arms race with hyperscaling and

These three entities are basically, you know, kind of in a way going after each other, but they have different strengths. And one of the questions I have for you that's not in the book, because it kind of evolves subsequently, is the coding, vibe coding story. Anthropic differentiated itself with uh Claude Code. And of course a open AI had codecs, but um

That's another thing I wonder. Is there another miss here? Because the coding you can't do the I I use Gemini three for a lot of things, right? My deep research, my notebook LM, nano banana. But It wouldn't be the place for coding. Is that another big miss or is that something that y you know, think eventually they're gonna uh put a lot of weight into?

Um look, I I'll come back to that in a second. I just want to double click on one thing you said about um the comeback story because I think if you'd asked any business school professor um to predict the success of a merger where you take Google Brain uh in Mountain View, um, and you smush it together with Deep Mind in London, and you do this in the middle of a in really intense race with your rival OpenAI, and you factor in that there's eight hours of time difference between

Mountain View and London and that the two laps didn't like each other because they'd been competing over computing resources for a long time. Yeah, I think The business school professor would have said, Hey, you know, mergers are always difficult. This one looks impossible, and certainly not in the time that you need to make it work.

to catch up with OpenAI in this race. And yet, as you just said, they did catch up. It took two and a half years uh from Chat GPT coming out, basically, for um for Google to have pretty much drawn even or got even gone slightly ahead.

So I think th this, you know, not only is this a frontier technology, but there's been some frontier um business management stories uh coming out as well. Um On the coding, I mean, I think it's too early to say that's really a big miss because I suspect that um it won't be long before Gemini is offering coding capability um on a par. My my feeling is that right now, uh all of these labs are racing, they overtake each other. If you'd ask

the question, which is the top model in November twenty twenty five, last year, um it would have been clearly Gemini. Now if you ask that question, it's pretty clearly Claude. Uh in three or six months from now, I assume it'll be somebody else. Yeah. Um I I I don't see the coding breakthrough as as that much of a key miss for Google. Right. I mean I think it's more for the people that are uh the the uh kind of open claw world that wanna have

you know, the agents that are doing efficiently, autonomously getting so much done and it isn't for the, you know, everyday person that's, you know, more using it as an in information resource. But we'll see how it evolves. It's really fantastic.

And the competition is really fierce. And that's another question I had for you because knowing Demis as as I do and you know, he he I of course, have the highest regard for him. I he's actually a hero to me, which is another reason why I love this book because I identify with him. I mean he's a scientist and he wants to do things the way science has done, not uh flashing the pan. But he's been put because he was made in charge of this whole effort.

as you said, the coalescence of brain and mountain view af uh resources of uh talent, human talent and computing resources with what is in London, he now is put in a position that takes him away from the research. He's now a product. He's now has to compete for not just benchmarks, but, you know, uh gotta be better than this model and that model. Like every week there's a it's it's completely nuts.

I wonder when I asked him if he's okay with that, I mean he he kinda said he was, but what do you think about that? Yeah, I mean you're right. He has these two different personas in in one in one person. And w you know, the first is the scientist. And I think, you know, he really is driven by that intellectual curiosity, the p burning desire to understand how the world works. And that's sort of his primary driver, but also the part of himself that he likes to think of as as the dominant one.

But then there's another side of him which is just super competitive going back to his youth, chess playing, uh, and you know, just kind of silly things he's competitive about, like table football or o or football, you know, that th that that table game. You know, he would say to me, Well, you know, you understand, Sebastian, when I was at Cambridge, I was the best player on the campus. I mean, nobody else at university was better than me at that game.

And I sort of didn't really care one way or the other, so I would kind of laugh and say, No, no, no, no, no. You have to understand I was the best. And I tell you, I watched the the the YouTube clips of the professional players in the American League and they didn't know the snake shot.

Where st where you you loop your arm under the lever and you you you kind of get extra oomph on it and uh whatever. But he is so competitive. Um that uh so I think You know, on this question of is he okay about being the chief executive of Google Deep Mind and being basically a a a corporate leader, um, you know, part of him loves it because he is competitive.

And part of him is a scientist and, you know, would openly talk to me about, well, you know, maybe I should take up a professorship and and and just think. Um Yeah. Probably he'll do more corporate leadership first and then the other thing afterwards. Yeah, in the book you mentioned he his uh affinity to go to a place like Princeton and just, you know, get into deep thinking mode.

I I have to say, Sebastian, I'm a little concerned because taking him out of his science mode, it could be this next big model that is, you know, the next chapter in the evolution of AI that we could be missing. Uh so we'll see how that plays out. But just extraordinary. Now, um before I go into my last um topic, I w I wanna just get back to this oligopoly of companies and players.

and the the the um the kind of killer instinct between them. So one thing I didn't know about was the ultimate double cross. When you had um Sam Altman, Elon Musk and Reed Hoffman when they were together to try to work on AI safety and then they just basically just went I mean That that whole story I had no idea, I don't know if it had been publicized previously, but that that just kind of typifies this.

Um i it's not just fierce competition, it's that i what w whatever it takes to take each other on, it seems like. Is that right? Yeah, I mean actually Sam Altman was not part of the meeting you're referring to, but it's true that Elon Musk and Reed Hoffman were, and this was a safety and ethics oversight board which was supposed to advise Google on the safe deployment of AI.

And they had their inaugural meeting, actually Elon Musk was asked to host it and that was a way of trying to bind him into um the kind of AI safety aspect of DeepMind's work. Uh and Elon had expressed a lot of concern about existential risk from AI, so that seemed uh fitting that he should be involved. Um and then, you know, he sat through the meeting.

sucked up all the presentations from the Deep Mind leadership and a few months later announced that he was founding OpenAI with Sam Altman and Reed Hoffman would be one of the funders. Um so yeah, it was It was pretty ruth ruthless. Um and and and Google's reaction to this was to say, you know what, we've had enough of these oversight committees where we're just gonna brief people who who will use the information against us as competitors.

Um but, you know, interestingly and to Demis' credit and also Mustafa Sulemas' co founder, you know, they really were serious about wanting safety oversight. And so then they embarked on this three year negotiation with Google to try to force Google to create a new safety oversight board.

And they threatened actually to spin out uh from Google and go independent again if they didn't get what they wanted. But that's a whole nother story. Demis wasn't so happy that I read about that because it was based on all kinds of internal leaks um from From Deep Mind. Um, but it is true. And um Markable story. And paradoxically th Reed Hoffman was one of the people that they were looking to to help fund it and all kinds of the cross talk about the these big players, all you know, just

The ruthlessness, competition, uh, you know, y y there's not that many examples that we can see like this, and obviously it's gonna continue because everything's at stake here. The dominance

o of AI. Now, that gets me to the last topic I wanted to get in with you, which which is the danger. The um the fact that We're threatening human sovereignty. We're we're have the w the world as we saw, the clash between anthropic and open AI about using AI uh in war, uh for surveillance. Uh no less all sorts of uh hur doomsday or even just short of that, horrendous scenarios that we could be seeing in You know, reasonable people would fear this.

And obviously you get into this it v various times in the book about the safety, the ethics, the need for uh pre uh readiness, uh and uh whatever, you know, not to have the loss of control. Now, as you got to in the book, there was times do we have a pause? Well, that doesn't seem very smart because what you have a pause for six months and you're back. How do we aim high? How do we go that route?

And especially when you have you know, it's not just the the oligopoly based in the US, you've got China, you've got other all sorts of other players here. Yeah. How are we gonna get this right? Yeah. I mean this is the very deep question I wrestled with all throughout the book, because Demis on the one hand, excuse me, um

He's he's a good person, as as you've been saying. He sincerely does want the systems to be safe. Um and he had various theories from the time he founded Deep Mine as to how he would. try to make them safe. One was to have that safety oversight committee. If that didn't work, he would negotiate with Google about having a new safety oversight committee. He had a set of principles on AI usage at one point.

Um and and and none of these things really worked because of the race dynamic. You could make one lab safe, um, but if there are four other labs that are not safe, you haven't really advanced the ball in terms of how safe society might be. And so, you know, this is the dilemma and this is something that Demis would talk to me a lot, you know, the race dynamic, that language.

came up almost in our first conversation, right after ChatGPG went viral. And, you know, he looked worried. He would say, you know, oh, they know this waste dynamics got worse. I don't know how we're going to control this. And it didn't change throughout the three years in which we would have those meetings upstairs in the pub. Um and I think the way you get out of this race dynamic, there's only really one thing, and that is that governments have to enforce

um restraint on all the labs at once. Because as I say, you have to have everybody observing the restraint, otherwise there's no point. Um and this means essentially you need both the US and China to do it together because China has a very big, excellent and uh independent ecosystem from the US. Uh it's able to distill models from American models, meaning that it can basically reverse engineer um any new model that comes out of the US within a few months.

Um and so whether we like it or not, China is a player. Um now I think if China and the US were able to agree on some sort of, you know, safety standards. then these could be enforced on all the other labs because basically if you're Mistral in France or Cohere in Canada or even if you're one of the labs in, you know, the Middle East or somewhere You probably want to train on American clouds, with American chips, with funding maybe from the US, maybe selling to US um users of your AI.

Um so so you have these touch points with the US, so you're gonna be forced to um respect US regulatory restraints. Uh and then China would take care of the other half of the ecosystem. And I think that's the vision. Um getting there with the current state of Sino American relations is very tough. Right.

But we just have to remember that in nineteen sixty two we had the deepest, darkest, scariest point of the Cold War with the Cuban Missile Crisis. Right. And then in nineteen sixty eight we had the nuclear nonproliferation treaty. So cold wars go in phases and you do get detente at some points, and I hope that happens with AI. I sure hope so because this is um you know, when you start thinking about what could go wrong.

and that we we uh the US there's basically no governance government kicking in, uh and w as you say, we've got the China um uh Arctis supremacy. Um so whatever wars are going on between these companies here, uh, is of course separate from the US versus China. Um competition, which also is equally fierce. Now, um, if anybody we have over five hundred folks on board still, and if anybody has questions you can put them in the message and we'll try to answer them.

Uh I can't do justice to this book in a forty some minute discussion because, you know, I v I view it as extraordinary. I mean, if there's a masterpiece about AI that's been written, this is it because it's not just about denim. Asados. It's not about deep mind. It's about the whole fifteen year plus, including the background, and how what is superintelligence? What are we aspiring for? What are the risks?

It is all encompassing. And it uses a protagonist, uh a quite a extraordinary one, to get there and a company which was hyperpression, but it isn't just about these people. It's about the whole now, some people say, Oh, well, they didn't give enough justice to Sam Altman or

you know, Ilya or or Jan Lacoon or whoever. But the point is is that you have to start somewhere and you can't really have um symmetry of all this why this reason I think this book is is so exceptional is because if if you want to get educated about AI and how it grew up Uh and how it was in the winter.

how I got through deep learning, reinforcement learning, generative AI, and all the ins and outs. I mean, th this is it. So I I think you got a tough act to follow for others who will write a book about AI, um, uh, Sebastian. Uh I haven't seen any messages. Is there anything that I've missed that you wanted to add on?

You know, one thing which I found interesting, and it gets to your bringing up of the safety question, is the way that, you know, going in I thought, okay, so um I'd like to draw the comparison between the Manhattan Project and some of these AI labs.

Robert Oppenheim obviously did an amazing scientific feat with the Manhattan Project, but he also regretted the consequences of his own work. And that sense of regret, you know, struck me as, you know, this sort of the scientists in society, the dilemma of

creating new technologies which could be dangerous. Right. Just just struck me as super interesting. But I was a bit nervous about bringing it up with Demis or with other people I was talking to because I thought, well maybe they don't want to be reminded that, you know, their hand is on the modern version of the nuclear material.

Um, yeah, that's a scary thing to have in your power. But it turns out you don't have to raise it with people in this field because they raise it for you. They're all thinking about it. Starting with, you know, Jeff Hinton who Essentially quotes Robert Oppenheimer when he says, you know, why am I doing this? It's for the sweetness of discovery. Right.

Um scientists can't resist the sweetness of discovery. Moving on to Sam Altman, who, you know, will volunteer to interview us that he's got the same birthday as Robert Oppenheimer, isn't that interesting? And then with Dennis with Demis, you know, one day I said to him

So Demis, let's go back to the moment in twenty ten when you'd finally raised the money for Deep Mine. You had your first office in London, it was in Russell Square, not far from University College London where you did your PhD. Now how did that feel? And I know from experience as a writer that normally when you say this to people you're asking them to reconstruct the emotion of fifteen years ago.

The answer you get is disappointing. They go, Yeah, it was cool or something like that. You know, they they've got no emotional recall of what it was like. But with Demis, you know, he's so articulate and fizzy that he said, Yes, it was amazing. So I was Yeah, we had this attic office, you see, it was low low ceilings, a little bit stuffy, so I would go out sometimes to get some some air and

And, you know, I'd come down the stairs,'cause of course there's no elevators, an old building. I'd go down the stairs, ding, ding, ding, ding, ding, ding. And then outside in front of me there would be the trees of Russell Square. And just to the right, Sebastian, you know what there is? There is the London Mathematical Society, of course. That's where Turin used to deliver his lectures about the origins o of computer science. And now we're completing that. So that was fantastic. But then

Beyond the London Mathematical Society's About, you know what there is there. They're on the corner, there is a pedestrian crossing where it goes black, white, black, white, black, white, black, white as you cross the road. And and you know who crossed the road in the nineteen thirties? It was the Hungarian nuclear physicist, uh Sailard, who had the idea for the nuclear chain reaction as he crossed that road, which then gave rise to the Manhattan Project, Sebastian. And and now

We are the modern version of the Manhattan Project. Isn't that amazing? So that just gives you a taste of what it's like to speak to Demis Susabis for more than thirty hours. I mean, it's just wild. Oh that parallel with with nuclear technology, which I find so immu immu not amusing, amazing. It is. And there are a lot of parallels and um your quote it's in the book of course from Jeff Hitton and the sweetness of discovery and what you just went through with Demos.

So we got a couple of questions. Uh one asked about Mustafa Sullivan's book, The Coming Wave. Is that a good book to uh how anchor in and I reviewed that book. Uh I thought Mustafa did a nice job of outlining the the pluses and minuses. It was published not quite a couple of years ago. I don't know if you have read it, uh, Sebastian. I have. Might have even been two or three years ago, but yes, I did read it.

Um and you know, I thought it was a very good attempt to lay out some thinking about both the upside and the downside of uh of AI. Um it's a tough one to do in the sense that You're trying to project forwards and develop policy ideas about a technology that is still changing all the time. We don't know where it's landing.

Um and so I thought perhaps, you know, almost the the essay that um Mustafa also wrote in Foreign Affairs was better because because it's more succinct and um You know, it's it is tough to write a policy book about policies relating to technologies that haven't been quite built yet.

I I'm completely uh concordant with your points there. The other question that's come in was Are the large language models of today you can also add the large reasoning models, um is that the path to artificial superintelligence or is something

else gonna have to happen. And I guess that also depends on how you define AGI because some people will say we're there now. Some people say we still have years to go. Um Wha do you see a gap, do you see something else that's missing here before we go to a level that will transcend where we are today?

Look, well my sense from from hanging around um all the scientists at Deep Mind and and in the rival labs and and talking to them for three years is that, you know, there may well be new innovations like the Transformer paper, which need to happen before we get to superhuman intelligence.

But these are going to build on top of the large language model architecture, um, the transformer architecture. I mean, that is just such a useful, flexible, um, you know, scalable But I I I you know I think it's it's uh there's a sort of thing. a minority critique of the industry which says, Oh, you know, industry is spending insane amounts on computing power.

and building all this electricity generation and inflicting burnouts on nearby cities and it's disgusting and really, you know, we don't need all that massive scaling. Um I I I think that's wrong for a couple of reasons. First is the point about what an infinity machine is. And the infinity machine is something that has nearly an infinity of data because you can't do good induction without lots of examples. So just fundamentally you need a lot of data and the more the better.

And then secondly, obviously the private labs don't love spending a hundred billion dollars on their R and D. If they could spend ten billion, they'd be very happy. So they have every incentive. To be as efficient as possible in how they build this technology out. They're spending all this money because they really think there's going to be a return, because they think it's necessary.

So I think that um there isn't an alternative path, but there might be an extension of the current path in order to get to some human human intelligence. Yeah. No, I think that's uh That's great because I think we have some features of it now where we've seen superhuman capabilities of certain tasks and obviously for some to conclude we're there it would have to be much broader than where we are today. Well this has been a fascinating discussion. Um this

Uh anybody interested in AI, this is the book you should be reading. Uh I I think it's gonna hold up. I don't think it's one that is gonna just because of new things happening in the field because you you get at it in a very um in in in such an insightful way that it's not time limited, uh, which is a it's very difficult because

you had to get this to the printer's month months ago and the field keeps coming up with new models, but that didn't change anything. And I don't think it's gonna that's what was I think particularly um uh i i surprising. I I did an interview with Al Gore yesterday uh at an AI conference here in actually in San Francisco and I he had known about your book but he didn't know it without And I I said to her, That's one of the things that's really remarkable is here it is

um in uh you know April twenty twenty six and you it doesn't matter that there's been some new models since we went to print because it's it's all there. So congratulations, Sebastian. Really a joy to have this discussion with you. It's been wonderful. Thank you so much, Eric. I've really enjoyed it. All right. Take care. Thank you. All right. 拜拜