I wouldn't be surprised if we had HGI-like systems within the next decade. It was pretty surprising to almost everyone, including the people who first worked on the Scaling hypothesis that how far it's gone. In a way, I look at the large models today and I think they're almost unreasonably effective for what they are. It's an empirical question whether that will hit an asymptote or a brick wall. I think no one knows.
What do you think about Superhuman Intelligence? Is it like, still controlled by a private company? As the Gemini are becoming more multimodal and we start ingesting audiovisual data as well as text data, I do think our systems are going to start to understand the physics of the real world better. The world's about to become very exciting, I think, in the next few years as we start getting used to the idea of what true multimodality means.
Okay, today it is a true honor to speak with Demis Hassabis, who is the CEO of DeepMind. Demis, welcome to Pat Yes. Thanks for having me. First question, given your neuroscience background, how do you think about intelligence? Specifically, do you think it's one higher level of general reasoning circuit, or do you think it's thousands of independent sub-skills in heuristics? Well, it's interesting because intelligence is so broad and what we use it for is so generally
applicable. I think that suggests that there must be some high level common things in common kind of algorithmic themes, I think, around how the brain processes the world around us. So of course, then there are specialized parts of the brain that do specific things, but I think there are probably some underlying principles that underpin all of that.
Yeah, how do you make sense of the fact that in these LLMs though, when you give them a lot of data in any specific domain, they tend to get asymmetrically better in that domain. Wouldn't we expect a general improvement across all the different areas? Well, I think you, first of all, I think you do actually sometimes get surprising
improvement in other domains when you improve in a specific domain. So, for example, when these large models sort of improve at coding, that can actually improve their general reasoning. So there is some evidence of some transfer, though, I think we would like a lot more
evidence of that. But also, you know, that's how the human brain learns, too, is if we experience and practice a lot of things like chess or writing, creative writing, whatever that is, we also tend to specialize and get better at that specific thing, even though we're using sort of general learning techniques and general learning systems in order to get good at that domain. Yeah. What's been the most surprising example of this kind of transfer for you?
Like, if we use the language and code or images and text, what's... Yeah, I think probably, I mean, I'm hoping we're going to see a lot more of this China transfer, but I think things like getting better at coding and math then generally improving your reasoning. That is how it works with us as human learners, but I think it's
interesting seeing that in these artificial systems. And can you see the sort of mechanistic way in which, let's say in the language and code example, there's like, I found the place in a neural network that's getting better with both the language and the code, or is it that too far down the... Yeah, well, I don't think our analysis techniques are quite sophisticated enough to
be able to hone in on that. I think that's actually one of the areas that a lot more research needs to be done on kind of mechanistic analysis of the representations that these systems build up. You know, I sometimes like to call it virtual brain analytics in a way. It's a bit like doing F.A.M.R.I. or a single cell recording from a real brain. What's the analogous sort of analysis techniques for these artificial minds? And there's a lot of great work going on on this sort of
stuff. People like Chris Ola, I really like his work and a lot of computational neuroscience techniques, I think, could be brought to bear on analyzing these current systems we're building. In fact, I try to encourage a lot of my computational neuroscience friends to start thinking in that direction and applying their know-how to the large models. Yeah, what do other AA researchers not understand about human intelligence that you have some sort of like insight on giving your
neuroscience background? I think neurosciences added a lot. If you look at the last sort of 10, 20 years that we've been at it at least, and I've been thinking about this for 30 plus years, I think in the earlier days of this sort of new wave of AI, I think neuroscience was providing a lot of interesting directional clues. So things like reinforcement learning, combining that with deep learning, some of our pioneering work we did there, things like
experience replay, even the notion of attention, which has become super important. A lot of those original sort of inspirations come from some understanding about how the brain works. Not the exact specifics, of course, ones in engineered system, the others in natural systems. So it's not so much about a one-to-one mapping of a specific algorithm. It's more kind of inspirational direction,
maybe some ideas for architecture or algorithmic ideas or representational ideas. Because the brains in existence prove that general intelligence is possible at all, I think the history of human endeavors has been that once you know something's possible, it's easier to push hard in that direction, because you know it's a question of effort then, and sort of a question of when, not if.
And that allows you to, you know, I think, make progress a lot more quickly. So I think neurosciences has had a lot of, has inspired a lot of the thinking, at least in a software behind where we are today. But as for, you know, going forwards, I think that there's still a lot of interesting things to be resolved around planning and how does the brain construct the right world models. You know, I studied, for example, how the brain does imagination, or you can think of it as mental simulation.
So how do we create, you know, very rich visual spatial simulations of the world in order for us to plan better? Yeah, actually, I'm curious how you think that will sort of interface with LLM. So obviously deep mind is a different year and has been for many years, you know, systems like Office Zero and so forth of having these agents who can like think through different steps to get to an end outcome. All right, will this just be, is a path for LLM's to have this sort of
tree search kind of thing on top of them? How do you think about this? I think that's a super promising direction in my opinion. So, you know, we've got to carry on improving the large models and we've got to carry on basically making the more and more accurate predictors of the world. So in effect, making the more and more reliable world models, that's clearly a necessary,
but I would say probably not sufficient component of an AGI system. And then on top of that, I would, you know, we're working on things like Office Zero, like planning mechanisms on top, that make use of that model in order to make concrete plans to achieve certain goals in the world. And perhaps sort of chain, you know, chain thought together or lines of reasoning together. And maybe you search to kind of explore massive spaces of possibility. I think that's kind of
missing from our current large models. How do you get past this sort of immense amount of compute that these approaches tend to require? So even the off-go system was, you know, a pretty expensive system because you had to do this sort of, like running an LLM on each node of the tree, how do you anticipate that again, more efficient? Well, I mean, one thing is, more more tends to, tends to, tends to, tends to help, you know, every, every year, of course,
more computation comes in. But we focus a lot on efficient, you know, sample efficient methods and, and we're using existing data, things like experience replay. And also just looking at more efficient ways, I mean, the better your world model is, the more efficient your search can be. So one example I always get with Alpha Zero, our system to play Go and chess and, you know, any game is that it's stronger than world champion level, human world champion level at all these
games. And it uses a lot less search than a brute force method, like deep blue, say to play chess, deep blue, one of these traditional stock fish or deep blue systems would maybe look at millions of possible moves for every decision it's going to make. Alpha Zero and Alpha Go made, you know, looked at around 10 tens of thousands of possible positions in order to make a decision about what
to move next. But a human grandmaster, a human world champion probably only looks at a few hundreds of moves, even the top ones, in order to make their very good decision about what to play next. So that suggests that obviously the brute force systems don't have any real model other than the heuristics about the game. Alpha Zero has quite a decent model, but the human top human players have a much richer, much more accurate model than of Go or chess. So that allows them to make,
you know, world class decisions on a very small amount of search. So I think there's still there's a sort of trade off there, like, you know, if you improve the models, then I think your search can be more efficient and therefore you can get further with your search. Yeah, I have two questions based on that. The first being with Alpha's Go, you had a very concrete wind condition of, you know, at the end of the day, I do, I went this game a go or not,
and you can reinforce on that. When you're just thinking of like an LLM putting out thought, what will do you think there will be this kind of ability to discriminate in the end, whether that was like a good thing to reward or not? Well, of course, that's why we, you know, we pioneered and deep mind sort of famous for using games as a proving ground partly because obviously it's
efficient to research in that domain. But the other reason is obviously it's, you know, extremely easy to specify a reward function, winning the game or improving the score, something that that sort of built into most games. So that is the, that is the one of the challenges of real-world systems is how does one define the right objective function, the right reward function, and the right goals, and specify them in a, you know, in a general way, but they're specific enough and actually
points the system in the right direction. And for real-world problems, that can be a lot harder, but actually if you think about it in even scientific problems, there are usually ways that you can specify the goal that you're after. And then when you think about human intelligence, who you're just saying, well, you know, the humans thinking about these such are just super sample efficient. I understand coming up with relativity, right? There's just like thousands of possible
permutations of the equations. Do you think it's also this sort of sense of like different heuristics of like, I'm going to try out this approach instead of this, or is it a totally different way of approaching, coming up with that solution, then what, you know, what AlphaGo does to plan the next move? Yeah, well, look, I think it's different because our brains are not built for doing Monte Carlo tree search, right? It's just not the way our organic brains
would work. So I think that in order to compensate for that, you know, people like Einstein have come up, you know, their brains have, using their intuition, and you know, we maybe come to what intuition is, but they use their sort of knowledge and their experience to build extremely, you know, in Einstein's case, extremely accurate models of physics, including these sort of
mental simulations. I think if you read about Einstein and how he came up with things, he used to visualize and sort of really kind of feel what these physical systems should be like, not just the mathematics of it, but have a really intuitive feel for what they would be like in reality. And that allowed him to think these, these, these sort of very outlandish thoughts at the time.
So I think that it's, it's the sophistication of the world models that we're building, which then, you know, if you imagine your world model can get you to a certain node in a tree that you're searching, and then you just do a little bit of search around that node, that leaf node, and that gets you to these original places. But obviously, if your model is, and your judgment on that model is, is very, very good, then you can pick which leaf nodes you should sort of expand
with search much more accurately. So therefore, overall, you do a lot less search. I mean, there's no way that, you know, any human could, could do a kind of brute force search over any, any kind of significant space. Yeah, yeah, yeah. A big sort of open question right now is whether RRL will allow these models to do the self-place synthetic data to get over the data bottle on that. It sounds
like you're optimistic about this. Yeah, I'm very optimistic about that. I mean, I think, well, first of all, there's still a lot more data, I think, that can be used, especially if one views like multi-modal and video and these kind of things. And obviously, you know, society's adding more data all the time. But I think to the internet and things like that, but I think that
there's a lot of scope for creating synthetic data. We're looking at different ways, partly through simulation and using very realistic games environments, for example, to generate realistic data, but also self-place. So that's where systems interact with each other or or converse with each other. And in the sense of, you know, work very well for us with Alpha Go and Alpha Zero, where we got the systems to play against each other and actually learn from
each other's mistakes and build up a knowledge base that way. And I think there are some good analogies for that. It's a little bit more complicated, but to build a general kind of world data. How do you get to the point where these models, the sort of synthetic data they're outputting and they're self-plated they're doing, is not just more of what they've already got in their data set, but is something they haven't seen before. Do you know what I mean to actually
improve the abilities? Yeah, so there, I think, there's a whole science needed and I think we're still in the nascent stage of this, of data curation and data analysis. So actually, analyzing the holes that you have in your data distribution. And this is important for things like fairness and bias and other stuff to remove that from the system is to is to try and really make sure that your data set is representative of the distribution you're trying to learn. And, you know, there are
many tricks there one can use like over waiting or replaying certain parts of the data. Or you could imagine if you identify some some gap in your data set, that's where you put your synthetic generation capabilities to work on. Yeah, so, you know, now a day people are paying attention to the RL stuff that deep-minded many years before. What are the sort of either early research directions or something that was done way back in the past, but people just haven't been
attention to that you think will be a big deal, right? Like there's a time where people weren't paying attention to scaling. What's the thing now where it's like totally underrated? Well, actually, I think that, you know, there's the history of this sort of last couple of decades has been
things coming in and out of fashion, right? And I do feel like a while ago, and you know, maybe five plus years ago when we were pioneering with AlphaGo and before that, DQN where it was the first system with, you know, that worked on Atari, but how first big system really more than 10
years ago now, that scaled up Q learning and reinforcement learning techniques to deal, you know, combine that with deep learning to create deep reinforcement learning and then use that to scale up to complete some, you know, master some pretty complex tasks like playing Atari games just from the pixels. And I do actually think a lot of those ideas need to come back in again. And as we talked about earlier, combine it with the new advances in large models and large multimodal
models, which is obviously very exciting as well. So I do think there's a lot of potential for combining some of those older ideas together with the newer ones. Is there any potential for something to come the AGI to eventually come from just a pure RL approach? Like the way we're talking about it sounds like there'll be the LLM will have formed a great prior and then this sort of research will go on top of that. Is it possibility of just
like completely out of the data? I think I certainly, you know, theoretically, I think there's no reason why you couldn't go full Alpha0 like on it. And there are some people here at Google DeepMind and in the RL community who work on that, right? And fully assuming no priors, no data and just build all knowledge from scratch. And I think that's valuable because of course, you could, you know, those those ideas and those algorithms should also work when you have some
knowledge too. But having said that, I think by far probably my betting would be the quickest way to get to AGI in the most likely plausible way is to use all the knowledge that's existing in the world right now on things like the web and that we've collected and we have these scalable algorithms like like Transformers that are capable of ingesting all of that information. And I don't see why you wouldn't start with a model as a kind of prior or to build on and to make
predictions that helps bootstrap your learning. I just think it doesn't make sense not to make use of that. So my my my betting would be is that, you know, the the final AGI system will have these large multi models models as part of their the overall solution, but probably won't be enough
on their own. You will need this additional planning search on top. Okay. This sounds like the answer to the question about to ask, which is what what is somebody who's been in this field for a long time and seen different trends come and go, what do you think that the strong version of the scaling hypothesis gets right and what does it get wrong? The just the idea that you just throw enough compute at a widen of distribution of data and you get intelligence.
Yeah. Look, my view is this is kind of an empirical question right now. So I think it was pretty surprising to almost everyone including the people who first worked on the scaling hypothesis that how far it's gone. In a way, I mean, I sort of look at the large models today and I think they're almost unreasonably effective for what they are. You know, I think it's pretty surprising some of the properties that emerge things like, you know, it's clearly my opinion got some form of
concepts and abstractions and some things like that. And I think if we were talking five plus years ago, I would have said to you, maybe we need an additional algorithmic breakthrough in order to do that. Like, you know, maybe more like the brain works. And I think that's still true if we want to explicit abstract concepts, neat concepts, but it seems that these systems can implicitly learn that. Another really interesting, I think, unexpected thing was that these systems have some
sort of grounding. You know, even though they don't experience the world multimodal, or at least until more recently, we have the multimodal models. And that's surprising that the amount of information that can be and models that can be built up just from language. And I think that I have high some hypotheses about why that is. I think we get some grounding through the RLHF feedback systems because obviously the human rate is by definition grounded people were grounded
right in reality. So our feedbacks also grounded. So perhaps there's some grounding coming in through there. And also maybe language contains more grounding, you know, if you're able to ingest all of it, then we then we perhaps thought or linguists perhaps thought before. So especially some very interesting philosophical questions. I think we haven't we people haven't even really scratched the surface off yet. That looking at the advances that have been made, you know,
it's quite interesting to think about where it's going to go next. But in terms of your question of like, you know, large models, I think we've got to push scaling as hard as we can. And that's what we're doing here. And, you know, it's an empirical question whether that will hit an asymptote or a brick wall. And there are, you know, different people that argue about that. But actually, I think we should just test it. I think no one knows. And but in the meantime, we should also
double down on innovation and invention. And this is something that the Google research and deep mind and Google Brain have have, you know, we've pioneered many, many things over the last decade. That's something that's abred and butter. And, you know, you can think of half our effort as to do a scaling and half our efforts to do with inventing the next architectures, the next algorithms that will be needed knowing that you've got this scaled larger and larger model coming along the lines.
So I, my betting right now, but it's a loose betting is that you would need both. But I think, you know, it's, you've got to push both of them as hard as possible and we're in a lucky position that we can do that. Yeah, I want to ask more about the grounding. So you can imagine two things that might change, which would make the grounding more difficult. One is that these models gets from harder. They're going to be able to operate in domains where we just can generate
enough human labels just because we're not smart enough, right? So if it does like a million line pull request, you know, how do we tell it like this is this is within the constraints of our morality and the end goal we wanted and this isn't. And the other is it sounds like you're saying more of the compute of so far we've been doing you an extra contradiction and in some sense it's a guardrail
because you're, you have to talk as a human would talk and think as a human would think. Now if additional compute is going to come in the form of reinforcement learning where just like get to the end objective, we can't really trace how you got there. When you combine those two, how worried are you that the sort of grounding goes away? Well, look, I think if the grounding, you know, if it's
not properly grounded, the system won't be able to achieve those goals properly, right? I think so I think in a sense you sort of have to have the grounding or at least some of it in order for a system to actually achieve goals in the real world. I do actually think that as these systems and things like Gemini are becoming more multimodal and we start ingesting things like video and and and and you know audiovisual data as well as text data and then you know the system starts
correlating those things together. I do I think that is a form of of proper grounding actually. So I do think our systems are going to start to understand, you know, the physics of the real world better and then one could imagine the active version of that is being in a very realistic simulation or game environment where you're starting to learn about what your actions do in the world and and how that affects the world itself, the world stay itself, but also what next learning
episode you're getting. So, you know, these RL agents we've always been working on and pioneered like Alpha Zero and Alpha Go, they actually affect their active learners. What they decide to do next affects what the next learning piece of data or experience they're going to get. So there's this very interesting sort of feedback loop and of course if we ever want to be good at things like robotics, we're going to have to understand how to act in the real world.
Yeah, so there's a grounding in terms of will the capabilities be able to proceed or will they be like enough in touch with the reality of you be able to like do the things we want and there's another sense of grounding of we've gotten lucky in the sense that since they're trained on human
thought they like maybe think like a human to what extent does that stay true when more of the compute for trading comes from just did you get the right outcome and not guard real by like are you like proceeding on the next token as a human would maybe the broader question not like post to you is and this is what I actually as well what would it take to a line of system that's smarter than the human maybe things in alien concepts and you can't like really monitor the million line pull
requests because it's you can't really understand the whole thing. Yeah, I mean, Shane and I and many others here we've had that forefront of our minds for since before we started deep mind and um because we planned for success crazy, you know, 2010. No one was thinking about AI let alone age AI, but we already knew that if we could make progress with these systems and these ideas
it you know that the technology would be created being unbelievably transformative. So we were already were thinking you know 20 years ago about well what how you know what would the consequences of that be both positive and negative of course the positive direction is amazing science things
like alpha fold incredible breakthroughs in health and science and maths and discovery scientific discovery, but then also we got to make sure these systems are sort of understandable and controllable and I think there's sort of several you know this would be a whole sort of discussion in itself,
but there are many many ideas that people have from much more stringent evil systems. I think we don't have enough at evaluations and benchmarks for things like can the system deceive you can it x-ray zone code sort of undesirable behaviors and then there's uh uh uh you know ideas of actually using AI maybe narrow aIs so not general learning ones but systems that are specialised for a domain to help us as the the human scientists analyze and summarize what the more general system
is doing right so kind of narrow AI tools. I think that there's a lot of promise in creating hard and sandboxes or simulations so that that are hard and with cyber security arrangements around the simulation both for to keep the AI in but also as cyber security to keep hackers out and
then you could experiment a lot more freely within that sandbox domain and I think a lot of these ideas are and there's many many others including the analysis stuff we talked about earlier where can we analyse and understand what the concepts are that this system is building what the representations are so maybe they're not so alien to us and we can actually keep track of
uh the kind of knowledge that is building. Yeah yeah um stepping back up fit I'm curious what your timelines are so Shane said he's like I think mode of outcome is 2028 I think those movies median yeah uh what what is yours yeah well I you know I don't have prescribed
kind of specific numbers to it because I think there's so many unknowns and uncertainties and and and you know human ingenuity and endeavor comes up with surprises all the time so that could meaningfully move the timelines but I will say that when we started deep mind back
in 2010 you know we thought of it as a 20 year project and and actually I think we're on track which is kind of amazing for 20 year projects because usually they're always 20 years away right so that's the joke about you know whatever it is to come to AI you know take your pick and um but I think we you know I think we're on track so I wouldn't be surprised if we had uh HGI like systems
it within the next decade. And do you buy the model that once you have an HGI you can have you have a system that basically speeds up further AI research maybe not like an overnight sense but you know over the course of months and years you have much faster progress than you would have
other rights. I think that's potentially possible um I think it partly depends what we decide we as society decide to use the first age you know nason agi systems or even proto agi systems for so uh you know even the current LLMs uh seem to be pretty good at coding so uh and you know we
have systems like alpha code we also got the they're improving systems so one could imagine uh combining these ideas together and and and and making them a lot better and then I could imagine these systems being quite good at designing and helping us build future versions of themselves
but we also have to think about the safety implications of that of course. Yeah I'm curious what you think about that so I mean I'm not saying this is happening this year or anything but eventually you'll be developing a model where during the process of developing you think you
know there's some chance that once it's fully developed it'll be capable of like an intelligence explosion like dynamic um what would have to be true of that model at that point where you're like I you know I've seen these specific evels I've like I've like understand it's internal thinking enough and like this future thinking that I'm comfortable continuing development of the system.
Well look we need um we need a lot more understanding of the systems than we do today before I would be even confident of even explaining to you what we would need to tick box there so I think actually what we've got to do in the next few years and the time we have before those systems start arriving is is come up with the right evaluations and metrics and maybe ideally formal proofs but you know it's going to be hard for these types of systems but at least empirical uh bounds
around what these systems can do and that's why I think about things like deception and as being quite root no traits that you don't want because if if you're confident that your system is is is is tell is is sort of exposing what it actually thinks then you could potentially that opens up possibilities of using the system itself to explain aspects of itself to you.
The way I think about that actually is like um if I was to play a game of chess against Gary Kasparov right which which I played in the past or Matt and Carlson you know the amazing chess players with a graceful time you I wouldn't be able to come up with a move that they could but but they
could explain to me um why they came up with that move and I could understand it uh uh post-hoc right and and that's the sort of thing one could imagine uh uh uh uh uh uh uh uh uh uh um capabilities that we could make use of these systems is for them to explain it uh it to us and even maybe the proofs behind why they're thinking something certainly in a
mathematical uh any mathematical problem. Got it um do you have a sense of what the the converse answer would be so what would have to be true where tomorrow morning you're like oh man I anticipate this I you see some specific observation tomorrow morning where like we
we got a stop Gemini 2 training like what would specifically imagine that like um and this is where uh you know things like the sandbox simulations I would hope we're experimenting in a in a safe secure environment and then you know uh something happens in it where um very unexpected happens
a new unexpected capability or something that we didn't want you know explicitly told the system we didn't want that it did but then lied about you know these are the kinds of things where one would want to then dig in carefully um you know net with the systems that are around today which are not dangerous in my opinion today but in a few years they might be have have potential um and then you would sort of ideally kind of pause and then really get to the bottom of um uh why it
was doing those things before one continued. Yeah going back to Gemini I'm curious uh what the bottlenecks were in the development um like why not make it immediately one order magnitude bigger uh uh if less scaling works. Well look first of all there are practical limits how much compute yeah. Then can you actually fit in one data center and actually you know you're you're bumping up against very interesting um uh uh uh uh just you know distribute a computing kind of
challenges right. Unfortunately we have some of the best people in the world on on those challenges and and you know cross data center training all these kinds of things very interesting challenges hardware challenges and we have our TPUs and so on that we're building and designing all the time
uh as well as using group use and so um there's all of that and then you also have to the scale or as you know they don't they don't just work by magic you sort of you still need to scale up the high parameters and various innovations are going in all the time with each new scale it's
not just about repeating the same recipe at each new scale you have to adjust the recipe and uh and that's a bit of an art form in a way and you have to sort of almost get new data points if you try and extend your predictions and extrapolate them say several orders of magnitude out sometimes
they don't hold anymore right because um new capabilities there can be step functions in in terms of new capabilities and and and and some things just some things hold and other things don't so often you you do need those intermediate data points actually to to correct uh uh some of your
hyper parameter optimization and other things so that the scale or continue to be true so um so there's sort of various practical limitations are on to onto that um so you know kind of one order of magnitude is about probably the maximum that you want to you want to carry on uh you want
to sort of do between each uh each era oh that's so fascinating uh you know in the GPT for technical report they say that they were able to predict the the training loss um you know it tens of thousands of times last compute then GPT-4 they could see the curve but the point
you're making is that the actual capabilities that loss implies uh we may not be so clear yeah the downstream capability sometimes don't follow from the you can often predict that the call metrics like training loss or or something like that but then um it doesn't actually translate
into mml you or or some other actual uh capability that you care about it's they're not they're not necessarily linear all the time yeah so there's sort of non-linear effects there well was the biggest surprise to you during the development of a Gemini so something like this is happening um well
I mean that I wouldn't say there was one big surprise but it's it was very interesting you know trying to train things at that at that size and and and learning about um uh all sorts of things from organization or how to babysit such a system and and to track it and and I think
things like getting a better uh understanding of of the metrics you're optimizing versus the the final capabilities that you want um I would say that's still not a perfectly understood mapping but but it's an interesting one that we're getting better and better at yeah yeah
there's a perception that maybe other labs are more compute efficient uh than uh deep mind has been with Gemini I don't know what you make of that for some uh I don't think that's the case I mean that you know it's uh uh I think that that actually Gemini one used roughly the
same amount of compute maybe slightly more than than what was roomed for GPT4 I don't know exactly what was was used so um I think it was in the same ballpark um I think we're very efficient with our compute and we use our compute for many things one is not just the scaling but going back
earlier to these more innovation and ideas you've got to you know it's only useful a new innovation a new invention if it also can scale so so in a way um you also need quite a lot of compute to do new invention uh because you've got to test many things at least some reasonable scale and make
sure that they work at that scale and also some of new ideas may not work at a toy scale but do work at a larger scale and in fact those are the more valuable ones so you actually if you think about that exploration process you need quite a lot of compute to be able to do that um I mean the good
news is is I think you know we we're pretty lucky at Google that we I think this year certainly we're going to have the most compute by far of of any sort of research lab and you know we hope to make very efficient and good use of that in terms of both scaling uh and the capability of our
systems and also new inventions yeah well it's been the biggest surprise to you uh if you go back to yourself in 2010 when you were starting to get mind in terms of what EA progress is looked like did you anticipate back then that it would in some large sense amount to spend as you know dumping billions of dollars into these models or did you have a different sense of what it would look like?
We thought that and actually you know if you I know you've interviewed my my colleague Shane and and he he always thought that and in terms of like um compute curves and and then maybe comparing roughly to like the brain and how many neurons and synapses there are very loosely but we're
actually interestingly in that kind of regime now roughly in the right order of magnitude of you know number of synapses in the brain and and and and the sort of compute that we have but I think more fundamentally you know we we always thought that um we bet on generality and learning right
so those were always at the core of the any technique we would use that's why we triangulated on reinforcement learning and search and and and and deep learning right as three types of algorithms that that would scale and um and and would be very general and and not require a lot of handcrafted
human priors which we thought was the sort of failure mode really of of the efforts to build AI in the 90s right places like MIT where where there were very you know logic-based systems expert systems you know masters of hand coded handcrafted human information going into that turned out to be
wrong or or too rigid so we wanted to move away from them I think we spotted that trend early and uh became you know and obviously we we use games as our proving ground and we did very well with that and I think all of that was very successful and I think it's maybe inspired others uh to
you know things are out for go I think was a big moment for inspiring many others to think oh actually these systems are ready to scale and then of course with the advent of transformers invented by our colleagues at Google you know research and brain that was the then you know
the the type of deep learning that allowed us to ingest masters of amounts of information and that of course is really turbocharged where we are today so I think that's all part of the same lineage um you know we couldn't have predicted every twist in turn now but I think the general direction
we were going in um was the right one yeah and in fact it's like fastening because actually if you like read your old papers or shanesville papers uh shanesdesis I think in 2009 he said like well you know the way we would test for AI is if you can you come press Wikipedia and it's like
literally the last function of our alarms or like your own paper in like 2016 before transformers to what we used to like uh you were comparing neuroscience and AI and he said attention is what we needed and exactly yeah exactly so we had these things caught out and actually we had some early
attention papers but they weren't as elegant as transformers in the end like new or true machines and things like this yeah uh and then transformers was the was the nicer and more general architecture of that yeah yeah yeah um when you when you extrapolate all this out forward and you think about
she's a procurement intelligence or is um like what does that landscape look like to you is it is it like still controlled by a private company like watch with the governance of that look like uh concretely yeah look I I would love um you know I think that this has to be uh uh uh this is so consequential this technology I think it's much bigger than any one company or or or even industry in general I think it has to be a big collaboration with many stakeholders from civil society, academia,
government and the good news is I think with the popularity of the recent chatbot systems and so on I think that has woken up uh many of these other parts of society that this is coming and what it will be like to interact with these systems and that's great so it's opened up lots of doors for
very good conversations I mean example of that was the safety summit at in the UK hosted a few months ago which other was a big success to start getting this international dialogue going and and and you know I think the whole of society needs to be involved in deciding what do we want
to deploy these models for how do we want to use them what do we not want to use them for you know I think we've got to try and get some international consensus around that uh and then also making sure that the benefits of these systems uh benefit everyone you know for the good of everyone
in society in general and that's why I push so hard things like AI for science and and I hope that you know with things like Aspin AI, it's a morpheic we're going to start curing diseases you know terrible diseases with AI and accelerate drug discovery amazing things climate change and other
things I think big challenges that face us uh and face humanity um massive challenges actually which I'm optimistic we can solve uh because we've got this incredibly powerful tool coming along down the line of AI uh that we can apply and I think help us and uh solve many of these problems so you know ideally we would have a big uh uh consensus around that and and and a big discussion
you know sort of almost like the UN level if possible. Well you know well my interesting thing is if you look at these systems they you chat with them and they're immensely powerful and intelligent but it's interesting to the extent of which they haven't like automated large-six times of the economy yet um whereas five years ago I showed you uh Gemini you'd be like wow this is like you know totally coming for a lot of things so how are you account for that like what's going on
where it hasn't had the had the broader impact. Yeah I think it's we're still I think not just we're still at the beginning of of of this new era yeah and I think that for these systems I think there are some interesting use cases you know um you know where you can use things to some you
know these these these chatbot systems to summarize stuff for you and and maybe do some simple writing and uh maybe more kind of boilerplate type writing but that's only a small part of what you know we we all do every day so I think for more general use cases um I think we need still need new
capabilities uh things like planning and search but also maybe things like personalization and uh memory episodic memory so not just long context windows but actually remembering what I what we spoke about a hundred conversations ago um and I think once those start coming in I mean I'm
really looking forward to things like recommendation systems that that helped me find better more riching material whether that's books or films or music and so on you know I would use that type of system every day so I think we're just scratching the surface of uh uh what these AI say assistance
could actually do uh for us in our general everyday lives and also in our work contexts as well I think they're not reliable yet enough to do things like science with them but I think one day you know once we fix factuality and grounding and other things um I think they could end up
becoming like you know the world's best research assistant for for you as a as a scientist or as a as a as a as a clinician hmm I want to ask about memory by the way um you had this fascinating paper in 2007 where you uh talk about the links between memory and imagination and how they
in some sense are very similar um uh people often claim that these models are just memorizing how do you think about that claim that people make um is is memorization all you need because is in some some deep sense that's compression or but you know what's your intuition here yeah I mean
sort of at the limit one one maybe could try and memorize everything but it wouldn't generalize out of out of your distribution and I think these systems are clearly I think the early the early um criticisms of these early systems uh were that they were just regurgitating and memorizing
but I think clearly the new era the Gemini GPT-4 type era they are definitely generalizing to new constructs um so but actually what you know in my thesis and that paper particularly uh that started that era of imagination in neuroscience was showing that you know first of all memory
certainly at least human memory is a reconstructive process it's not a videotape right we sort of put it together back from components that seems familiar to us that the ensemble and that's what made me think that imagination might be the same thing except in this case we're using the same
semantic components but now you're putting it together into a way that your brain thinks is novel right for a particular purpose like planning and um and so I do think that uh that kind of idea is still probably missing from our current systems this sort of pulling together different um parts of
your world model to simulate something new that then helps with your planning uh which is what I would call imagination yeah for sure so yeah now you guys have the best models in the world um you know with the Gemini models uh do you have do you have uh do you plan on putting out some sort of
framework like the other two major AI labs have of you know once we see these specific capabilities unless we have these specific safeguards we're not going to continue development or we're not going to ship the product out uh yes we we have actually we I mean we have already lots of internal
checks and balances but we're going to start publishing actually you know sort of watch this basis we're working on a whole bunch of um blog posts and technical papers that uh we'll be putting out in the next few months that um you know along the similar lines of things like
responsible scaling laws and so on we have those uh implicitly internally and various uh safety councils and so on people like Shane Chair and so on um but but uh it's time for us to talk about that more publicly I think so we'll be doing that throughout the course of the year uh that's good
here um and another thing I'm curious about is um so it's not only the risk of like uh you know the the deployed model being something that people can use to do bad things but also uh rogue actors that foreign agents so forth being able to steal the weights and then fine tune them to do
crazy things um uh how do you think about securing the weights to make sure something like this doesn't happen uh making sure a very like key group of people have access to them and so forth yeah it's interesting so first of all there's sort of two parts of this one is security one is
open source maybe we can discuss but the security I think is super key like mate just as sort of um normal cybersecurity type things and I think we're lucky at Google DeepMind we're kind of behind Google's firewall and and cloud protection which is you know I think best at you know besting class
in the world corporately so we already have that protection and then behind that we have specific deep-mind uh uh protections within our co-base so it's sort of a double layer of protection so I feel pretty good about that that that's I mean we you know you can never be complacent on that but
I feel it's it's it's already sort of best in the world in terms of cyber uh defenses um but we got to carry on improving that and again things like the hard and sandboxes could be a way of doing that uh as well and and maybe even there are um you know uh specifically secure data centers or
hardware solutions to this too that we're thinking about I think that maybe in the next three four five years we would also want um air gaps and various other things that are known in the security community so I think that's key and I think all frontier labs should be doing that because otherwise
you know nation states and other things rogue rogue nation you know states and other other dangerous actors um that that that there would be obviously a lot of incentive for them to to steal things like the weights um and then you know of course open sources and other interesting question
which is we're huge proponents of open source and open science I mean almost every you know with probably thousands of papers and and things like alpha fold and transformers of course and alpha goal of these things we put out there into the world uh published and open source many of them uh
graph cosmol most recently our weather prediction system but when it comes to uh uh you know the core technology the foundational technology and very general purpose I think the question I would have is if you you know uh uh first sort of open source proponents is that how does once uh uh a stop bad actors um individuals or rogues you know up to rogue states um taking those same open source systems and repurposing them uh because they're general purpose for harmful ends right so we have
to answer that question yeah yeah and and I haven't heard a compelling I mean I don't know what the answer is to that but I haven't heard a compelling clear answer to that from uh uh uh proponents of just sort of open source and everything yeah so I think there has to be some balance there but um
you know obviously it's a complex question of off to what that is yeah yeah I feel like tech doesn't get the credit it deserves for like funding you know hundreds of billions of dollars with the R&D yeah um and you know obviously I have deep bind with systems like alpha fold and so on um
but when we talk about securing uh the weights uh you know as we said like maybe right now it's not something that like it's going to cause the end of the world or anything but as these systems get better and better the worry that uh yes um a foreign agent or something gets access to them
presumably right now there's like dozens to hundreds of researchers who have access to the weights how do you uh what's that plan for like getting into like the situation or getting the weights in the situation rooms if you're like if you need to access to them you it's like you know some extremely
strenuous process you nobody nobody individual can really take them out yeah yeah I mean one has to balance that with with with allowing for collaboration and speed of progress actually another interesting thing is you of course you want uh uh uh you know brilliant independent
researchers from academia or or things like the UKI say FD Institute and US one um to be able to uh uh uh uh kind of red team these systems so so one has to expose them to a certain extent um although that's not necessarily the weights um and then you know we have a lot of processes in place
about um making sure that um you know only if you need them that that you have access to you know those people who need access have access um and right now I think we're still in the early days of those kinds of systems being at risk and as that as these systems become more powerful and more
general and more capable um I think one has to look at the access question um uh some of these other labs of specialized in different things uh relative to safety like anthropic present with interpretability and um um do you have some sense of where uh you guys might have an edge as so that
you know now that you have the frontier model you're going to scale up safety where you guys are going to be able to put out the best frontier research on safety yeah I think we know well we helped pioneer RLHF and other things like that which can also be obviously useful performance but
also for safety um I think that um you know a lot of the self-play ideas and these kinds of things could also be used potentially to to auto test uh a lot of the the the boundary conditions that you have with the new systems I mean part of the issue is that um with these sort of very
general systems uh there's so much surface area to to cover like about how these systems behave so I think we are going to need some automated testing and and again with things like simulations and games and very realistic environments uh virtual environments I think we have a long history
in in that and and using those kinds of systems and making use of them for for for building AI algorithms so I think we can leverage all of that uh history um and then you know around at Google we're very lucky we have some of the world's best cybersecurity experts hardware designers so I
think we can bring that to Baron in you know for security and safety as well great great um let's talk about Gemini yeah um so you know now you know you guys have the best model in the world um so uh i'm curious we know the default way to interact with these systems has been through chat uh
so far now that we have multi-model and all these new capabilities how do you anticipate that changing or do you think that'll still be the case yeah I think we're just at the beginning of actually understanding what a full multi-model model system uh how exciting that might be to interact
with and and uh it'll be quite different to I think what we're used to today with the chat bots I think um uh the next versions of this over in the next year 18 months you know maybe we'll have some contextual understanding around the environment around you through a camera or whatever it is a
phone um you know I could imagine that as an ex awesome glasses at the next step um and then I think that that we'll start becoming more fluid in understanding oh let's let's let's sample from a video let's let's use voice um um maybe even eventually things like touch and you know if
you think about robotics and other things uh you know sensors other types of sensors so I think uh the world's about to become very exciting I think in the next few years as we start getting used to the idea what true multi-modality means um uh you were under the robotics subject
uh Elias said when he noticed on the podcast that the reason opening I gave up on robotics was because they didn't have enough data in that domain at least at the time they were pursuing it um well I mean you guys have put out different things like robot transformer and other things
how what do you think that's still about unlike for robotics progress or will we see in up progress in the world of atoms as well as the world of it yeah well we're very excited about our progress with things like gato and and and and and our t2 robotic transformer and uh and we
actually think um so we've always liked robotics and we've we've had you know amazing research and now we still have that going now because we like the fact that it's a data poor regime because that pushes us on so i'm very interesting research directions and we think that it'd be useful
anyway like sampling efficiency and data efficiency in general and transfer learning learning from simulation transferring that to reality all of these very you know simtoreal all of these very interesting uh actually general challenges that we would like to solve um so the control problem
so um we've always pushed hard on that and actually I think uh uh uh uh so so Elias right that that is more challenging because of the data problem um but it's also I think we're starting to see the beginnings of um these large models being transferable uh to the robotics regime learning in
the general domain language domain and other things and then just treating tokens like gato as any type of token you know the token could be an action it could be a word it could be uh part of an image a pixel or whatever it is and that's what I think true multi-modality is and to begin with
it's harder to train a system like that than a straightforward uh uh text a language system um but uh actually you know going back to our early conversation of transfer learning you start seeing the a true multi-modal system the other modalities benefit uh some some different modalities so you get
better at language because you you now understand a little bit about video so um I do think it's harder to get going but actually ultimately um we'll have a more general more capable system like that uh whatever happened to gato like are those super fascinating that you could have like play games
yeah also do like video and also do check yeah we're still we're still working on those kinds of systems but you can imagine we're just trying to uh those ideas we're trying to build into our uh uh future generations of Gemini you know to be able to do all of those things and and and robotist
transformers and you know things like that are a kind of you can think of them as sort of follow-ups to that um well we see asymmetric progress towards the domains in which the self-flake kinds of things we're talking about will be especially powerful so math and code you know obviously
recently you have these papers out about this um or yeah you can you can use these things to do um really cool novel things uh will they just be like superhuman coders but like in other ways they might be still worse than humans or how do you think about that sort of yeah so look I think that
that that um you know we're making great progress with math and and and and things like theorem proving and coding um but uh it's still interesting you know if one looks at uh I mean creativity in general and scientific endeavor in general I think we're getting to the stage where
our systems could help the best human scientists make their breakthroughs quicker like almost triage the search space in some ways uh or perhaps find a solution like alpha-fold does with the protein structure um but it can't it's they're not the the level where they can create the hypothesis
themselves or or ask the right question and any so as any top scientists will tell you that that's the hardest part of science is actually asking the right question uh boiling down that space to like what's the critical question we should go after the critical problem and then formulating
that problem in the right way to attack it and that's not um something our systems will we have really have any idea how our systems could do um but they can uh they are suitable for searching large combinatorial spaces if one can specify the problem in that way with a clear objective
function so that's very useful for already many of the problems we deal with today but not the the most higher level creative problems um um uh read when you so deep mind obviously has published all kinds of interesting stuff and uh you know speeding up science in different areas um
how do you think about that in the context of if you think AGI is going to happen in the next 10 20 years uh why not just wait for the AGI to do it for you uh why build these domain specific solutions well I think um we don't know how long AGI is going to be and we always used to say uh
you know back even when we started deep mind that that uh uh uh we don't have to wait for AGI in order for to bring incredible benefits to the world um and uh especially um you know my personal passion has been AGI for science and and and health and and you can see that with
things like alpha fold and all of our various nature papers of different domains and material science work and so on I think there's lots of exciting directions uh and also impact in the world through products too I think it's very exciting uh and a huge opportunity unique opportunity
we have as part of Google of of of the you know they you know they got dozens of of of billion user products right that we can immediately ship our advances into and then uh billions of people can you know improve their daily lives right and enriches their daily lives and enhances the
daily lives so I think it's it's a fantastic opportunity for impact on all those fronts and I think the other reason from a point of view of of AGI specifically is that it it battle tests your ideas right so you don't want to be in a sort of uh research bunker where you just you know
theoretically are pushing things something forward but then actually your internal metrics start deviating from uh uh uh real world things that would keep people would care about right or real world impact um so you get a lot of feedback uh direct feedback from these real world applications
that then tells you whether your systems really are scaling or or actually is you know do we need to be more data efficient or sample efficient because most real world uh uh challenges require that right and so it kind of keeps you honest and um pushes you you know keep
sort of nudging and steering your research directions to make sure they're on the right path so I think it's fantastic and of course the well benefits from that society benefits from that on the way many many many many many years before AGI arrives yeah um well the the development of Gemini is super interesting because it comes right at the heels of merging these uh different organizations brain and deep mind um um yeah I'm curious what what have been the challenges there what have been
the synergies uh and it's a it's been successful in the sense that you have the best model in the world now what was I been like well look it's it's it's been fantastic actually over last year of course it's been challenging to do that like any any big integration coming together um but you're talking about to you know world class organizations um uh long storied histories of inventing many many important things um you know from deep reinforcement learning to transformers and so it's very
exciting actually pulling all of that together and and collaborating much more closely we always used to be collaborating but more on a on a on a you know sort of project by project basis versus a much deeper broader collaboration like we have now and Gemini is the first fruit of of that
collaboration uh including the name Gemini actually you know we're implying twins and uh and of course a lot of other things are made more efficient like pulling compute resources together and ideas and engineering which um I think at the stage we're at now where there's huge amounts of
world class engineering that has to go on to build the frontier systems um I think it makes sense to to coordinate that more closely yeah so I mean you you and Shane started deep mind um partly because you were concerned about safety um and you you saw aji coming as like a live
possibility do you uh do you think the people who were formerly a part of brain the half of Google Deep My Now do they do you think the approach it in the same way have them in cultural differences there in terms of that question yeah no I think overall and this is why you know I think one of the
reasons we joined forces we google back in 2014 is I think um the entirety of Google on alphabet not just brain and deep mind take these questions very seriously of responsibility and um you know I kind of mantra is to try and be bold and responsible with these systems so you know I would
I would class it as I'm obviously a huge techno optimist but I want us to be cautious with that given the transformative power of what we're bringing bringing into the world you know collectively and um I think it's important uh you know getting to me one of the most important
technology humanity will ever invent so we we've got to put you know all our efforts into getting this right and be thoughtful and sort of also humble about what we know and don't know about the systems that are coming and the uncertainties around that and in my view the only the only
sensible approach when you have huge uncertainty is to be sort of cautiously optimistic and use the scientific method to try and have as much foresight and understanding about what's coming down the line and the consequences of that before it happens you know you don't want to be live
AB testing out in the world sure with these very consequential systems because I'm intended consequences maybe maybe quite severe so um you know I want us to move away as a as a field from a sort of move fast and break things attitude which is you know maybe serve the valley very well
in the past and obviously created uh important innovations um but but I think in this case you know we want to be uh bold with the with the positive things that it can do and make sure we realize things like medicine and science and advancing all of those things whilst being um you
know responsible and thoughtful with with as far as possible with with mitigating the risks yeah yeah and that that's why it seems like the responsible scaling process is something that you look at is a very good empirical way to pre commit to these kinds of things yes exactly yeah
and I'm curious to be able to sense of like for example when you're doing these evaluations as if it turns out your next model um good how to help a layperson build a pandemic class will buy a weapon or something uh how you would think first of all of secure making sure those
weights are secure so that that doesn't get out and second uh what would have to be true for you to be comfortable deploying that system how comfortable like how how would you make sure that that that latent capability is an exposed yeah well first I mean you know that the secure model part I think we've covered with the cyber security and and make sure that's well class in your monitoring all those things I think um if the capability was was was was discovered like that through
red teaming or external testing by you know uh uh uh uh government institutes and or academia or whatever independent testers um then we would have to fix that that loophole would depending what it was right um if that required more um uh uh uh different kind of perhaps constitution or or
or different guardrails or more RLHF to avoid that or removing some training data um that could I mean depending on what the problem is I think there could be a number of of of mitigations and uh so the first part is making sure you detect it ahead of time so that's
about the right evaluations uh and right benchmarking and right and right testing and then um the question is how one would fix that before you know you deployed it so I think it would need to be fixed before it was deployed generally for sure if if if that was an exposure surface
right right um final question um uh uh you know you've been thinking in terms of like the end goal of Asia at a time in other people thought it was ridiculous in 2010 now that we're seeing um this like slow takeoff where we're actually seeing these like generalization and intelligence um
what is like the psychologically seeing this what has that been like it has just sort of priced into a world model so you like it's not new news for you or is it like actually just seeing a live you're like wow like uh this is something's like really changed or how hard what does it feel like
yeah well for me um yes it's it's already priced into my world model of how things were going to go at least from the technology side but um obviously I didn't we didn't necessarily anticipate the general public would be that interested this early in the sequence right of of things like
maybe one could think of if we were to produce more if if say like a uh chat GPT and chat bots hadn't got the kind of got the interest they'd ended up getting which I think was quite surprising to everyone that people were ready to use these things uh even though they they were lacking in certain
directions right you're impressive though they are um then we would have produced more specialized systems I think built off of the main track like alpha folds and alpha goes and uh and so on and our scientific work and then um I think the the the the the general public maybe um would have
only paid attention later down the road where in a few years time where we have more generally useful assistant type systems so that's been interesting so that's created a different type of environment that we're now all operating in as a as a as a as a field so um it's a little bit
more chaotic because there's so many more things going on and there's so much VC money going into it and everyone's sort of almost losing their minds over it I think and I and I and I and what I just the thing I worry about is I want to make sure that as a field we act responsibly and thoughtfully
and and scientifically about this and use the scientific method to approach this in a in a as I said an optimistic but careful way and I think that's the I've always believed that's the right approach for for for something like AI and um I just hope that doesn't get lost in this huge rush sure sure well I think that's a very place to close. Demon still much thanks to thank you so much for your time and for coming on the podcast thanks has been a real pleasure.
Hey everybody I hope you enjoyed that episode as always the most helpful thing you can do is just share the podcast then did to people you think might enjoy it put it in Twitter your group chats etc just blitz the world appreciate your listening I'll see you next time cheers