Some great guests here. I think you guys know me and opposite Prime Minister, but if you guys could introduce yourselves and say a bit about you and you know you've done both done amazing things, so please let me shy. Sure thing. So I'm Greg Brockman. I'm the president and co founder of Open Aie. But we started Opening Eye out of my apartment in twenty fifteen, twenty sixteen. So yeah, it's been a really exciting time to just, you know, be able to help not just move this field forward,
but try to help steer it in a positive direction. I think that's really foundational to how I think about AI. What got me excited about this field is the fact it's going to be so transformative and I think it can benefit everyone in huge ways, but we also really need to mitigate the downsides. And so I think that's something that really resonated with me about the earlier conversation.
I hope we'll talk about a lot upcoming minutes. And since Greg was too modest to say so, I'm going to add something to your introduction of yourself there, which is just a few years ago when Greg and his colleagues were saying, yo, in a few years we can act. We believe we can actually build AI that can master human language and common knowledge. A lot of my colleagues and AI were like, that's crazy science fiction. It's
going to take decades, but you did it. We literally had an intern in twenty six teams our very first summer who we had this conversation about how to solve the touring test, and we had all these ideas and whatever, and we were like, all right, three years, we'll solve the touring test. And that intern told me I'm out, like I'm not joining full time, like you guys were crazy. And three years later we had GPT
two, so we didn't quite solve the Turing desk. So he was right, But I think that we had far more progress than I think most people thought possible. So I'm Max tech Mark, professor doing AI research at MIT, and I think it was fought on what we heard earlier here from you, that artificial intelligence will be both a blessing and a curse. I've very for a long time and very excited about the blessing part, how we can use it to solve all the problems that we haven't been smart enough to solve
before. And go for beyond and help life flourished for billions of years on Earth and beyond. And I also I'm very concerned about the curse part. So nine years ago I founded this nonprofit called the Future Life Institute. Together with Elon we launched the first ever attempt to the mainstream AI safety by launching a worldwide grants program and educate and do conferences and so on that you Greg participated also to make sure we can steer this every more powerful technology away from
the curse towards the blessing direction. Well, can I say that I want to plug this book? He wrote this book and I read it. It's a good book. It's a very good book. Was it chat guptwo or I have an alibi because not even Greg was fast enough to have it written by the time chat gupt ain't good enough yet to do this. But from what I hear you guys saying, it's good. Yeah, give it a
minute. It was thirty seconds. Well you see, Well let's talk about that, because you know, I think that the kind of things that you ring here are I think a monumental. You know, when I see the talent on this stage, excluding myself, I'm reminded of what JFK. John Kennedy said, you know, when he brought his Gifted team into the White House. He said, this is the most gifted team ever since that's sat here, ever since Thomas Jefferson dined alone. So I think you're about to
surpass that, and you are changing history. And I'm sure you know when such power comes enormous responsibility. And I think that's the crux of what we're talking about here is how do we inject a measure of responsibility and ethics into this, into this exponentially changing development. How do you do that? That's really what I'd like to know from you. I think that this should be
a very interesting discussion. And you know, I think I know almost everyone in AI, and I think I really can't think of anyone who's got bad motivations and AI. I think everyone I know an AI has good motivations. They want to create technology that helps the world. But one you can also say, if you think of someone like Einstein, who was obviously a very sort of peace loving person, and you know, he didn't think that there would be a nuclear bomb created as a result of his discoveries in physics.
He wasn't thinking, let me, let's make super weapons. He was thinking, let's try to understand that the truth of the universe. So it's just
important too. I always want to just preface, you know, any criticism of AI development with it not being a criticism of the people doing it, but just fair in mind that just as Einstein didn't expect his work in physics to lead to nuclear weapons, we need to be cautious that even with good intentions, the best of intentions, that we could create something bad, that that that's one of the possible outcomes, and we want to try to take
whatever actions we can to ensure that the future of humanity is good and that
is much more blessing than That's definitely why I work on it. I think that not for evil it turns out, But I think that balance is really key, and I think that one of the things that is important is to really calibrate the choices with respect to the technology, right because I think this technology it has this way of being very surprising, the way that people pictured like you read asthmav you read kind of all the science fiction you read Ellie
as you're a Kowski, and kind of all this thinking people are doing about very smart machines that are smarter than humans, and somehow none of it really predicted GPT three, right. Somehow the technology that we're building, it's just surprising. It has very different properties. And I think that even from you think about like chess, people used to think chess would be we solve that, we solve human intelligence, and it turns out that's the first thing to
go. And so I think that what we're really seeing is that we need to be calibrated and see not just what the current systems are doing. And this is one of the reasons we deployed chat GPT is it's just really important to not get in our heads and think, oh, this is how it's all going to play out, this is how people are going to misuse it. But you need to actually see that. You need to actually have people spending a lot of time thinking about where the slots in, where it works,
where it doesn't, but also to really try to peer ahead. And so we've been putting a lot of effort into capability prediction. One of the things that I think was kind of undersung in our GPT four release is that we actually had very good predictions of exactly how good GPD four would be before we even trained it, and that's something we can never do with previous models. And I think that this and doing that it's a sidal scale, Like
I think that's one answer for how do we actually get this right? Because if you can see around the corner, then you have an answer and an AI. It's always been the opposite. We've always been taken by surprise. Well, I want to ask you because there are sort of layers of questions here. The Max's book takes you to the existential question of whether you know you project basically machine intelligence or human intelligence into the cosmos, human intelligence turned
into machine intelligence into the cosmos and so on. That's a big philosophical question. I'd like to think we have about six years for that, okay, But the other questions that I'm dealing with as a leader of a country that is blessed with a lot of AI talent rated about number two and talent relative to population size and AI. But I wanted to be among the top three because it's important to advance the country. But I think also for it raises
questions that we raised before. But here's another one. Okay, as chat GUP four or chat. GPT eight comes into being, and you have Hollywood strikes already about screenwriters and directors and actors. Okay, what do you think will happen to the job market? I mean, we had this conversation, what do you think will happen? I mean, you know, you can tell me, you can comfort me with all previous revolutions that there were more jobs, many more jobs created than lost. But you really believe that now?
Yeah, I think it's I think it's far from obvious. I think it is a correct question. And I also think that it's important to kind of look at the whole picture of for sure, their jobs are going to go away, what's going to be the balance of like, you know, marginal job creation? And you can tell the story of oh, it's going
to be very comforting you look at every previous technology. I do think AI is not like every previous technology, right that this is usually we kind of are seeing automation of mechanical parts of creation, and here it's almost this creative
aspect of creation. But I think that we want to even go one step further from where we've been, right and not just okay, like you know, I could have been an artist and now an AI is a competing artist, but you want AI that can help us come up with ideas and solve problems that we just couldn't before. And so what does that world look like? And you know, if your post scarcity world but people don't have to work to survive, you know what set of people? Because people I identify
themselves, you know, your whole identity is around your work. You identify yourself by your job, yes, exactly, And so what will happen then? And what does that look like? And so I don't come with an answer to this question of exactly what the balance will be like. I think there will be fundamental changes and how we even relate to work and to meaning that will happen and may result in a landscape that just looks very different.
That the answer to that question is, you know, if you're a hunter gatherer and someone's like, you know, what's going to happen to our hunter gatherer job post agriculture of revolution, it's just a weird question, right, It's like the framework changes on you. And so that's that's not a cop out, right. I think that it might be the case that actually many more jobs go away than get created, and there's a lot of chaos and
turmoil, but I think we got to be ahead of it. And the more that we can predict it, the more that we can greet answers. And we have teams that try, right, But I think that we're just getting started and we need help. Well, it'll change our economic models. Okay. I'm a clear free market guy. I liberated and I helped liberate the Israeli economy to make it a sort of a technological powerhouse. But that's because of free markets, and free markets means open competition. And it's fine,
but you stop at the monopolies and monopolies edge. Okay, well you do that. I had a conversation with one of your colleagues, Peter Tail, and he said to me, oh, it's all scale advantages, it's all monopolies. I said, well, yeah, I believe that, but I think we have to stop it at a certain point because we don't want
it to press competition. Okay. Well, Ai is producing you know these this wall and you have these trillion dollar companies that are produced to one overnight, okay, and they concentrate enormous wealth and power with smaller and smaller number of people. Okay, and the question is what do you do about that
monopoly power? What do you do about competition? And if you are going to if you are going to cannibalize a lot more jobs than you create, then we have to change the structure of our economic policies and our political policies to take care of the people who are not going to find jobs. We're not going to contribute added value to the economy. We have to make sure that they have a living, a decent one, and they have all the
services. Will probably have the money to do that, but it requires a challenging model, certainly for a free market guy, a free market disciple like me. And that's coming very fast. It's not going to come slowly, and we have time to adjust. I like how you challenge the question what's going to happen? I like to continue on that thing because I think if we asked this very passive kind of question, what's going to happen to us, let's just sit here and eat popcorn and wait for the future to happen,
then we're steading straight for the curse. I think this is a sort of very passive approach which I think is just gonna you know, We've done this experiment before when a new species showed up on the planet that was smarter than all the other ones, and what happened was then the underthal automentics think, if we just very passively go into this, I think it's very likely humans will go extinct. Also, I think that's and I think that's the
wrong attitude. Even in economics in a country, you don't just sit back and wait to see what happens and get child labor and the super monopoly of doom. No, you ask yourself what do you want to happen, and then you put in place institutions, you ban child labor, you put an anti trust. And in this case, I think the analogous question is how do we make sure that with this ever more powerful tech, it's we who control the AI rather than the other way around. Right, So I was
honored that you had this book. If this book still hasn't put you to sleep, because this is a technical nerd paper I wrote with this one's written with technological approach to how we can control machines that are even smarter than us
by actually having other AIS formally prove that the I is safe. And the reason why I'm so excited about the proving part is, you know, the only way you can trust something much smarter than you to still do what you want is if you can prove it, Because no one, how clever it is, it can never do what's provably impossible, and it turns out it's super It's much harder to come up with proofs about math or about what a program is going to do than it is to check that the proof is correct.
You can write a three hundred line computer program which will check any proof claiming anything about anything else. We can understand that if we force AI to prove things to ourselves, to us humans that's going to meet our specifications, then we actually have a reason a way we can control things that are smarter than us. How do you know? How do you know that the proof
is not disputable? That's the beauty of mathematics. You can actually check even if it's a brilliant proof, rather than I couldn't come up with myself. I can just see I just plug this into this and this, Yeah, you know it checks out. And there's a big science in doing this with proving things about programs for cybersecurity because companies are so fed up with losing millions of dollars by begetting hacked all the time because yet another bug was found and
or a shell or whatever. That there's been years of investment. Greg and I had a fun conversation actually just before we went on stage here about how this field of trying to prove that things are safe is still in the stone age and there's pre large language model phase where it's most the humans trying to make the proofs. I think it's ripe for a revolution where the kind of technology that Open Eyes building can totally turbocharge our ability to prove things about the
code. And I think there's a bigger picture too, because in my mind, if you think about cybersecurity, it's always a cat and mouse game. You have attackers, you have defenders. Each one's leveling up defeating the other. Is there ever going to be a time when the defenders totally win? Is there going to be a time when the attackers totally win? And I think with AI that balance can change. And I think that formal verifications an
example where maybe you can have defenders just win once and for all. And
I think there's a lot of asterisks there. I think it's definitely a maybe, But I think that at least at a technical level, formal verification has been this idea of if you could mathematically prove these systems are safe and good and that you don't have bugs, and you're linings, colonel, then all these bugs we've seen we were talking about heart bleed and shell shock and all these these bugs from the past ten years, those would be solved, those
would go away. You would know there'd be no future one of those. Now you need to be very careful because you need to make sure that your undergoing machine model is like actually correct in a bunch of details like that. But it's just interesting to think that these fundamental things we believe about how security works might just be up it. And we've never had an opportunity for this. That will give us a lot of free time exactly. And it's worth
adding for the nerds listening to this to like the code. You know, normally, debugging, all that ever does is is all it can never accomplish is proved that there is something wrong. If you don't find any bugs, is there one more? What when you prove it? When it's what's called formal verification, As Greg said, that means guaranteed you'll seal the deal. Seal the deal, all right. That assumes, by the way, that mathematics is, you know, that's it. It's the law of the universe.
It's not going to change. You're not going to get alien mathematics. You don't need to assume that. Actually, well you're sort of getting the girls incompleteness theorem and things like that. That has comes in a big way when you're dealing with infinities of various kinds. All the computers, even Gregg's very biggest servers, it's all finite. I would say, you still got
to watch out for the alien mathematics. But you can totally formalize what it means that it's impossible to hack into his laptop, and if you have a rigorous proof, I think that's heals the deal. So I think that beyond these, you know, these enormous questions that you raise, I think from me, the immediate problem is to fashion a national policy, which I tend to announce in a few months, and you guys are simulated me. I have to say this visit, but you sort of think, Okay, what
is it that you what is it that you were trying to achieve. You're trying to better all the systems transportation, health, education, agriculture, production, everything for government, for the private sector, for philanthropy, for academia and so on. And what is the role of the government in this? And you know, various kind trees are seeking solutions to this. What do
you do? What kind of instruments do you build? I think the larger question is you still have the monopolization of AI power in small concentrations because of what you said before, Elon, you said, basically, it's still big. You know, you need the big data, you need the big you know, the big computational power, you need the talent for creating the algorithms,
and that's concentrated in a few countries. We have another test, not only to make sure that we take care of ourselves, but how we have these benefits percolate to you know, the rest the scientific and technological have nots. And I think that's that's another real issue. But so far, you know the advances that we've made, at least in Israel, I don't know
anything from disconquy. Remember that you used to have something like that, And I don't know that a drug, the camera pill that goes into your intestine or any other thing that you have, from ICQ to cherry tomatoes that benefits everybody. Drip irogation, okay, in Ai, it seems to me that you're gonna have concentration of power that will create a bigger and bigger distance between the haves and the have nots. And that's another thing that causes tremendous instability
in our world. And I don't know if you have an idea how you overcome that. Well, first of all, I want to say just on the talent density front, we have a number of Israeli engineers, scientist managers and they are top notch, no question. We have a manager who runs one of our core teams who is ex Special Forces and there's nothing that phases him. So I think there's something about the Israeli way that I think is
actually very conducive to this field. And I think that it is really important to counteract this rich, get richer phenomenon, at if it comes at the expense of everyone else. Right, I think that there's a the world we should shoot for is one where all the boats are rising, and I think that there will be some sort of inequities in the landscape, and I think that's something to address, and that's something that I think that leaders such as
yourself, I think need to be very cognizant about. You know, I think that we've talked about UBI like opening actually has run. I think the world's most comprehensive UBI. Something people talk about universal basic income, the idea of just the government just distributes money, right, that's just what it does. If most people are not working, producing, you know, dollars themselves, and there's a world of plenty. Is that a mechanism that can work?
Maybe it can, maybe it can't. There's a lot of assumptions that would go into a world where it works. It's clearly I think also has a lot of downsides to it. But I think that we need to have creative solutions because again, I think that just like defense versus offense might fundamentally change economic production and what it takes to actually build even a stable society I think is going to fundamentally change. Well, Ellen, you said when I
asked you this question one of our nighttime conversations. I asked you, well, what are you gonna do when people don't have jobs and we shell out money for them and you said. I think you said, oh, what's bad and living in paradise? Right, I think you said that, Yeah, I mean it's I mean this is a question of like, you know, the concept of heaven. Now, in heaven, I don't think you
need to work in heaven. Haven't read that anywhere, So they're not laughing, you know if you want so, you know, the very positive scenario of AI is actually in a lot of ways the description of heaven in that really nobody would need to work. I wouldn't even quote universal basic income. I'd say it's probably universal high income. I'm describing the best case scenario here, so I'm not saying this is definitely what occurred. There's a range of
scenarios from very negative to very positive. Very positive scenario is basically sounds like heaven. You can have whatever you want, you don't need to work, you have no obligations. Any illness you have can be cured. And when do we die? Well, that's a good question. I think it'll be a choice. I think it probably ends up being somewhat of a choice. So you want this world, well, I'm not saying I don't know. There's some there's a sort of a philosophical debate is the concept of heaven as
it is normally described. Actually, what do you want? I mean, do you want to be in a future or world where you know the in the AI sense that a I could do everything better than you by far? That's that's because you know you have, like man, search for meaning questions thing good work and what meaning if you if, like I said, if I can do everything better than you do, we live a life of hedonism. I don't know, it's hard to say, but that is where things
ahead. In a positive scenario is that there would be no scarcity of goods and services. The robots and computers can make as much as much of whatever you want, for as many people as you want, with really no limitation. Any scarcity we have would be artificial, meaning like if you define that in particular artwork done by a human is unique and special, then it's artificial scarcity. But there will be no actual scarcity unless we define it arbitrarily,
define something to be scarce. So I think for a lot of people that is a good, good future. I might be personally a bit frustrated because I'm like, well, what am I supposed to do? Now? You'll think of think of something. I think that this is a very interesting set of challenges. What would we do if people have no jobs? But I think it's important to remember that we're not automatically going to end up in this
situation in the way we're thinking about it now. We could also, I think, very likely end up in a situation where people have no jobs because there are no people anymore, where we literally go extinct. You know, if we had a statement cut that came out in May, right that I think Sam Olmen and you both signed right saying and ours left under Lyon came out with it a couple of days ago saying we have to take seriously human extinction. And if this sounds like very crazy science fiction, you know we
were joking. But just a few years ago all your success sounds like science fiction. Two and you said that we don't have a lot of time. These are things that could happen relatively soon. And there's how it's very important to me, because I want the good future, to think about how we steer away from those sort of scenarios. One way in which we could end up and this kind of bad future is pretty obvious. The idea that we can lose control to something more intelligent is not new at all. That Alan
Turing himself said this in the fifties. That's kind of the natural thing to expect. That's what always happened on our planet before. When a news more speechies came in, it took control over the older ones, and then you know, bye bye and Neanderthals by by mammoths whatever. More concretely, I think, even though Hollywood is full of scenarios where someone somehow loses control of AI and it takes over, there are more sneaky ways in which we could
end up in this sort of bad state, even more willingly. So for example, right now, I'm a big fan of the free market too, you know, But it's the fact that if people are competing against other people, they will often seed control voluntarily at the machines to get an edge on the other guys. So companies who replace workers with machines out compete companies who
don't. Armies who seed control to machines out compete armies who don't. Once we have really powerful any of that, that's better at running companies and countries and humans, you know, than countries that replaced their prime ministers by AI might outcompete others, and then we end up in this future where all these things are happening, but it's not our future anymore, and we're like, wait a minute, where did we go along wrong? Here? It was
supposed to be for us. But that's the point, you know, forgiving for borrowing a page from your book, literally a page from your book, I'm confessing I'm a speciest Okay, Okay, So I'm for the human species. Okay. We can talk about other evolutions and you know, machine launched intelligence and so on, but I'm you know, I'm for you know, in the Jewish tradition, there's a saying, how do we translate this? The impoverished of your own city, of your own town, proceed the impoverished
of other towns. So the human species, as far as I'm concerned, is something that I'm vitally interested in maintaining. And I'll tell you that the problem we have is not merely extinction, which I hope you can, maybe you can extemporaries just elaborate on. But here's something else. Even if we don't go extinct, how do we define ourselves? You talked about that, Greg, Okay, so we defined ourselves. We had this myth in the Bible, we had heaven, you know, paradise you can just pick off
the trees, you know, pick off the fruit from the trees. You'd have to do nothing, and then the serpent comes and messes things up, and then we're condemned to toil for our labors. Life is a struggle. It's defined as a struggle to define all the time, as a struggle where you're competing with the forces of nature, or with other human beings or with animals, and you constantly better your position. This is how the human race has defined itself, and our self definition is based on that, both as
individuals, as nations, as humanity as a whole. Now that could be challenged, and it is challenged. So it's both our self definition throughout human history and evolution that is being challenged, and also the question of our continued existence, which you know you can please explain why you think we're in danger
of extension. Well, well, I was gonna say. I mean, I think this whole arc in my mind is all about paradigm shift, right, And I think that even the question of what would that heaven push agi positive future look like? I think even that is hard for us to imagine
what the true upside could be because it's not just material abundance. It's not just we all get nice cars or something hopefully I'll tell us, but also that everyone gets a great education, right, you know, everyone has that story of that one teacher that they had who paid attention to them and inspire them on a specific subject. Imagine if you had that teacher for anything you wanted constantly, and just how much better if people would we be? How
much better would we relate to others? Like that is just one example of the kinds of upsides that are possible. And I think the question of well what does it mean to be a human? You know, if you back to how do you predict what's going to come next? Actually the thinker who I think had the best foresight about how the AI revolution was going to play out is actually Ray Kurtzwile you know he yeah, and his book Singularity is
near gets like a lot of shit. I think that people that you know, kind of assume it is going to be this almost like just sort of religious text, but instead it's a very dry analytical text. And he just looks at the compute curves and he says this is the fundamental unlock or of intelligence. Everyone thought that was crazy, and now it's basically true. It's basically common wisdom. And part of what he says is, look, what's going to happen is in twenty thirties. First of all, he says,
AGI twenty twenty nine. Yeah, I keep telling people it seems to be almost exactly right. It's spooky. It's spooky. Twenty thirties is when the merge happens. So we've got neurallink coming, and you know, maybe other systems like that, And what does it mean once you actually are kind of merging with an intelligence. Well, neuralalink is necessarily moves slower than AI because when when have you put a device in a human? You have to be
incredibly careful. So I think it's not clear to me that the neuralink will be ready before a GI. I think AGI is probably going to happen first, but you know, we'll have the sort of AGI singularity, you know, so as digital superintelligence is called like a singularity, like a black hole, because just like with a black hole, is difficult to predict what happens after you pass the event. Arison of black hole, and we are currently so you know, circling the event horizon of the black hole, that is
at digital superintelligence, the event horizon. So I bet part of the reason for neurallink, although initially we're very helpful to people who have brain or spine injuries, just doing basic stuff like enabling people who are tetraplegics or like a Stephen Hawking to communicate actually even faster than I'm communicating with a fully functional body. So and ultimately to improve the bandwidth between the cortex and your AI version
of yourself. In fact, I think a lot of people perhaps don't quite realize that they're already a syebalk. So you've got your lymic system, your sort of basic drives, your cortex, which is the thinking and planning, and then you have tertiary layer, which is your computers, your devices, your phones, laptops, all the servers that exist, the applications. And in fact, I think probably a lot of people have found that if you leave your cell phone, take it away, if you get if you forget
your cell phone, it's like missing Limb syndrome. You know, you've like read that thing, go Losing your cell phone is like missing Lin syndrome. So, because it is your cell phone is extension extension of yourself, the limitation is badwidth. So you the rate which you can input or i should say output in nation into your phone or computer is very slow. So the phone, it's really just the speed of your thumb movements, and with you
know, best case scenario, you're a speed typist on a keyboard. But even that data rate is very slow, which we're talking about tens, maybe hundreds of bits per second, whereas a computer can communicate in trillions of bits per second. So so the and this is admittedly somewhat of a you know, hail Mary shot or whatever lung shot, is that if you can improve the bad with between your cortex and the your digital tertiary self, then you
can achieve better cohesion between what humans want and what AI does. At least that's one theory. I'm not saying this is a short thing. It's just one potential iron in the fire. If ultimately, you know, one hundreds of millions or billions people get a high bound with interface to their digital tertiary self, their AI self effectively, then that that seems like that probably leads to a better future for humanity. So that's this is I'm getting somewhat esoteric.
Hopefully some of this is making sense, and like I said, it's a long shot. But you know, what I find interesting is that I've I've not found any human who wishes to delete either their cortex or their limits system. I probably I suspect nobody in this room wants to do that. It's a role or their cell phone. Well, now I've met people who want to who don't use a cell phone. It's rare. Okay, Well,
I confess for years I avoided this, and I avoided it. I didn't even have Not only did I not have a cell phone within my immediate radius, I didn't have a television set. My phones were thirty five year old phones because I was Prime Minister Visual and I knew what this means. Okay, I mean maybe your audience doesn't know, but we knew. Now, having spent some time in the opposition, I lapsed into the use of the cell phone, and I find it very hard to dissociate from it.
For the reasons that you said. It's become an extension of, you know, a repository of memory, the ability to get information very fast, which you say correctly, is very slow, but compared to what it was before. Remember you had research assistance, have to go to libraries, things like that. Where is that that's gone? But this will be seen as very vastly primitive compared to what you're talking about. Yeah, I'll just add something
is that to max? But the I mean, it's it's sort of a funny thing like if you assume, like a best case scenario, imagine if if you're the AI and you're trying to you just want the human to tell you what it wants, just please spit it out. But it's speaking so slowly like a tree. Okay, Like trees communicate, Okay, they if you watch a tree, like a you know, sped up a version of a tree growing, it's actually communicating. It's communicating with the soil. It's
trying to find the sunlight. You know, it's reacting to other trees and that kind of thing very slowly. But from a tree standpoint, it's you know, not that slow. So so what I'm saying is we don't want to be a tree. That's that's the idea behind a high band with the neural interface is just even when the AI it desperately wants to do good, good things for us that we can actually communicate several orders of attitude fast,
and we currently can. I love that you mentioned the trees there, because if anyone is still struggling to understand why it's so likely that we could get wiped out by AI, you know, as superintelligent AI, we are like trees. They are as much faster than us, and they're thinking than we are. The trees. So if some trees in the rainforest are a little bit worried that some humans are going to come chop them down, you know, and they're like, Oh, don't worry, We're so smart, we'll
stop those humans. Good luck with that, right. I love how you went philosophical here and talked about what it means to be human, and I wanted to follow up on that, and but before that, maybe just continue that. Okay, So, how you're assuming that the danger of human extinction is that you have machines, supermachines, superintelligent machines that outpass humans by orders and orders of magnitude who will not necessarily value the continued human existence as we
know it, and basically program is out. That's basically what you're saying, so, if we were to be so dumb as to create the entities that are much more powerful than us that don't share our goals, yeah, that's a really bad idea. I mean, in Israel, I think you understand really well that if you have a bunch of other beings who are intelligent and don't share your values, you know, it can go really badly, right, and let's not make that mistake by giving too much power to machines don't
share our values. But of course we can tackle that in various ways. So coming back to what can we concretely do, I just want to say one thing about the philosophical point about being human, just for myself. One of the things that I really cherish the most about being human is the agency that I can actually make choices and make a difference. And one of the most beautiful things about what you were all doing with technology is giving us more
agency. Back in the Middle Ages, we had a life expectancy of freaking thirty years and knowing that we might starve to death next year because of the weather. There, you know what kind of it's so much more inspiring now that we're becoming more of the captain of our own ship with technology able to actually have more agency as a species. So I say, let's use that
agency to create a wonderful future, not to throw it away. And three things I wanted to just suggest for concrete action items we can do here. I'll say a little more about each of them. One has to do with pausing risky stuff, one has to do with controlling the machines, and one has to do with inspiration for the future. So for the controlling, we already talked quite a lot about the importance of staying in control over these machines,
so I don't need to doell on it anymore. But we are I think it's saying things like, oh, what is going to happen to us bring out the popcorn. That's too passive for me. Let's celebrate our agency and ask ourselves not what we think will happen, but what we want to happen. And clearly we want to control these machines we're building them. Second thing, so, I, you know, agree with Gregg on many things,
and I greatly respect your work. One thing where we have a friendly disagreement is you know, I signed this pause letter saying I think we should pause the riskiest things, and you don't. I think that it would be a good strategy for us to emulate what biotech does and just have clear safety standards. You know, if you want to release a new drug, you can't just sell it in the supermarket. You have to do a clinical trial,
persuade some experts somehow that this is safe. And I would like to see really high stakes, very powerful systems, some sort of system like this with standards, and then the companies have and it's on them to, you know, prove that they're safe before they get released it. Who does the standards? Well, and we've solved this, It's been solved in Israel. It's been solved in the US for biotech, not for biotech. But here, you know, it's a's a do you want it? Regulators? Who
do you want it? Legislatures? I mean, it's not the only choice, but you know, it's a pretty pretty clear dichotomy. I don't think we need to reinvent this wheel. The people who decide whether a drug gets approved in Israel is not people in parliament in the Kinnessets, right, it's Israeli scientists, but the government involved in approving them. Exactly we can emulate that, and Elon can tell us all about how we deal with the cars. I don't want to dwell on, and I just think this is not
rocket science. We can put these standards in place and then we can continue doing all sorts of wonderful stuff with a tech. Some stuff will wait a little longer for but it'll be worth the wait because it'll be safe. The last thing I just wanted to say is I think for us to really seize this agency and work together, not just between companies but also across borders,
we really need a shared positive vision. And this is something I really appreciate so much about You, Elon have the guts more than most to talk about like really audacious positive visions for the future. What is it that makes people collaborate and put little squabbles aside. It's always a shared positive vision where people realize that if we collaborate, we'll all be better off. You know,
technology is not a zero some game, right. You can take countries like Sweden and Denmark that had been killing each other for centuries and flat GDP, you know, and then technology came along and they both got dramatically better off. It's so obvious that the technology that you're building. You're building you know, can make all humans just spectacularly better off with AI, and not just on this planet. I like the list before we started you had there about
how it can revolutionize healthcare and so many other things. And I would add to that, you know, why limit ourselves to this planet when there's this beautiful universe where life can flourish for billions of years. If the more we can can just appreciate how huge the upside is if we do this right, you know, the more incentive we will have to actually be a little patient where we need to be a little bit patient, make everything safe and therefore
get this wonderful future. Well, maybe we're sort of lagging behind in our social and political development. This is like having nuclear technology and the Stone Age, what do you do with it? Which I think is you know, because more That's the thing that I talked about in terms of the pace of development, pacing basically what solutions we need to put in place to maximize the benefits and limits the risks. I think we're there. I think we should
have a meeting of minds. You were in you were in the Senate the other day. I presume you were talking about this. I mean, yeah, you don't have to divulge what you can divulge, but divulge would you can? Well, a lot of people in the meeting, so I don't think it was secret. It was just a discussion on AI regulation. And actually Sam Often from open I was there and a number of eye leaders were
were there. And I think generally it would be a good idea to have some kind of AI regulatory agency and start off with a team that gathers insight to get a maximum understanding. Then you have some proposed rulemaking, and then eventually you have regulations that are put in place. And this is something we have for everything that is potential danger to the public. So if it is you know, if we're in drugs in drug administration, we got aircraft with
fa and rockets. You know, there's ever anything that is a danger to the public. Over time, we have learned that often the hard way, after many people have died, to have a regulatory agency to protect public safety. I'm not someone who thinks that regulations some panacea where it's only good. Of course, there are some downsides to regulation and that things move a bit
slower, and sometimes you get regulatory capture and that kind of thing. But but on balance, I think the public would not want to get rid of most regulatory agencies. And you can think of it also as like the regulatory agency being like a referee. You know, what sports game doesn't have a referee? You need someone to make sure that that people are playing fairly, not breaking the rules. And that's why basically every sport has a referee of
one kind or another. So that's the rationale for AI safety. And I've been pushing this all around the world, and when I was in China a few months ago meeting with some of the senior leadership, that my primary topic was AI safety and regulation and they, after we had a long discussion, agreed that this merit to AI regulation and immediately took action in this regard.
So so sometimes it will get this comment of like, well, if the West does AI regulation surely, then what about what if China doesn't and then leaves ahead. And I think they are also taking it very seriously, because you know the opposite of what of a moral constraints you program into the system? I mean, Greg, I don't know what you what are your thoughts on this? Yeah, I mean I think again it's just like this technology
is almost backwards from how you'd expect it to work. Like I think that a lot of people talk about the paper clips problem of if you think of that as if we literally program into a machine make as many paper clips as possible, and it gets really smart, and it's like, well, there's all these houses and factories and other things, they're not producing paper clips.
They're all going to be just paper clip production. Now, all these humans that are in the way paper clip would be a better arrangement of their atoms. And that's just not the technology path we actually seem to be on. You look at GPT three, and it is a system that gets the nuance, It gets the common sense. It has other problems, right, it makes mistakes that no human ever would like kind of loses this chain of thought and says all these weird things and sometimes make things up like we have real
problems we have to encounter. But I think that for me, that the hard thing that I think gets lost in some of the conversations is the nuance right of really being coupled to what's the technology we actually see and particularly how does it depart from our intuitions about where we thought it would go. I actually think it's even interesting. You know, Max mentioned the pause letter.
The pause letter, one of the key parts of it was to not train GPD five for the next six months, and we looked at that and we're like, well, we're not training GPT five for the next six months anyway. And so you know, it's just like again if you kind of miss the sort of nuances of what's going on at a mechanical level. And some of this does require getting like really in the weeds and coming and talking to
us and also opening up and showing kind of in progress work. And I think that's something that will be very important as you think about moving to these very powerful systems. And I also think there's another blessing and curse of how the technology is playing out, where we talk about digital superintelligence and we also talk about chat. These are very different things, but somehow there's going to be a smooth continuing between them. And so I think that the blessing is
that we see a little bit into the future. We have a little taste of a system that you know, I talked about a personalized tutor for everyone. You kind of have it now. It's not a perfect tutor. It's got a lot of problems, but like you know, you can go use shat GPT for free to ask about anything and it'll give you something. Sometimes it's actually be pretty insightful. Well fair enough, I'll withdraw the seven and a half, but but you'll get there. You'll get to a ten.
We will, we will, we will. Definitely, that's right, And I think it's great for people to see it while we can still laugh about it. Right. I think that when it starts coming up with the new ideas, you type in some problem you're having at home, trying to figure out how to do something, and it just comes back with, oh, you should say this to this person, and you should do that, you should you know, here's say you should revenge your schedule. And it's just
like great ideas, Like that's gonna feel so different. And then one day, when we're actually having a system that can actually just autonomously go off and do a bunch of research and say, here's this crazy chemical formula, like please mix them together in these ways. You want to make sure that thing
trustworthy right before we start mixing those chemicals. And so I think that this question of how do we separate out the sort of diffusion of today's AI technology and application and really, you know, all the debates that we're having, let those happen, open source technologies like everyone's building. We should not squelch
those, we should actually promote those. But I think that also looking ahead, being ahead of the game here and saying there is going to be a point and it's where we're hitting that wall, where we're going to be building the most massive machines that humans have ever built. Like Youan talks about these massive rows of data centers with all these computers, like like it's one of those things where you kind of have to see it to really feel it.
Like I've walked through some of these data center halls and you just get a sense of the scale of what's being built. And it's very different when it's abstractly just like you're looking at, oh is this many GPUs and you know, looking at numbers, it's like you just realize the mechanical force that is behind what you're building for cognitive intelligence. And I think that kind of thing
we need international answers for. I fully agree with you. I think, Chris, of all you know, with all due respect of the pause button, we ain't going to get it. It's not going to happen. I think you can't put the genie back into the bottle, and I'm not sure you can slow it down. The only thing you can do is just to accelerate our own efforts. And I think you're right. It has to be international. I think there. I think you start with a like minded and
like valued states. Start with that. And frankly, I think you know, obviously, you know what AI stands for. It stands for America and Israel. Obviously, I'm just plugging my own country. But it's a it's not an empty plug, because I think that there's a concentration of talent an Israel that makes it more than a mouse compared to the American elephant. It's more than that. It's a very nimble mammal, very fast compact, and it's it's also got this feed of military AI which moves very fast and takes
a very large part of the population. The talents there screens it and it trains it very quickly. So I think that you start with a like minded states, you definitely should have a very robust conversation. First of all, decide among us what is something there are things that we wouldn't want to share, and it's obvious related to our security. There are standards that we have, ethical standards and other regulations that we have to achieve quickly among between industry
and governments quickly. And then I think we have to conduct a robust discussion with the other powers of the world based on their self interest, as you began to do. And I think that's a pioneering work, and I think we have a shot maybe at getting to some degree of control over our future, which could be amazing. You know, I'm not sure I want to live in heaven. It could be boring. I don't mean the boring company. I mean it could be boring, But I think we're headed. I'd
much rather have a boring heaven than an interesting hell. So here at Tesla, they have a different name for pause button. They call it the break. And when we launched this pause letter, right, I personally didn't think that there was immediately going to cause a pause for me. The real purpose of it was the mainstream the conversation about these issues, which really did happen.
And you know, I really appreciate you and Sam and your colleagues for saying very openly that there may come a time when you will want to pause certain things if you meet certain criteria. Right now, if Elon says that may come a time when he's going to slow down his Tesla, I would feel I would trust that more if I knew that Tesla actually had breaks in it, so that they had like a mechanism by which you could actually slow
it down. So this is something I'm very curious to actually to hear it from you about if what criteria and would you say that you all have can can you say it in the future, say it already now, Like what are some things which would make you want to pause? And and then won't it be difficult with the actual mechanism, like to resist commercial pressures to continue
even if you and Sam want to pause. Well, I actually kind of agree with both of you right that, I think that my picture of how the future is going to play out in terms of AI development is not as simple as oh, you're scaling models and then you see something a little bit weird and then you like stop all scaling and you all go home or something, right'. It's it's definitely not going to be that. I think that we need to be making maximum velocity efforts towards a good future, right,
And I think that is something that you don't want to pause. You actually want to put the foot on the pedal. Now, of course, the question is what actions actually get you towards there, which ones are against it, you know, Like I think that nuance is something that I think is
what we should be debating. But that's why I actually bounce a little bit off of the framing of a pause, because I think even to the point of GPD four, doing all this prediction work ahead of time, like if that prediction work was a little bit wobbly, weren't quite there, and we actually do this all the time, that there's like some experiment that came back a little bit strange, we're not really sure we understand it. We actually
spend a lot of time to fix that. And it's not even you know, even if you forget about the safety reasons, and actually one of the reasons we do it is safety motivated. But if you even forget about that, you're just not going to build a good system. And so I think that there's a lot of places in AI where there's actually great alignment between capability and safety, between building something useful and something that is safe. It's not
perfect, but I think it is actually surprising where there are divergences. Actually, the single biggest divergence we've seen has not been owe too much pressure to deploy quickly because I actually think that getting things out, Like you can see, the market it hates hallucinations, It hates it when the model makes something up. All the jokes we made about oh it doesn't give you good insights, the market hates that stuff, right, And actually I think it's very
aligned with safety to fix them. But what the market actually pushes us against is actually when it comes into conflict with other values, for example privacy. Right, So that the way we launch the initial gpt APIs was that we said, hey, we're going to monitor everything. We're going to record everything, like if something sketchy comes up, we'll go and look and we'll be able to look back in the logs. We'll be able to see what you did. People hate that, right, like you know you kind of want
that for everyone else, but for yourself you want privacy. And so how do you balance those two factors? And we actually see this very much in terms of to actually win deals to decide to be able to give companies the assurance that hey, we're not going to retain any of your data, will delete it immediately. That removes our ability to do any investigation post talk.
And so this is actually a place where I think government can be very helpful, where norms can be very helpful to say, hey, even though you know that this is what would be you know commercially most viable, that this is what the rule is going to be, and everyone has to play with
it. You don't ever race to the bottom. So I think that again, it's all going to be nuanced, it's all going to be I think, like I've just accepted that this field it's totally counterintuitive, Like everything we learn this is like it just would make you just like if you told yourself ten years ago you have said, there's no way. That sounds totally crazy, and somehow we have to navigate through that. Well, the only thing that I can add to this conversation is to invite you all to as well.
I'd like to, in fact continue this conversation, in fact, bring a few more of your colleagues and a few leaders to discuss this international strategy for like minded states to begin with that, you know, first like minded states, then the world. Okay, how about that as a plan of action. But I think that I think that the work that you're doing is amazing. Nothink short of it. You know, Elon was saying, what
is a human being? It's sort of this two legged thing ape like true, with a cortex and a computer and its wants, desires and a brain. But the brain is developing in ways that we couldn't possibly imagine, and
it just doesn't stop. Well, I mean, from my other comments, like the thing that paps should provide some hope for digital not being controllable in a very very specific way but acting in a way that is beneficial to humans is that our cortex and our limc system, the cortex is actually in service to the limits system. So you know, the cortex, the thinking part, is constantly trying to make the instinctual part happy, even though it is
much smarter than the instinctual part. And so I think there's hope there that digital superintelligence, even though it is much smarter than our cortex, will none less want to make the cortex happy and ultimately LIMICS system happy. I mean not to get too off color here, but the sheer amount of thinking and about two that people have done to have sex is a lot, but not
sex will procreation, just for the to make the limics system happy. So you know, given how much you know thought has gone into getting laid essentially, but for the sole purpose of getting laid, not procreation, and all that does is make the limbic system, you know, give us block of
joy that gives. I think that's actually suggests that there is a good future where even if we don't understand the digital superintelligence, just as the Olympic system does not under understated the cortex, it might still actually at the end of the day, if you're trying to make our limbic system and cortex happy, is that a happy note to end? I think it's a great note to end on. Thank you, Thank you everyone, Thank you athletes. So that's really amid