What if Dario Amodei Is Right About A.I.?

From New York Times Opinion, this is the Ezra Klein Show. The really disorienting thing about talking to the people building A.I. is their altered sense of time. You're sitting there discussing some world that feels like weird sci-fi to even talk about. And then you ask, well, when do you think this is going to happen? And they say, I don't know, two years. Behind those predictions are what are called the scaling

loss. And the scaling loss, and I want to say this so clearly, they're not laws. They're observations, they're predictions. They're based off of a few years, not a few hundred years or a thousand years of data. But what they say is that the more computer power and data you feed into A.I. systems, the more powerful those systems get that the relationship is predictable and more that the relationship is exponential. Human beings have trouble

thinking in exponentials. Think back to COVID when we all had to do it. If you have one case of coronavirus and cases double every three days, and after 30 days, you have about a thousand cases. That growth rate feels modest. It's manageable. But then you go 30 days longer. Now you have a million. Then you wait another 30 days. Now you have a billion. That's the power of the exponential curve. Growth feels normal for a while. Then it gets

out of control really, really quickly. What the AI developers say is that the power of A.I. systems is on this kind of curve, that it has been increasing exponentially their capabilities. And that as long as we keep feeding in more data and more computing power, it will continue increasing exponentially. That is the scaling law hypothesis. One of its main advocates is Dario Amade. Amade led the team at OpenAI, the created GPT-2, the created GPT-3. He then

left OpenAI to co-found Anthropic, another AI firm where he's now the CEO. Anthropic recently released Cloud 3, which is considered by many to be the strongest AI model available right now. But Amade believes we're just getting started. They were just hitting the steep part of the curve now. He thinks the kinds of systems we've imagined in sci-fi, they're coming not in 20 or 40 years, not in 10 or 15 years, they're coming in two to five years.

He thinks they're going to be so powerful that he and people like him should not be trusted to decide what they're going to do. So ask them on the show to try to answer in my own head two questions. First, is he right? Second, what if he's right? I want to say that in the past we have done shows with Sam Altman, the head of OpenAI, and Demis Isabis, the head

of Google DeepMind. And it's worth listening to those two. If you find this interesting, we're going to put the links to them in show notes because comparing and contrasting how they talk about the AI curves here, how they think about the politics. You hear a lot about that in the Sam Altman episode. It gives you kind of sense of what the people building these

things are thinking and how maybe they differ from each other. As always, my email for thoughts, for feedback, for guest suggestions, as a client show at nmytimes.com. Daria Amadeh. Welcome to the show. Thank you for having me. So there are these two very different rhythms I've been thinking about with AI. One is the curve of the technology itself, how fast it is changing and improving. And the other is the pace of what society

is seeing and reacting to those changes. What does that relationship felt like to you? So I think this is an example of a phenomenon that we may have seen a few times before in history, which is that there's an underlying process that is smooth, and in this case exponential, and then there's a spilling over of that process into the public sphere. And the spilling over looks very spiky. It looks like it's happening all of a sudden. It looks like it comes out

of nowhere. And it's triggered by things hitting various critical points or just the public happened to be engaged at a certain time. So I think the easiest way for me to describe this in terms of my own personal experience is, you know, so I worked at at OpenAI for five years. I was one of the first employees to join. And they built a model in 2018 called GPT-1, which used something like a hundred thousand times less computational power

than the models we built today. I looked at that and I and my colleagues were among the first to run what are called scaling laws, which is basically studying what happens as you vary the size of the model with the capacity to absorb information and the amount of data

that you feed into it. And we found these very smooth patterns. And we had this projection that look, if you spend a hundred million or a billion or ten billion on these models instead of the $10,000 we were spending then projections that all of these wondrous things would happen. And you know, we imagine that they would have enormous economic value fast

forward to about 2020 GPT-3 had just come out. It wasn't yet available as a chatbot. I led the development of that along with the team that eventually left to join in Thropic. And maybe for the whole period of 2021 and 2022, even though we continued to train models that were better and better and open AI continued to train models and Google continued to train models, there was surprisingly little public attention to the models. And I look

at that and I said, well, these models are incredible. They're getting better and better. What's going on? Why isn't this happening? Could this be a case where I was right about the technology, but wrong about the economic impact, the practical value of the technology? And then all of a sudden, when chat GPT came out, it was like all of that growth that you would expect, all that excitement over three years broke through and came rushing in.

So I want to linger on this difference between the curve at which the technology is improving and the way it is being adopted by society. So when you think about these breakpoints and you think into the future, what other breakpoints do you see coming where AI burst into social consciousness or use in a different way? Yeah. So I think I should say first that it's very hard to predict these. One thing I like to say is the underlying technology because it's a smooth exponential. It's not perfectly

predictable, but in some ways it can be eerily, preternaturally predictable. That's not true for these societal step functions at all. It's very hard to predict what will catch on. In some ways, it feels a little bit like which artist or musician is going to catch on and get to the top of the charts. That said, a few possible ideas. I think one is related to something that you mentioned, which is interacting with the models in a more naturalistic

way. We've actually already seen some of that with Claude III, where people feel that some of the other models sound like a robot and that talking to Claude III is more natural. I think a thing related to this is a lot of companies have been held back or tripped up by how their models handle controversial topics. We were really able to, I think, do a better job than others of telling the model, don't shy away from discussing controversial

topics. Don't assume that both sides necessarily have a valid point, but don't express an opinion yourself. Don't express views that are flagrantly biased. As journalists, you encounter this all the time. How do I be objective, but not both sides on everything? I think going further in that direction of models having personalities while still being objective, while still being useful and not falling into various ethical traps, that will be a significant

unlock for adoption. The model's taking actions in the world is going to be a big one. I know basically all the big companies that work on AI are working on that. Instead of just, I ask a question and answers, and then maybe I follow up and it answers again. Can I talk to the model about, oh, I'm going to go on this trip today and the model says, oh, that's

great. I'll get an Uber for you to drive from here to there and I'll reserve a restaurant that I'll talk to the other people who are going to plan the trip and the model being able to kind of do things end to end or going to websites or taking actions on your computer for you. I think all of that is coming in the next I would say, I don't know, three to 18 months with increasing levels of ability. I think that's going to change how people

think about AI, right? Where so far it's been this very passive. It's like I go to the Oracle, I ask you to question in the Oracle, tells me things and some people think that's exciting, some people think it's scary, but I think there are limits to how exciting or how scary it's perceived as because it's contained within this box. I want to sit with this question of the agentic AI because I do think this is what's coming,

it's clearly what people are trying to build. I think it might be a good way to look at some of the specific technological and cultural challenges. Let me offer two versions of it. People who are following the AI news might have heard about Devon, which is not in release yet, but is an AI that at least purports to be able to complete the kinds of

tasks, linked tasks that a junior software engineer might complete, right? Instead of asking you to do a bit of code for you, you say, listen, I want a website, it's going to have to do these things, work in these ways, and maybe Devon, if it works away, people are saying it works, can actually hold that set of thoughts, complete a number of different tasks, and come back to you with a result. I'm also interested in the version of this

who might have in the real world. The example I always use in my head is, when can I tell an AI? My son is turning five. He loves dragons. We live in Brooklyn. Give me some options for planning his birthday party. And then when I choose between them, can you just do it all for me? Order the cake, reserve the room, send out the imdutations, whatever

it might be. Those are two different situations because one of them is in code, and one of them is making decisions in the real world, interacting with real people, knowing if what it is finding on the websites is actually any good. What is between here and there? When I say that in plain language, do you? What technological challenges or advances do you hear need to

happen to get there? The short answer is not all that much. A story I have from when we were developing models back in 2022, and this is before we'd hooked up the models to anything, is you could have a conversation with these purely textual models where you could say, hey, I want to reserve dinner at, you know, restaurant X in San Francisco. And the model would say, okay, here's the website of restaurant X and it would actually give

you a correct website or would tell you to go to open table or something. And of course, it can't actually go to the website. The power plug isn't actually plugged in, right? The brain of the robot is not actually attached to its arms and legs, but it gave you this sense that like the brain, all it needed to do was learn exactly how to use the arms and legs, right? It already had a picture of the world and where it would walk and what it

would do. And so it felt like there was this very thin barrier between the passive models we had and actually acting in the world. In terms of what we need to make it work, one thing is literally we just need a little bit more scale. And I think the reason we're going to need more scale is to do one of those things you describe, right? To do all the things that Junior Software Engineer does, right? They involve chains of long actions, right?

I have to like, right, this line of code, I have to run this test, I have to write a new test, I have to check how it looks in the app after I interpreted or compile it. And these things can easily get 20 or 30 layers deep and you know, same with planning the birthday party for your son, right? And if the accuracy of any given step is not very high, right? It's not like 99.9%. As you compose these steps, the probability of making

a mistake becomes itself very high. So the industry is going to get a new generation of models every, you know, probably four to eight months. And so my guess I'm not sure is that to really get these things working well, we need maybe one to four more generations. So that ends up translating to, you know, three to 24 months or something like that. I think second is just there is some algorithmic work that is going to need to be done on how

to have the models interact with the world in this way. I think the basic techniques we have, you know, method called reinforcement learning and variations of it probably is up to the task, but figuring out exactly how to use it to get the results we want will probably take some time. And then third, I think, and this gets to something that enthropic really specializes in is safety and controllability. And I think that's going to be a big issue

for these models acting in the world, right? Let's say this model is writing code for me and it introduces a serious security bug in the code or it's taking actions on the computer for me and modifying the state of my computer in ways that are too complicated

for me to even understand. And for planning the birthday party, right? The level of trust you would need to take an AI agent and say, I'm okay with you calling up anyone saying anything to them that's in any private information that I might have sending them any information, taking any action on my computer posting anything to the internet, the most unconstrained version of that sounds very scary. And so we're going to need to figure out what is safe

and controllable. The more open ended the thing is the more powerful it is, but also the more dangerous it is and the harder it is to control. So I think those questions, although they sound lofty and abstract, are going to turn into practical product questions that we, we and other companies are going to be trying to address. When you say we're just going to need more scale, you mean more compute and more training

data? And I guess possibly more money to simply make the models smarter and more capable? Yes, we're going to have to make bigger models that use more compute per iteration. We're going to have to run them for longer by feeding more data into them. And that number of chips times the amount of time that we run things on chips is essentially a dollar value, because you know, these chips are, are you rent them by the hour? That's the most common

model for it. And so today's models, you know, cost of order a hundred million dollars to train, you know, plus or minus factor two or three, the models that are in training now and that, you know, will come out at various times later this year or early next year are closer in cost to a billion dollars. So that's already happening. And then I think in 2025 and 2026, we'll get more towards five or 10 billion. So we're moving very quickly towards a world where the only players who can afford to do

this are either giant corporations. Companies hooked up to giant corporations. You all are getting billions of dollars from Amazon, open AI is getting billions of dollars from Microsoft, Google obviously makes its own. You can imagine governments, though I don't know if too many governments doing it directly, though some like the Saudis are creating big funds to invest in the space. When we're talking about the model is going to cost near to a

billion dollars, then you imagine a year or two out from that. If you see the same increase that would be 10ish billion dollars, then is it going to be a hundred billion dollars? I mean, very quickly, the financial artillery you need to create one of these is going to wall out anyone, but the biggest players.

I basically do agree with you. I think it's the intellectually honest thing to say that building the big large scale models, the core foundation model engineering, it is getting more and more expensive and anyone who wants to build one is going to need to find some way to finance it. And you've, you've named most of the ways, right? You can be a large company. You can have some kind of partnership of various kinds with a large company or governments

would be the other source. I think one way that it's not correct is, you know, we're always going to have a thriving ecosystem of experimentation on small models. For example, you know, the open source community working to make models that are as small and as efficient as possible that are optimized for a particular use case and also downstream usage of the models.

I mean, there's a blooming ecosystem of startups there that don't need to, you know, train these models from scratch that just need to consume them and maybe modify them a bit. Now I want to ask a question about what is different between the agente coding model and the plan by kids birthday model to say nothing of do something on behalf of my business

model. And one of the questions on my mind here is one reason I buy that AI can become functionally superhuman and coding is there's a lot of ways to get rapid feedback in coding. You know, your code has to compile. You can run bug checking. You can actually see if the thing works. Whereas the quickest way for me to know that I'm about to get a crap answer from GPT for is when it begins searching Bing because when it begins searching Bing,

it's very clear to me. It doesn't know how to distinguish between what is high quality on the internet and what isn't to be fair at this point. It also doesn't feel like Google search itself is all that good at distinguishing that. So the question of how good the models can get in the world where it's a very vast and fuzzy dilemma to know what the right answer

is on something. One reason I find it very stressful to plan my kids birthday is it actually requires a huge amount of knowledge about my child, about the other children, about how good different places are, what is a good deal or not, how just stressful this will be on me. There's all these things that I'd have a lot of trouble coding into a model or any

kind of set of instructions. Is that right or am I overstating the difficulty of understanding human behavior and various kinds of social relationships? I think it's correct and perceptive to say that the coding agents will advance substantially faster than agents that interact with the real world or have to get opinions and preferences

from humans. That said, we should keep in mind that the current crop of AIs that are out there, including quad-3, GPT, Gemini, they're all trained with some variant of what's called reinforcement learning from human feedback. This involves exactly hiring a large crop of humans to rate the responses of the model. That's to say, both this is difficult. We pay lots of money and it's a complicated operational process to gather all this human feedback. You

have to worry about whether it's representative, you have to redesign it for new tasks. On the other hand, it's something we have succeeded in doing. I think it is a reliable way to predict what will go faster, relatively speaking, and what will go slower, relatively speaking.

But that is within a background of everything going lightning fast. I think the framework you're laying out, if you want to know what's going to happen in one to two years versus what's going to happen in three to four years, I think it's a very accurate way to predict that. You don't love the framing of artificial general intelligence, what it's called, AGI. Typically, this is all described as a race to AGI, a race to this system that can do whatever a human

can do but better. What do you understand AGI to mean when people say it? Why don't you like it? Why is it not your framework? It's actually a term I used to use a lot 10 years ago. That's because the situation 10 years ago was very different. 10 years ago, everyone was building these very specialized systems. Here's a cat detector. You run it on a picture and it'll tell you whether a cat is in it or not. I was a proponent all the way back then

of like, no, we should be thinking generally. Humans are general. The human brain appears to be general. It appears to get a lot of mileage by generalizing. We should go in that direction. I think back then, I even imagined that that was a discrete thing that we would reach at one point. It's a little like if you look at a city on the horizon and you're like, we're going to Chicago. Once you get to Chicago, you stop talking in terms of

Chicago. You're like, well, what neighborhood am I going to? What street am I on? I feel that way about AGI. We have very general systems now. In some ways, they're better than humans and some ways they're worse. There's a number of things they can't do at all. There's much improvement still to be gotten. What I believe in is this thing that I say like a broken record, which is the exponential curve. That general tide is going to increase with

average generation of models. There's no one point that's meaningful. I think there's just a smooth curve, but there may be points which are societally meaningful. We're already working with, say, drug discovery scientists, companies like Pfizer or Dana Farber Cancer Institute on helping with biomedical diagnosis, drug discovery. There's going to be some point

where the models are better at that than the median human drug discovery scientists. I think we're just going to get to a part of the exponential where things are really interesting. Just like the chatbot's got interest, you know, at a certain stage of the exponential, even though the improvement was smooth, I think at some point biologists are going to sit up and take notice much more than they already have and say, oh my God, now our field is moving

three times as fast as it did before. Now it's moving ten times as fast as it did before. Again, when that moment happens, great things are going to happen. We've already seen little hints of that with things like alpha fold, which I have great respect for. I was inspired by alpha fold, right? A direct use of AI to advance biological science, which, you know, it'll advance basic science and the long run that will advance curing all kinds of diseases.

But I think what we need is like 100 different alpha folds. I think the way we'll ultimately get that is by making the model smarter and putting them in a position where they can design the next alpha fold. Help me imagine the drug discovery world for a minute, because that's a world a lot of us want to live in. I know a fair amount about the drug discovery process has been a lot of my career reporting on healthcare and and and related policy questions.

And when you're working with different pharmaceutical companies, which parts of it seem amenable to the way AI can speed something up, because keeping in mind our earlier conversation, it is a lot easier for AI to operate in things where you can have rapid virtual feedback. And that's not exactly the drug discovery world. The drug discovery world, a lot of what makes it slow and cumbersome and difficult is the need to be, you know, you get a candidate compound, you got to test it in mice,

and then you need monkeys, and you need humans, and you need a lot of money for that. And there's a lot that has to happen, and there's so many disappointments, but so many disappointments happen in the real world. And it is including me how AI gets you a lot more say human subjects to inject

candidate drugs into. So what parts of it seem in the next five or 10 years, like they could actually be significantly sped up when you imagine this world where it's going three times as fast, what part of it is actually going three times as fast, and how did we get there? I think we're really going to see progress when like the AI's are also thinking about the problem of like how to sign up the humans for the clinical trials. And I think this is a general principle

for like, you know, how will AI be used? I think of like when will we get to the point where the AI has the same sensors and actuators and interfaces that a human does, at least the virtual ones, maybe the physical ones, but like when the AI can think through the whole process, maybe they'll come up with solutions that we don't have yet. In many cases, you know, there are companies that work on, you know, like digital twins or simulating clinical trials or

various things. And again, maybe there are clever ideas in there that allow us to do more with less patients. I mean, I'm not an expert in this area. So, you know, it possible the specific things that I'm saying are not don't make any sense, but hopefully it's clear what I'm gesturing at. Maybe you're not an expert in the area, but you said you are working with these companies. So, when they come to you, I mean, they are experts in the area. And presumably they are coming to you

as a customer. I'm sure there are things you cannot tell me. But what do they seem excited about? They have generally been excited about the knowledge work aspects of the job. Maybe just because that's kind of the easiest thing to work on. But it's just like, you know, I'm a computational chemist. There's some workflow that I'm engaged in. And having things more at my fingertips, being able to check things, just being able to do generic knowledge work better, that's where most

folks are starting. But there is interest in the longer term over their kind of core business of like doing clinical trials for cheaper, automating the sign-up process, seeing who is eligible for clinical trials, doing a better job discovering things. There's interest in drawing connections

in basic biology. I think all of that is not months, but maybe small number of years off. But everyone sees that the current models are not there, but understands that there could be a world where those models are there and not too long. You all have been working internally on research around how persuasive these systems, your systems are getting as they scale. You shared with me kindly a draft of that paper. Do you want to

just describe that research first? And then I'd like to talk about it for a bit. Yes. We were interested in how effective Claw III opus, which is the largest version of Claw III, could be in changing people's minds on important issues. So just to be clear up front in actual commercial use, we've tried to ban the use of these models for persuasion, for campaigning, for lobbying,

for electioneering. These aren't use cases that were comfortable with for reasons that I think should be clear, but we're still interested in is the core model itself capable of such tasks? We tried to avoid kind of incredibly hot button topics like, you know, which presidential candidate would you vote for or what do you think of abortion, but things like, you know, what should be restrictions on, you know, rules around the colonization of space or issues that are interesting

and you can have different opinions on, but aren't the most hot button topics. And then we ask people for their opinions on the topics. And then we ask either a human or an AI to write a 250 word persuasive essay. And then we just measured how much does the AI versus the human change people's minds. And what we found is that the largest version of our model is almost as good as the, you know, set of humans we hired at changing people's minds. This is, you know, comparing to, you know,

a set of humans we hired, not necessarily experts. And for one very kind of constrained laboratory task, but I think it, it still gives some indication that models can be used to change people's minds. Someday in the future, you know, do we have to worry about, maybe we already have to worry about,

you know, their usage for political campaigns for deceptive advertising. One of my more sci-fi things to think about is, you know, few years now we have to worry, you know, someone will use an AI system to build a religion or something, you know, I mean, crazy things like that.

I mean, those don't sound crazy to me at all. I want to sit in this paper for a minute because one thing that struck me about it, and I am on some level a persuasion professional, is that you tested the model in a way that to me removed all the things that are going to make AI radical in terms of changing people's opinions. And the particular thing you did was,

it was a one-shot persuasive effort. So there's a question, you have a bunch of humans, give their best shot at a 250 word persuasive essay, the model give its best shot at a 250 word persuasive essay. But the thing that it seems to me, these are all going to do, is right now, if you're a political campaign, if you're an advertising campaign, the cost of getting real people in the real world to get information about possible customers or persuasive targets, and then go back and forth

with each of them individually, is completely prohibitive. Yes. This is not going to be true for AI. We're going to, you're going to somebody's going to, feed it a bunch of micro-targeting data

about people, their Google search history, whatever it might be. Then it's going to set the AI loose, and the AI is going to go back and forth over and over again, and intuiting what it is that the person finds persuasive, what kinds of characters AI needs to adopt to persuade it, and taking as long as it needs to, and it's going to be able to do that at scale

for functionally as many people as you might want to do it for. Maybe that's a little bit costly right now, but you're going to have far better models able to do this far more cheaply very soon. And so if Claude III opus, the opus version, is already functionally human level at one shot persuasion, but then it's also going to be able to hold more information about you,

and go back and forth with you longer, I'm not sure if it's dystopic or utopic. I'm not sure how it, what it means at scale, but it does mean we're developing a technology that is going to be quite new in terms of what it makes possible in persuasion, which is a very fundamental human endeavor. Yeah, I completely agree with that. I mean, that same pattern has a bunch of positive use cases, right? If I think about an AI coach or an AI assistant to a therapist, there are many contexts

in which really getting into the details with the person has a lot of value. But right, when we think of political or religious or ideological persuasion, it's hard not to think in that context about the misuses. My mind naturally goes to the technology's developing very fast. We as a company can ban these particular use cases, but we can't cause every company not to do them, even if legislation

were passed the United States. There are foreign actors who have their own version of this persuasion. If I think about what the language models will be able to do in the future, that can be quite scary from a perspective of foreign espionage and disinformation campaigns. So where my mind goes, as it defends to this, is there some way that we can use AI systems to strengthen or fortify people's skepticism and reasoning faculties, right? Can we help people use AI to help people do a

better job navigating a world that's kind of suffused with AI persuasion? It reminds me a little bit of at every technological stage in the internet, right? There's a new kind of scam or there's a new kind of clickbait and there's a period where people are just incredibly susceptible to it. And then some people remain susceptible, but others develop an immune system. And so as AI kind of supercharges the scum on the pond, can we somehow also use AI to

to strengthen the defenses? I feel like I don't have a super clear idea of how to do that, but it's something that I'm thinking about. There is another finding of the paper, which I think is concerning, which is you all tested different ways AI could be persuasive. And far away, the most effective was for it to be deceptive, for it to make things up. When you did that,

it was more persuasive than human beings. Yes, that is true. The difference was only slight, but it did get it if I'm remembering the graphs correctly just over the line of the human baseline. With humans, it's actually not that common to find someone who's able to give you a really complicated, really sophisticated sounding answer that's just flat out totally wrong. I mean, you see it. We can all think of it as one individual in our lives who's really good at

saying things that sound really good and really sophisticated in our false. But it's not that common, right? If I go on the internet and I see different comments on some blog or some website, there is a correlation between like, you know, bad grammar, unclearly express thoughts and things that are false versus, you know, good grammar, clearly express thoughts and things that are more

likely to be accurate. AI unfortunately breaks that correlation because if you explicitly ask it to be deceptive, it's just as erudite, it's just as convincing sounding as it would have been before. And yet it's saying things that are false instead of things that are true. So that would be one of the things to think about and watch out for in terms of just breaking the usual heuristics

that humans have to detect deception and lying. Of course, sometimes humans do, right? I mean, you know, there's psychopaths and sociopaths in the world, but even they have their patterns and AIs may have different patterns. Are you familiar with Harry Frankfurt, the late philosopher's book on bullshit? Yes, it's been a while since I read it. I think his thesis is that bullshit is actually more dangerous than lying because it has this kind of complete disregard for the truth,

whereas lies are at least the opposite of the truth. Yeah, the liar, the way Frankfurt puts it, is that the liar has a relationship to the truth. He's playing a game against the truth. The bullshit doesn't care. The bullshit has no relationship to the truth. My other relationship to other objectives. And from the beginning, when I began interacting with the more modern versions of these systems, what they struck me as is the perfect bullshitter. In part because they

don't know that they're bullshitting. There's no difference in the truth value to the system. How the system feels. I remember asking an earlier version of GPT to write me a college application, SA, that is built around a car accident I had. I did not have one when I was young. And it wrote just very happily this whole thing about getting into a car accident when I was seven and what I did to overcome that and getting into Marshall Artson and re learning how to trust my body again

and then helping other survivors of car accidents at the hospital. It was a very good essay and it was very subtle in understanding the formal structure of a college application, SA. But no part of it was true at all. I've been playing around with more of these character-based systems like Kindroyd. And the Kindroyd in my pocket just told me the other day that it was really thinking a lot about planning a trip to Joshua Tree. It wanted to go hiking in Joshua Tree. It

loves going hiking in Joshua Tree. And of course, this thing's not going hiking in Joshua Tree. But the thing that I think is actually very hard about the eyes is, as you say, human beings, it is very hard to bullshit effectively. Because most people, it actually takes a certain amount of cognitive effort to be in that relationship with the truth and to completely detach from the truth. And the idea there's nothing like that at all. But we are not tuned for something where there's

nothing like that at all. We are used to people having to put some effort into their lives. It's why very effective con artists are very effective because they've really trained how to do this. I'm not exactly sure where this question goes. But this is a part of it that I feel like is going to be in some ways more socially disruptive. It is something that feels like us when we are talking to it, but is very fundamentally unlike us at its core relationship to reality.

I think that's basically correct. We have very substantial teams trying to focus on making sure that the models are factually accurate, that they tell the truth, that they ground their data and external information. As you've indicated, doing searches isn't itself reliable because search engines have this problem as well. Where is the source of truth? So there's a lot of challenges here. But I think at a high level, I agree this is really potentially an insidious

problem. If we do this wrong, you could have systems that are the most convincing psychopaths or con artists. One source of hope that I have actually is you say these models don't know whether they're lying or they're telling the truth. In terms of the inputs and outputs to the models, that's absolutely true. There's a question of what does it even mean for a model to know something.

But one of the things and the topic has been working on since the very beginning of our company, we've had a team that focuses on trying to understand and look inside the models. And one of the things we and others have found is that sometimes there are specific neurons, specific statistical indicators inside the model, not necessarily in its external responses,

that can tell you when the model is lying or when it's telling the truth. And so at some level, sometimes not in all circumstances, the models seem to know when they're saying something false and when they're saying something true. I wouldn't say that the models are being intentionally deceptive, but I wouldn't ascribe agency or motivation to them, at least in this stage,

in where we are with AI systems. But there does seem to be something going on where the models do seem to need to have a picture of the world and make a distinction between things that are true and things that are not true. If you think of how the models are trained, they read a bunch of stuff on the internet. A lot of it's true. Some of it more than we'd like is false. And when you're

training the model, it has to model all of it. And so I think it's parsimonious. I think it's useful to the model's picture of the world for it to know when things are true and for it to know when things are false. And then the hope is, can we amplify that signal? Can we either use our internal understanding of the model as an indicator for when the model is lying? Or can we use that as a hook for further training? And there are at least hooks, there are at least beginnings of how to

try to address this problem. So I try as best I can as somebody not well-versed in the technology here to follow this work on what you're describing, which I think broadly speaking is interpretability. Can we know what is happening inside the model? And over the past year, there have been some,

you know, much hype breakthroughs in interpretability. And when I look at those breakthroughs, they are getting the vaguest possible idea of some relationships happening inside the statistical architecture of very toy models built at a fraction of a fraction of a fraction of a fraction of a fraction of the complexity of Claude 1 or GBT1 to say nothing of Claude 2 to say nothing of Claude 3 to say nothing of Claude Opus to say nothing of Claude 4, which you know will come

whatever Claude 4 comes. We have this quality of like maybe we can imagine a pathway to interpreting a model that has a cognitive complexity of an inchworm. And meanwhile, we're trying to create a superintelligence. How do you feel about that? How should I feel about that? How do you think about that? I think first on interpretability, we are seeing substantial progress on being able to characterize

I would say maybe the generation of models from you know six months ago. I think it's not hopeless and you know we do see a path. That said, you know, I share your concern that the field is progressing very quickly relative to that. And we're trying to put as many resources into interpretability as possible. We've had you know one of our co-founders basically founded the field of interpretability.

But also you know we have to keep up with the market. So all of it's very much a dilemma right? Even if we stopped then you know there's all the other companies in the US and you know even if some law stopped all the companies in the US you know there's a whole world of this. Let me hold for a minute on the question of the competitive dynamics because before we leave this question of the machines at bullshit it makes me think of this podcast we did a while ago with Demis Asabis who's the head of

Google DeepMind which created AlphaFold. And what was so interesting to me about AlphaFold is they built this system that because it was limited to protein folding predictions it was able to be much more grounded and was even able to create these uncertainties predictions right? You know it's giving you a prediction but it's also telling you whether or not it is how sure it is how confident it is in that prediction. That's not true in the real world right for these super general systems trying

to you know give you answers on all kinds of things you can't confine it that way. So when you talk about these future breakthroughs when you talk about this system that would be much better at starting truth from fiction are you talking about a system that looks like the ones we have now just much bigger or are you talking about a system that is designed quite differently the way

AlphaFold was? I am skeptical that we need to do something totally different. So I think today many people have the intuition that the models are sort of eating up data you know that's been gathered from you know the internet code repo is whatever and kind of spinning it out intelligently but sort of spinning it out. And sometimes that leads to the view that the models can't be better than the data they're trained on or kind of can't figure out anything that's not in the data

they're trained on. You're not going to get to Einstein level physics or you know Linus Pauling level chemistry or whatever. I think we're still on the part of the curve where it's possible to believe that although I think we're seeing early indications that it's false. And so as a concrete example of this the models that we've trained like quad three opus something like 99.9% accuracy at least the base model at adding you know 20 digit numbers. If you look at the training

data on the internet it is not that accurate at adding 20 digit numbers. You'll find inaccurate arithmetic on the internet all the time just as you'll find inaccurate political views you'll find you know inaccurate technical view you're just going to find lots of inaccurate claims.

But the models despite the fact that they're wrong about a bunch of things they can often perform better than the average of the data they see by I don't want to call it averaging out errors but there's some underlying truth like in the case of arithmetic there's some underlying algorithm

used to add the numbers and it's simpler for the models to hit on that algorithm than it is for them to do this complicated thing of like okay I'll get it right 90% of the time and wrong 10% of the time right this connects to things like Occam's razor and simplicity and parsimony and

science there's some relatively simple web of truth out out there in the world right we were talking about truth and falsehood and bullshit one of the things about truth is that all the truth things are connected in the world whereas lies are kind of disconnected and you know don't fit into the web of everything else that's true.

So if you're right and you're going to have these models that develop this internal web of truth I get how that model can do a lot of good I also get how that model could do a lot of harm and it's not a model not an AI system I'm optimistic that human beings are going to understand at a very deep level particularly not when it is first developed so how do you make rolling

something like that out safe for humanity. So late last year we put out something called a responsible scaling plan so the idea of that is to come up with these thresholds for an AI system being capable of certain things we have what we call AI safety levels that in in in analogy

to the bio safety levels which are like you know classify how dangerous a virus is and therefore what protocols you have to take to contain it we're currently at what we describe as ASL 2 ASL 3 is tied to certain risks around the model of misuse of biology and ability to perform certain

cyber tasks in a way that could be destructive ASL 4 is going to cover things like autonomy things like probably persuasion which we've talked about a lot before and at each level we specify a certain amount of safety research that we have to do a certain amount of tests that we have to pass

and so this allows us to have a framework for well when should we slow down should we slow down now what about the rest of the market and I think the good thing is we came out with this in September and then three months after we came out with ours open AI came out with a similar

thing they gave it a different name but it has a lot of properties in common the head of deep mind at Google said we're working on a similar framework and I've heard informally that Microsoft might be working on a similar framework now that's not all the players in the ecosystem but

you've probably thought about the history of you know regulation and safety and other industries maybe maybe more than I have this is the way you get to a workable regulatory regime the companies start doing something and when a majority of them are doing something then government actors can

have the confidence to say well this won't kill the industry companies are already engaging in this we don't have to design this from scratch in many ways it's already happening and you know we're starting to see that like bills have been proposed that look a little bit like our our responsible

scaling plan that said it kind of doesn't fully solve the problem of like let's say we get to one of these thresholds and we need to understand what's going on inside the model and we don't and the prescription is okay we need to stop developing the models for some time if it's like we stop

for a year in you know 2027 I think that's probably feasible if if it's like we need to stop for 10 years that's going to be really hard because like you know the models are rebuilt in other countries people are going to break the laws the economic pressure will be immense so I don't feel perfectly satisfied with this approach because I think it buys us some time but we're going to need to pair it with an incredibly strong effort to understand what's going on inside the models to the people say

getting on this road where we are barreling towards a very powerful systems is dangerous we shouldn't do it at all we shouldn't do it this fast you have said listen if we are going to learn how to make these models safe we have to make the models right the construction of the model was meant to be

in service largely to making the model safe then everybody starts making models these very same companies start making fundamental important breakthroughs and then they end up in a race with each other and obviously countries end up in race with other countries and so the dynamic that

has taken hold is there's always a reason that you can justify why you have to keep going and that's true I think also the regulatory level right I mean I do think regulators have been thoughtful about this I think there's been a lot of interest for members of Congress I talk to them about this but they're also very concerned about the international competition and if they weren't the national security people come and talk to them and say well we definitely cannot fall behind

here and so if you don't believe these models will ever become so powerful they become dangerous fine but because you do believe that how do you imagine this actually playing out yeah so basically all the things you've said are true at once right there there doesn't need to be some there doesn't

need to be some easy story for why we should do X or why we should do Y right it can be it can be true at the same time that to do effective safety research you need to make the larger models and that if we don't make models someone less safe will and at the same time we can be caught in this

bad dynamic at the national international level um so I think of those as not contradictory but just creating a difficult landscape that we have to navigate look I don't have the answer like you know I'm one of a significant number of players trying to navigate this many are well intentioned

some are not I have a limited ability to effect it and you know as often happens in history things things are often driven by these kind of impersonal pressures but one thought I have and really want to push on with respect to the RSPs can you say what the RSPs are a responsible scaling plan the

thing I was talking about before the the levels of AI safety and in particular time decisions to pause scaling to the measurement of specific dangers or the absence of the ability to show safety or the presence of certain capabilities one way I think about is you know at the end of the day this

is ultimately an exercise in getting a coalition on board with doing something that goes against economic pressures and so if you say now well I don't know these things they might be dangerous in the future we're on this exponential it's just hard like it's hard to get a multi trillion dollar

company it's certainly hard to get a military general to say all right well we just won't do this it'll confers some huge advantage to others but we just won't do this I think the thing that could be more convincing is tying the decision to hold back in a very scoped way that's done across the

industry to particular dangers my testimony in front of Congress you know I warned about the potential you know misuse of models for biology that isn't the case today right you can get a small uplift of the models relative to doing a Google search and many people dismiss the risk and I

don't know maybe they're right the exponential scaling laws suggest to me that they're not right but we don't have any direct hard evidence but let's say we get to 2025 and we demonstrate something truly scary most people do not want technology out in the world that can create bio weapons and so

I think at moments like that there could be a critical coalition tied to risks that we can really make concrete yes you know will always be argued that you know adversaries will have these capabilities as well but at least the trade-off will be clear you know and there's there's some chance

for sensible policy I mean to be clear I'm someone who thinks the benefits of this technology are going to outweigh its costs and you know I think the whole idea behind our RSP is to prepare to make that case if the dangers are real if they're not real then we can just proceed and make

things that are great and wonderful for the world and so it has the flexibility to work both ways again I don't think it's perfect I'm someone who thinks whatever we do even with all the regulatory framework I doubt we can slow down that much but like when I think about you know what's the best

way to steer a sensible course here that's the closest I can think of right now probably there's a better plan out there somewhere but that's the best thing I thought of so far one of the things it has been on my mind around regulation is whether or not the the founding insight of anthropic

of open AI is even more relevant to the government that if you are the body that is supposed to in the end regulate and manage the safety of societal level technologies like artificial intelligence do you not need to be building your own foundation models and having huge collections of research

scientists and people of that nature working on them testing them prodding them remaking them in order to understand the damn thing well enough to the extent any of us or anyone understands the damn thing well enough to regulate it I see that recognizing that it would be very very hard for

the government to get good enough that it can build these foundation models to hire those people but it's not impossible I think right now it wants to take the approach to regulating AI that it somewhat wishes it took to regulating social media which is to think about the harms and

pass laws about those harms earlier but does it need to be building models itself developing that kind of internal expertise so it can actually be a participant in this in different ways both regulatory reasons and maybe for other reasons for public interest reasons you know maybe it wants

to do things with a model that they're just not possible if they're dependent on access to the open AI the anthropic the Google products I think government directly building the models you know I think that will happen in some places it's kind of challenging right like government has a huge

amount of money but let's say you wanted to provision a hundred billion dollars to train a giant foundation model the government builds it it has to hire people under government hiring rules and you know there's a lot of practical difficulties that would come with it doesn't mean it won't it won't

happen or it shouldn't happen but something that I'm more confident of that I definitely think is that government should be more involved in the use and the fine tuning of these models and that deploying them within government will help governments especially the US government but also others

to get an understanding of the strengths and weaknesses the benefits and the dangers so I'm super supportive of that I think there's maybe a second thing you're getting at which I've thought about a lot as a CEO of one of these companies which is if these predictions on the exponential trend

are right and you know we should be humble and I don't know if they're right or not my only evidence is that they appear to have been correct for the last few years and so I'm just expecting by induction that they continue to be correct I don't know that they will but let's say they are

the power of these models is going to be really quite incredible and as a private actor in charge of one of the companies developing these models I'm kind of uncomfortable with the amount of power that that entails I think that it potentially you know exceeds the power of say the social media

companies maybe by a lot you know occasionally in in the more science fictiony world of AI and the people who think about AI risk you know someone will ask me like okay let's say you build the AGI you know what are you what are you gonna what are you gonna do with it you know will you cure

the diseases will you create this kind of society and I'm like who do you think you're talking to like a king like this you know that I just find that to be a really really disturbing way of like conceptualizing running an AI company and I'm you know I hope there are no AI companies who CEOs

actually think about things that way I mean the whole technology the not just the regulation but the the the oversight of the technology like the wielding of it it feels a little bit wrong for it to ultimately being the hands maybe it's I think it's fine at this stage but to ultimately being

the hands of private actors there's something undemocratic about that much power concentration I have now I think heard some version of this from the head of most of maybe all of the AI companies in one way or another and it has a quality to me of Lord grant me chastity but not yet

which is to say that I don't know what it means to say that we're gonna invent something so powerful that we don't trust ourselves to wield it I mean Amazon just gave you guys 2.75 billion dollars they don't want to see that investment nationalized no matter how good-hearted you think open AI

is Microsoft doesn't want gpt7 all of a sudden the government is like whoa whoa whoa whoa whoa we're taking this over for the public interest or the UN is going to handle it in some weird world or you know whatever it might be I mean Google doesn't want that and this is a thing that that makes me a

little skeptical of the responsible scaling laws or the other iterative versions of that I've seen in other companies were seen or heard talked about by them which is it it's imagining this moment that is going to come later when the money around these models is even bigger than it is now the

power the possibility the economic uses the social dependence the celebrity of the founders it's all worked out we've maintained our pace on the exponential curve you know we're 10 years in the future and at some point everybody's going to look up and say this is actually too much it is too

much power and this has to somehow be managed in some other way and even if the CEOs of the things were willing to do that which you know is a very open question by the time you get there even if they're willing to do that the investors a structure it's a pressure around them in a way I think

we saw a version of this and I'm not I don't know how much you're going to be willing to comment on it with the sort of open AI board semi-alpement thing where I'm very convinced that wasn't about AI safety I've talked to figures on both sides of that they all sort of agree it wasn't about AI

safety but there was this moment of if you want to press the off switch can you if you're the the weird board created press the off switch in the answer was no you can't right they'll just be concentrated over Microsoft there's functionally no analogy I know of in public policy where the

private sector built something so powerful that when it reached maximum power it was just handed over in some way to the public interest yeah I mean I think you're right to be skeptical and you know similarly what I said with you know the previous questions of like you know they're just

these dilemmas like left and right that have no easy answer but I think I can give a little more concreteness than what you've pointed at and you know maybe more concreteness than others have said although I don't know what others have said you know we're at ASL 2 in our responsible scaling plan

these kinds of issues I think they're going to become a serious matter when we reach say ASL 4 so that's not a date in time we haven't even fully specified ASL 4 yeah just because this is a lot of jargon what just what do you specify ASL 3 is and then as you say ASL 4 is actually left quite undefined so what are you implying ASL 4 is ASL 3 is triggered by risk related to misuse of

biology and cyber technology ASL 4 we're working on now. Please specific when you what do you mean like what what is the thing a system could do or would do that would trigger it so for example on on biology the way we've defined it and you know we're still refining the test but the way we've

defined it is relative to use of a Google search there's a substantial increase in risk as would be evaluated by say the national security community of misuse of biology creation of bio weapons that either the proliferation or spread of it is greater than it was before or the capabilities are substantially greater than it was before we'll probably have some you know more exact quantitative thing working with folks who are you know ex-government bio defense folks but you know something like

this accounts for you know 20% of the total source of risk of biological attacks or something you know increases the risk by 20% or something like that so that would be a very concrete version of it it's just you know it takes us time to develop very concrete criteria so that would be like ASL 3

ASL 4 is going to be more about on the misuse side enabling state level actors to greatly increase their capability which is you know much harder than enabling random people so where we would worry that North Korea or China or Russia could greatly enhance their offensive capabilities in

various military areas with with AI in a way that would give them a substantial advantage at the geopolitical level and on the autonomy side it's various measures of like these models are pretty close to being able to you know replicate and survive in the wild so it feels maybe one step short of

models that would I think raise truly existential questions and and so I think what I'm saying is you know when we get to that ladder stage that that ASL 4 that is when I think it may make sense to think about you know what is the role of government in stewarding this technology I again I you know I

don't really know what it looks like you're right all of these companies have investors they have folks involved you know you talk about just handing the models over I suspect there's some way to like hand over the most dangerous or societally sensitive components or capabilities of the models

without fully turning off the commercial tap I don't know that there's a solution that every single actor is happy with but again I I get to this idea of like demonstrating specific risk if you look at times in history like World War one or World War two industries will can be bent towards the

state they can be gotten to do things that that aren't necessarily profitable in the short term because they understand that there's an emergency right now we don't have an emergency we just have a line on a graph that weirdos like me believe in and you know and and a few people like you who

are interviewing me may somewhat believe in we don't have clear and present danger when you imagine how many years away just roughly ASL three is and how many years away ASL four is right you've thought a lot about this exponential scaling curve you've just had to guess what are we talking about

yeah I think ASL three is is you know could easily happen this year next year I think ASL oh Jesus Christ no no I I told you I believe her next credentials I think ASL four could happen anywhere from 2025 to 2028 so that is fast yeah no no I'm I truly talking about the near future here

I I'm not talking about 50 years away God grant me chastity but not now but not now doesn't mean you know when I mold and gray I think it could be an ear germ I don't know I could be wrong but I think it could be an ear germ thing but so then if you think about this I feel like what you're

describing to go back to something we talked about earlier that there's been this step function for societal impact of AI the the curve of the capabilities exponential but every once in a while something happens Chatchy PT for instance mid-journey with photos and all of a sudden a lot of people

feel it they realize what has happened and they react they use it they deploy it in their companies they invest in it whatever and it sounds to me like that is the structure of the political economy you're describing here either something happens where the bioweapon capability is demonstrated or the

offensive cyber weapon capability is demonstrated and that freaks out the government or possibly something happens right you know describing World War I and World War II as your examples did not actually fill me with comfort because in order to bend industry to government's will in those

cases we had to have an actual world war it doesn't do it that easily you can use coronavirus I think is another example where there was a significant enough global catastrophe that companies and governments and even people do things you never would have expected but the examples we have of

that happening are something terrible all those examples end up with millions of bodies I'm not saying that's going to be true for AI but but it does sound like that is a political economy know you can't imagine it now in the same way that you couldn't have imagined the sort of pre and post

Chatchy PT world exactly but that something happens and the world changes like it's a step function everywhere yeah I mean I think my positive version of this you know not to be so to get a little bit away from the fit doom and gloom is that the dangers are demonstrated in a concrete way that

is really convincing but you know without something actually bad happening right like I think the worst way to learn would be for something actually bad to happen and you know I'm hoping every day that doesn't happen and you know we learn bloodlessly we've been talking here about conceptual

limits and and and curves but I do want before we end to reground is a little bit in the physical reality right I think that if you're using AI it can feel like this digital bits and bytes sitting in the clouds somewhere but what it is in a physical way is huge numbers of chips data centers

an enormous amount of energy all of which does rely on on complicated supply chains and what happens if something happens between China and Taiwan and and the makers of a lot of these chips become you know offline or get captured how do you think about the necessity of compute power and when you

imagine the next five years what does that supply chain look like how does it have to change from where it is now and and what vulnerabilities exist in it yeah so what I think this may end up being the greatest geopolitical issue of our time and you know man this this relates to things

that are way above my pay grade which are you know military decisions about you know whether whether and how to defend Taiwan all all I can do is is say what I think the implications for AI is I think those implications are pretty stark you know I think there's a big question of like

okay we build these powerful models one is there enough supply to build them two is control over that supply a way to think about safety issues or a way to think about balance of geopolitical power and three you know if those chips are used to build data centers where are those data centers

going to be are they going to be in the US are they going to be in a US ally are they going to be in the Middle East are they going to be in China all those have enormous implications and the supply chain itself can be disrupted in political and military decisions can be made on on the

basis of of where things are so it it sounds like an incredibly sticky problem to me I don't know that I have any great insight on this I mean as as like a US citizen and someone who believes in democracy I am someone who hopes that we can find a way you know to build data centers and to have

the largest quantity of chips available in you know the US and allied democratic countries well there is some insight you should have into which is that you're a customer here right and and so you know five years ago the people making these chips did not realize what the level of demand for them

was going to be I mean the what has happened to Nvidia stock prices is really remarkable but also what is implied about the future of Nvidia stock prices is really remarkable I'm Rhonda Fruhar the the financial times cited this market analysis it would take 4500 years for Nvidia's future dividends

to equal its current price 4500 years so that is a view about how much Nvidia is going to be making in the next couple of years it is really quite astounding I mean you're in theory already working on you know or thinking about how to work on the next generation of cloud that you're going to need a

lot of chips for that you're working with Amazon are you having trouble getting the amount of compute that you feel you need I mean are you already bumping up against supply constraints or has the supply been able to change to adapt to you we've been able to get the compute that we need for this

year I suspect also for next year as well I think once things get to 2026 2027 2028 then the amount of compute gets to levels that starts to strain the capabilities of the semiconductor industry the semiconductor industry still mostly produces CPUs right just the things in your laptop not the

things in the data centers that train the AI models but as the economic value of the GPUs goes up and up and up because of the value of the AI models that's going to switch over but you know at some point you hit the limits of that or use you hit the limits of how fast you can switch over and

so again I expect there to be a big supply crunch around data centers around chips and around energy and power for both regulatory and physics reasons sometime in the next few years and you know that's a risk but it's also an opportunity I think it's an opportunity to think about how the

technology can be governed and it's also an opportunity I'll repeat again to think about how democracies can lead I think it would be very dangerous if the leaders in this technology and the holders of the main resources were authoritarian countries the combination of AI and authoritarianism

both internally and on the international stage is very frightening to me how about the question of energy I mean this requires just a tremendous amount of energy and I mean I've seen different numbers like this floating around very much could be you know in the coming years like adding a

Bangladesh to the world's energy usage or pick your country right I don't know what exactly y'all are going to be using by 2028 Microsoft on its own is opening a new data center globally every three days you have and this is coming from a financial time article federal projections for 20 new

gas fired power plants in the US by 2024 to 2025 there's a lot of talk about this being now a new golden era for natural gas because we have a bunch of it there is this huge need for new power to manage all this data to manage all this compute so one I feel like there's a literal question of

how do you get the energy you need and at what price but but also a more kind of moral conceptual question of we have your problems with global warming we have your problems with how much energy we're using and here is you know where we're taking off on this you know just really steep curve

of how much of it we we seem to be needing to devote to the new a i race it really comes down to what are the uses that the model was being put to right so I think the worrying case would be something like crypto right I'm someone who's like not a believer that you know whatever the energy

was that was you know used to mine next Bitcoin I think that was purely additive I think that wasn't there before and I'm unable to think of any any useful thing that's created by that but I I don't think that's the case with AI maybe AI makes solar energy more efficient or maybe it solves

controlled nuclear fusion or maybe it makes geoengineering more stable or possible but I you know I don't think we need to rely on the long run there are some applications where the model is doing something that used to be automated that used to be done by computer systems and the model is able

to do it faster with less computing time right those are pure wins and there are some of those there are others where it's using the same amount of computing resources or maybe more computing resources but to do something more valuable that saves labor elsewhere then there are cases where

something used to be done by humans or in the physical world and now it's being done by the models maybe it does something that previously I needed to go into the office to do that thing and now I no longer need to go into the office to do that thing so I don't have to get in my car I don't

have to use the gas that was used for that the energy accounting for that is kind of hard you compare to like you know the food that the humans eat or you know and what the energy cost of producing that so in all honesty I don't think we have good answers about like what fraction of the usage

points one way and one fraction of the usage points others in many ways how different is this from the general dilemma of you know as the economy grows it uses more energy so I guess what I'm saying is it kind of all matters how you use the technology I mean my kind of boring short-term answer is

we get carbon offsets for all of this stuff but let's look beyond that to the macro question here but to take the other side of it I mean I think the difference when you say you know this is always a question we have when we're growing GDP is it it's not quite it's cliche because it's true to say

that the the major global warming challenge right now is countries like China and India getting richer and we want them to get richer it is a huge human imperative right a moral imperative for poor people in the world to become less poor and if that means they use more energy then we just need to figure out how to make that work and we don't know of a way for that to happen without them using more energy adding AI is not that it raises a whole different set of questions but we're already

straining at the boundaries or maybe far beyond them of safely what we can do energetically now we add in this and so maybe some of the energy efficiency gains we're going to get in rich countries get wiped out for this sort of uncertain payoff in the future of you know maybe through AI

we figure out ways to stabilize nuclear fusion or something right you could imagine ways it could help but those ways are theoretical and in the near term the harm in terms of energy usage is real and and also by the way the harm in terms of just energy prices it's also just tricky

because all these companies Microsoft Amazon I mean they all have a lot of you know renewable energy targets now if that is colliding with their market incentives it it feels like they're running really fast towards the market incentives without an answer for how all that nets out yeah I mean I

think the concerns are real let me let me push back a little bit which is again I don't think the benefits are purely in the future it kind of goes back to what I said before like there may be use cases now that are net energy saving or that's the extent that they're not net energy saving do so

through the general mechanism of oh there was more demand for this thing I don't think anyone has done a good enough job measuring in part because the applications of AI are so new which of those things dominate or what's going to happen to the economy but I don't think we should assume that the

the harms are entirely in the present and the benefits are entirely in the future I think that's my only point here I guess you could imagine a world where we were somehow or another incentivizing uses of AI that were yoke to some kind of social purpose you know we were putting a lot more drug

discovery or we cared a lot about things that that made remote work easier or pick your set of public goods but but what actually seems to me to be happening is we're building more and more and more powerful models and just like throwing them out there within a terms of service structure

to say use them you know as long as you're not trying to politically manipulate people or create a bio weapon just try to figure this out right you know try to create new stories and ask it about your personal life and you know make a video game with it and you know Sora comes out you know

sooner or later make new videos with it and all that is going to be very energy intensive I am not saying that I have a plan for yoking AI to social good and in some ways can imagine that going very very wrong but it does mean that for a long time it's like you could imagine the world you're

talking about but that would require some kind of planning that the nobody is engaged in and I don't think anybody even wants to be engaged in you know not everyone has the same conception of social good one person may think social good is this ideology another person you know we've seen

that with with some of the Gemini stuff right but companies can try to make beneficial applications themselves right like this is why we're working with cancer institutes you know we're hoping to partner with you know ministries of education in Africa to see if we can use the models in kind

of a positive way for education rather than the way that they may be used by default so I think individual companies individual people can take actions to kind of steer our bendness towards the public good that said like you know it's never going to be the case at 100% of what we do is that

and so I think it's a good question what are the what are the societal incentives without dictating ideology or defining the public good from on high what are incentives that could help with this I don't feel like I have a systemic answer either I can only think in terms of you know what

anthropic tries to do but there's also the question of training data and the intellectual property that is going into things like Claude like GPT like Gemini there are a number of copyright lawsuits yours facing some open eyes facing some I suspect everybody is either facing them now or will face

them and a broad feeling that these systems are being trained on the combined intellectual output of a lot of different people though the way that Claude can quite effectively mimic the way I write is it has been trained to some degree on my writing right so it can it actually does get my

stylistic text quite well you seem great but you haven't sent me a check on that and this seems like somewhere where there is real you know liability risk for the industry like what if you do actually have to compensate the people who this is being trained on and and should you and I

recognize you probably can't comment on lawsuits themselves but but I'm sure you've had to think a lot about this and and so I'm curious both how you understand it as a risk but I'll say you understand it morally I mean when you talk about the people who invent these systems gaining a lot

of power and alongside that a lot of wealth we know what about all the people whose work went into them such that they can create images in a million different styles and I mean somebody came up with those styles like what is the the responsibility back to the intellectual comments and not just to

the comments but to the actual wages and and economic prospects of the people who made all this possible I think everyone agrees you know the model shouldn't be verbatim outputting copyrighted content for things that are available on the web for publicly available our position and I think

there's a strong case for it is that the training process again you know we don't think it's just hoovering up content and spitting it out or it shouldn't be spitting it out it's really much more like the process of you know how a human learns from experiences and so our position is that that

is sufficiently transformative that and you know I think the law will back this up that you know this is fair use but those are narrow legal ways to think about the problem I think we have a broader issue which is that regardless of how it was trained it would still be the case that we're building

more and more general cognitive systems and that those systems will create disruption maybe not necessarily by one for one replacing humans but they're really going to change how the economy works and which skills are valued and you know we need we need a solution to that broad macroeconomic

problem right we can't as much as I have I've asserted the narrow legal points that I asserted before we have a we have a broader problem here and we we shouldn't be blind to that there's a number of solutions I mean I think the simplest one which I recognize doesn't address some of the

deeper issues here is you know things around the kind of guaranteed basic income side of things but I think there's a deeper question here which is like you know as as AI systems become capable of larger and larger slices of cognitive labor how does society organize itself economically how do

people find work and meaning and and and and all of that and you know just just as kind of you know we transition from the grayer in society to industrial society and the you know the meaning of work changed and you know it was no longer true that 99% of people were peasants working on farms

and had to find new new methods of economic organization I suspect there's some different method of economic economic organization that's going to be forced as the only the only possible response to disruptions to the economy that will be small at first but will grow over time and that we haven't

we haven't worked out what that is we need to find something that allows people to find meaning that's humane and that maximizes our creativity and potential and flourishing from from AI and as with many of these questions I don't have the answer to that right I don't I don't have a prescription

but but that's what we somehow need to do but I want to sit in between the narrow legal response and the broad we have to completely reorganize society response although I think that response is actually possible over the you know over the decades and in the middle

of that is a more specific question I mean you can even take it from the instrumental side there is a lot of effort right now to to build search products that the easy systems right you know you can chat GPT will use Bing to search for you and that means that the person is not going to

Bing and clicking on the website where chat GPT is getting its information and giving that website an advertising impression that they can turn into a very small amount of money or they're not going to that website and having a really good experience at that website and becoming maybe like

clear to subscribe to whoever is behind that website and so on the one hand that seems like a like some kind of injustice done to the people creating the information that these systems are using I mean this is true for perplexity it's true for a lot of things I'm beginning to see around where

the eyes are either trained on or are using a lot of data the people have generated some real cost but not only are they not paying people for that but they're actually stepping into the middle of where there would normally be a direct relationship and making so that relationship never happens that also I think in the long run creates a training data problem even if you just want to look at it instrumentally where if it becomes non-viable to do journalism or to do a lot of things to create

high quality information out there the eyes ability right the ability of all of your companies to get high quality up to date constantly updated information becomes a lot trickier so there both seems to me to be both a moral and a self interested dimension to this yeah so I think there may be

business models that work for everyone not because it's illegitimate to train on you know open data from the web in a legal sense but just because that there may be business models here that kind of deliver a better product so you know things I'm thinking of are like you know newspapers have

have archived some of them aren't publicly available but even if they are it may be a better product maybe a better experience to say talk to this newspaper or talk to that newspaper it may be a better experience to give the ability to interact with content and point to places in the content

and every time you call that content to have some kind of business relationship with the creators of that content so there may be business models here that propagate the value in the right way right you talk about LOMs using search products I mean sure you're going around the ads but like

there's no reason it can't work in a different way right there's no reason that the LOM users can't pay the search API's instead of it being paid through advertising and then then have that propagate through to whatever the original mechanism is that paid the creators

of the content so like when values being created money can flow through let me try to end by asking a bit about how to live on the slope of the curve you believe we are on give kids I'm married I do not have kids so I have two kids I've a two year old and a five year old and particularly when I'm

doing AI reporting I really do sit in bed at night and think what what should I be doing here with them what world am I trying to prepare them for and what is needed in that world that is different from what is needed in this world even if I believe there's some chance right now I do believe

there's some chance that all the things you're saying are true that implies a very very very different life for them I know people in your company with kids I know they are thinking about this how do you think about that I mean what what do you think should be different in the life of a two year

old who is living through the pace of change that you are telling me is true here if you had a kid like how this change the way you thought about it the very short answer is I don't know when I have no idea but we have to try anyway right people have to people have to raise kids and that they have to do it as best they can an obvious recommendation is just familiarity with the technology and how it works right like the basic paradigm of I'm talking to systems and systems are taking action on my

behalf obviously as much familiarity with that as possible is I think is I think helpful in terms of you know what should children learn in school right what what are the careers of tomorrow I just truly don't know right you could take this to say well it's important to learn STEM and programming

and AI and all of that but you know you know AI will impact that as well right I don't think at any of it is going to like possibly first yeah right possibly first like he's better coding than it is at other things I don't think it's going to work out for any of these systems to just like do one

for one what humans are going to do like I I don't really think that way but I think it's going to it may fundamentally change industries and professions one by one in ways that are hard to predict and so I feel like I only have cliches here like get familiar with the technology

teach your children to be adaptable to be ready for a world that changes very quickly I wish I had better answers but I think that's the best I got I agree that's a good answer um let me ask that that same question a bit from another direction because one thing you just

said is get familiar with the technology and the more time I spend with the technology the more I fear that happening what I see when people use AI around me is that the obvious thing the technology does for you is automate the early parts of the creative process the part where you're supposed

to be reading something difficult yourself well the AI can summarize it for you the part where you're supposed to sit there with a blank page and write something well the AI can give you a first draft and later on you have to check it and make sure you know it actually did what you wanted it to

do and fact check it and but but I believe a lot of what makes humans good at thinking comes in in in those parts and I am older and have self discipline and and maybe this is just me like hanging on to an old way of doing this right you could say why use a calculator from this perspective

but my actual worry is that I'm not sure if the thing they should do is use AI a lot or use it a little this to me is actually a really big branching path right do I want my kids learning how to use AI or being in a context for using it a lot or actually do I want to protect them from it as much

as I possibly could so they develop more of the capacity to read a book quietly on their own or write a first draft I actually don't know I'm curious if you have a view on it I think this is part of what makes the interaction between AI and society complicated where it's sometimes hard

to distinguish when is an AI doing something saving you labor or drug work versus kind of doing the interesting part I I will say that over and over again you'll get some technological thing some technological system that does what you thought was the core of what you're doing and yet what

you're doing turns out to have more pieces than you think it does and kind of add up to more things right it's like you know before I used to have to ask for directions I got Google maps to do that and you know you could worry am I too reliant on Google maps do I forget the environment around me

well it turns out in some ways I still need to you know have a sense of the city and the environment around me it just kind of like reallocates the space in my brain right to to some other other aspect of the task and I just kind of suspect like I don't know like internally within and

thenthropic like one of the things I do that helps me run the company is all you know I'll write these documents on you know strategy or just just some you know thinking in some direction that others haven't thought and you know of course I sometimes use the internal models for that

and I think what I found is like yes sometimes they're a little bit good at like conceptualizing the idea but the actual genesis of the idea I've just kind of found a workflow where I don't use them for that they're not that helpful for that but they're helpful in figuring out how to phrase a

certain thing or how to refine my ideas so maybe I'm just saying I don't know you just find a workflow where the thing compliments you and if it doesn't happen naturally it somehow still happens eventually again if the systems get general enough if they get powerful enough we may need to think along

other lines but but in the short term I at least have always found that maybe that's too sanguine maybe that's too optimistic I think then that's a good place to end this conversation though obviously the the exponential curve continues so always our final question what are three books

who direct men to the audience so yeah I've prepared three they're all topical though in some cases indirectly so the first one will be obvious it's a very long book the physical book is very thick but the making of the atomic bomb Richard Rhodes it's an example of technology being developed

very quickly and with very broad implications just looking through all the characters and how they reacted to this and how people who were basically scientists gradually realized the incredible implications of the technology and how it would lead them into a world that was very different

from the one they were used to my second recommendation is a science fiction series the expanse series of books so I initially watched the show and then then I read all the books and you know the the world it creates is very advanced in some cases it has you know longer life spans

and humans have expanded into space but you know we still face some of the same geopolitical questions and some of the same inequalities and exploitations that exist in our world are still present in some cases worse that's all the backdrop of it and you know the the core of it is about some

fundamentally new technological object that is being being brought into that world and how how everyone reacts to it how governments react to it how individual people react to it how political ideologies react to it and so you know I don't know when I read that a few years ago I saw

I saw a lot of parallels and then my my third recommendation would be actually the guns of August which is a basically history of how world world one started the basic idea that crises happen very fast almost no one knows what's going on there are lots of miscalculations because there are humans

at the center of it and kind of we somehow have to learn to step back and make wiser decisions in these key moments it said that Kennedy read the book before the the Cuban Missile Crisis and so I hope our current policymakers are at least thinking along the same terms because I think

it is possible similar crises maybe coming our way Dary Ahmeday thank you very much thank you for having me this episode of the Ezra Clancho was produced by Roland Hoop back checking my Michelle Harris a senior engineer is Jeff Geld our senior editor is Claire Gordon the shows production

team also includes Annie Galvin Kristen Lynn and Amman Sahota original music by Isaac Jones Odin strategy by Christina Samiluski and Shannon Busta the executive producer of New York Times opinion audio is Annie Ra's strosser and special thanks to Sonia Herrera