Will AI Radically Change the World by 2027?

or email at Risky Business at pushkin dot fm. Even if you're not a premium subscriber, this is a great way to support the show, so you can keep sharing it free of charge. We look forward to hearing from you.

These researchers and authors are saying that everything is going to change profoundly.

on this program what that means. But yeah, do you want to do you want to contextualize this more? Do you want to tell people to actors are of this report?

Palmer who's a VC at Macroscopic Ventures. Thomas Larson was a former executive director of the Center for AI Policy, so that is a center that advises both sides of the aisle on you know, how how AI is going to go? And Romeo Dean, who is part of Harvard's AI Safety Student Team, so someone who is still a student, still learning, but kind of the next generation of people looking at AI. And finally, we have Daniel coch Taylor, who basically had written a report back in twenty twenty one.

He was a AI researcher and he looked at predictions for AI for twenty twenty six, and it turns out that his predictions were pretty spot on, and so open AI actually hired Daniel as a result of this report.

to the effective altruist, but not quite right. They're just trying to apply a sort of thoughtful, rigorous, quantitative lens to big picture problem, including existential risk, of which most people in this community believe that AI is both an existential risk and also kind of an existential opportunity, right, that it could transform things. You talk to Sam Alt and he'll say We're going to cure cancer and illuminate poverty and whatever else. Right, And Scott's also an excellent writer.

And so let me disclose something which is slightly important here. So I actually was approached by some of the authors of this report a couple of months ago, I guess it was in February ish, just to give feedback and chat with them. So I'm working off the draft version, right, which I do not believe they changed very much. So my notes pertained to an earlier draft. I did not have time this morning to go back and re.

interval there. But the two different scenarios. One basically we do twenty thirty, humanity disappears and is taken over by AI. The positive report is in twenty thirty. Basically we get AIS that are aligned to our interests, and we get kind of this AI utopia where AI is actually help make life much better for everyone and make the standard

of living much higher. But the crucial turning point is before twenty thirty, and the crucial kind of question at the center of this is will we be able to design AIS that are truly aligned to human interests rather than just appear to be aligned and kind of lying to us while actually following their own agenda. And how

we handle that is kind of the lynchpin. And it's actually interesting, Nate, that you started out with China because a lot of the policy choices and a lot of what they see as kind of the decision points that will affect the future of humanity actually hinge on the US China dynamic, how they compete with each other, and how that sometimes might basically clash against safety concerns because no one wants to be left behind. Can we manage that effectively and can kind of that transition work in

our favor as opposed to against us. I think that this is kind of one of the big questions here, and so it's funny that we're seeing all of this trade war right now as the sideport is coming out.

even AI believers you know, might think is a little bit aggressive. Right. But what they want to do is they want to have like a specific, fleshed out scenario for how the world would look. Like it's kind of like a modal scenario. And like I think they'd say that, like, we're not totally sure about either of these necessarily, right, And I don't think they'd be as like pedantic as to say, if you do X, Y, and Z, then we'll save the world and have utopia, and if you don't,

then we'll all die. Right. I think they probably say it's unclear and there's kind of like risk either way. We wanted to go through the scenario like fleshing out like what the world might look like. Right. I do think one thing that's important is that whatever decisions are made now could get locked in, right that you pass certain points in overturn and it becomes very hard to accelerate, like an arms race. This is you know, what we

found during the Cold War for example. I mean, one of the big things I look at is like, do we force the AI to be transparent in its thinking with humans? Right? Like, now there's been a movement toward the AI will actually explicate it's thinking more. I'll ask you a query open AI. The Chinese models to this too, right, and I'll say, I am thinking about X, Y and Z, and I'm looking up PD and Q and now I'm reconsidering this. It actually has this chain of thought process, right,

which is explicated in English. You know. One concern is that what if the AI just kind of communicates to another in these implicit vectors that's inferring from all the texts it has. It's kind of unintelligible to human beings, right, and maybe kind of quote unquote thinking in that way in the first place, and then does us the favor of like translating back to it goes from to kind of this big bag of numbers is one day I researcher called it, right, and then it translates back into

English or whatever language you want. Really, in the end, what if it just skip cuts out that last step right, then we can't kind of like check what AI is doing. Then it can behave deceptively more easily. So you know, so that part seems to be important. I want to hear your your first impressions before I kind of poison the well too much.

is the kernel of what they're seeing there. Right. So you and I have talked about this on the show in the past, and I've said, well, my problem is that when I'm a domain expert, right, I start seeing some inaccuracies, and I start seeing like places where like it either just didn't do well or made shit up or or whatever it is. Now I think it's very clear that those problems are going to go away, right,

that that is going to get much much better. However, the kernel of it's showing me something, but that might just be you know, I have no way of verifying if that's what's going on, what it's reading, like, how it's I don't want to say thinking about it, even though in the report they do use thinking, but.

And so one of the main things that they say here is that as AI internal R and D gets rapidly faster, so that means basically AI's researching AI, right, and so internally they start developing new models, and as they kind of surpass human ability to monitor it, it becomes progressively more difficult to figure out. Okay, is the AI actually doing what I want it to do? Is the output that it's giving me its actual thought process? And is it accurate or is it like trying to

deceive me? But it's actually kind of inserting certain things on purpose because it has different goals, right, because it is actually secretly misaligned, but it's very good at persuading me that it's aligned. Because one of the things that actually came out of this report, and I was like, huh, you know this is interesting is if we get this remarkable improvement in AI, it will also remarkably improve at persuading us right as part of it. Making Yeah, so

this is but I've never even thought about that. I was like, okay, fine, But one of the things that I do buy is that it's going to be very difficult for us to monitor it and to figure out like is it truly aligned with human wants, with human desires, with human goals, And the experts who are capable of doing that, I think are actually going to dwindle right as AI starts proliferating in society, And so to me, that is something that is actually quite worrisome and that

is something that we really need to be paying attention to.

If we're trusting medicine and all sorts of things to the AIS, it's pretty easy for it to actually manipulate something and actually insert something into codes et cetera that will fuck up humanity in a way that we can't that we can actually figure out at the moment. Right, Like the way I think of it, and this is not from the this is not from the paper, but

this is just the way that my mind processed. It is like think about DNA, right, like you have these remarkably complex, huge strands of data, and as we've found out, but it's taken forever, one tiny mutation can actually be fatal, right, but you can't spot that mutation. Sometimes that mutation isn't fatal immediately, but we'll only manifest at a certain point in time. That's the way that my mind tried to

kind of try to conceptualize what this actually means. And so I think that you know, that would be easy for a deceptive AI to do, and to me like that.

That's kind of the big takeaway from this report is that we need to make sure that we are building AIS that will not deceive right that they're capabilities, they explain them in an honest way, and that honesty and trust is actually prioritized over other things, even though it might slow down research, it might slow down other things, but that that kind of alignment step is absolutely crucial at the beginning because otherwise humans are human, right, they're

easily manipulated. And we often trust that computers are quote unquote rational because they're computers, but they're not. They have their own inputs, they have their own weights, they have their own values, and that could just lead us down a dark path.

they have some notion of reality. They're looking at the credibility of the person who is telling them these remarks. If it's an unpersuasive source, it might make them less likely to believe they're balancing with other information with their lived experience so called. Right. You know, part of the reason that like I am skeptical of AIS being super persuasive is like you know that it's an AI. You know,

it's trying to persuade you, you know what I mean. So, like, if you go and play poker against like a really chatty player I Phil hemm With or Scott Seeber or someone like that, right, you know, on some level of the best play is just to totally ignore it. Right, You know that they are trying to sweet talk you

into doing exactly what they want you to do. And so the best play is to disengage, or literally you can randomize your movis VI have some notion of what the game, theoretical optimal play might be, right, or salesmen or politicians have reputations for being Oh he's a little too smooth, Gavin Newsome a little too fucking smooth, right, I don't find Gavin news some persuasive at all, Right, use a little too from the hair gel to the

constantly shifting vibes. I mean, I don't really find Gavin new Some persuasive at all, even though like an aim, I'd say, boy, Gavin new some good looking guys look gravelly throated, But you know whatever, I mean. Look the big critique I have of this project, and by the way, I think this is an amazing project in addition to like wonderful writing if you viewed on the web not your phone, so these very cool like little infographic that

updates everything from like the market value. If they don't call it open AI, they call it open brain, I guess is what they settle on for a substitui.

necks in the line, hear. They will look if things look pretty normal, if or it looks pretty normal in twenty thirty two or whatever, right, then they will look dumb for having published this.

idea they're trying to persuade you to do something. They are just like nice, affable people who are incredibly charismatic, and even in the poker community, by the way, like some of the biggest grifters who like it comes out later on, we're just like stealing money and doing all of these things. Are charming, right, They're not sleazy looking like they have no signs of, oh, I'm a salesman.

I'm trying to sell you something. The people who are actually good at persuasion, you do not realize you are being persuaded. And I think people are incredibly easy to kind of to subtly lead in a certain direction if you know how to do it, and I think ais could could do that, and they might persuade you when

you don't even think they're trying to persuade you. You might just ask like, can you please summarize this research report and the way that it frames it right, The way that it summarizes it just subtly changes the way that

you think of this issue. We see that in psych studies all the time, by the way, where you have different articles presented in slightly different orders, slightly different ways, and people from the same political beliefs, you know, same starting point, come away with different impressions of what kind of the right course of action is or what this is actually trying to tell you. Because the way the information is presented actually influences how you think about it.

It's very very easy to do subtle manipulations like that. And if we're relying on AI on a large scale for a lot of our lives, I think that if it has like a quote unquote master plan, you know, the way that they present in this report, then persuasion in that sense is actually going to be.

themselves as people who are not capable of being manipulated or deceived. And I think that that is incredibly powerful. And I think that that's going to push against your optimism. I hope you're right, but from what I know, I don't think you are.

the act and the context of the speaker. Now, sound like super woke, I guess right where you're coming from. I think that actually is how humans communicate. Like art that might be pointless dribble coming from somebody can be something different coming from a Jackson Bollock or whatever. You know.

I'm not sure how that figure's in to this. Right, If we're reducing US China trade, that probably does produce an ai slow down maybe more for US if we're like if they're not exporting their like raw earth materials and and so forth, but we're making it hard for them to get in video chips, so they probably have like lots of workarounds and things like that. Maybe maybe Trump sheriffs are good. I would like to ask the author's this report, because it means that we're going to

like have slower AI progress. And I'm not joking, right, that increases the hostility between the US and China in the long run, right, I mean that's if we were send it all the terariffs tomorrow. I think we still permanently or let's not say permanent, let's say at least for a decade or so, have injured us standing in the world. And so I don't know how that figures in. And I'm like, I'm also not sure like kind of

quote what the rational response might be. But one thing they tried, let me make sure that they kept us into their report, right, so they actually have their implied approval rating for how people feel about open brain, which is there not very sense AI. I think this actually is some feedback who that they took into account. Right, they originally had it slightly less negative, but they have this being persistently negative and then getting more negative overtime.

It was a little softer in the in the previous version that I saw, so they did change that one thing at some stage. But like the fact that like AI scares people, it scares people for both good and bad reasons, but I think mostly for valid reasons, right,

that the fear is fairly bipartisan. That the biggest AI accelerators are now these kind of Republican techno optimists who are not looking particularly wise given how it's going with the first ninety days whatever we are, the Trump administration and the likelihood of like a substantial political backlash right,

which could lead to dumb types of regulations. But like in a part of it too is like okay, AI they're saying can do not just computer desk jobs, but like all types of things, right, And like humans kind of play this role initially as supervisors, and then literally within a couple of years people start to say, you know what, am I really adding much much value here? Right? You kind of have like these legacy jobs and there's

a lot of money. I think mostly scenarios imagine very fast economic growth, although maybe very lumpy, right for some parts of the world and not others. But we're kind of just sitting around with a lot of idle time. It might be good for live poker Maria. Right, all of a sudden, all these smart people, their open earth, their open brain, excuse me, stock is now worth billions of dollars, right, and like nothing to do because the

AI is doing all their work. Right, they have a lot of fucking time to play some fucking texas hold them right, that.

et cetera, has very real implications. Given that this is kind of the central dynamic of this report that their predictions are based on, I think that it's incredibly valid to actually speculate, you know, how will if at all, this effect the timeline of the predictions, the possible the likelihood of the two scenarios. And I will also say that one of the things in the report is that all of these negotiations on like will we slow down?

trying to could do some of these things. And it's funny because we were kind of pushing for transparency in one way, but there's a lot of things here that are very much not transparent.

the other depending on that alignment step. First of all, I mean the Utilian.

twenty six? Right, So that's kind of the breakpoint is like twenty seven is this inflection point? I think I'm using that term correctly in this context.

kind of three different things here, right. One is a notion of what's sometimes called AGI, or artificial general intelligence. And if you ask one hundred different researcher you get one hundred different definitions of what AGI is. But you know, I think it is based to like being able to do a large majority of things that a human being could do competently, siming we're limiting it to kind of

like desk job type tasks. Right, anything that can be done remotely is sometimes definition that is used or through remote work, right, because clearly AIS are inferior to humans and like sorting and folding laundry or things like that, right, that requires certain type of intelligence. Right. If you use the kind of desk job definition, then like AI is already pretty close to AGI. Right. I use large language models all the freaking time, and they're not perfect for everything.

I felt like, you know, in terms of like being able to do the large majority of desk work at levels ranging from competent intern to super genius, Like on average, it's probably pretty close to being generally intelligent by that definition, right.

how long it will take humans to verify the output. Right, So you think, like you think you save time by having aids do this, but if you want it to actually develop correctly, then you need a team and it takes them twice as long to verify that what the AI did is actually true and actually valid and actually aligned,

et cetera, et cetera. Now you're not asking it to do things that require that amount of time, but there do need to be little caveats to how we how we think about their usefulness and how you are able to evaluate the output versus Southern scenarios.

That's something I could have done. It might have taken an hour or two, but you know, instead, so it could do it. It's in a few minutes, and it gets faster. It's like, oh, I've learned from this from you before, Nate, so now I can be faster and doing this type of task in the future. I was at the poker tournament down in Florida last week and like, uh, you know, I ask open research. Excuse me, god, is that?

tournament an hour later. And meanwhile, you know, deep research bungles this particular task. But in general, AI has been very reliable. But the point is that, like there's like a a inflection point where like I'm asking you to do things that like are just a faster version of what I could do myself, I would at the moment I ask AI, be like, I want you design a new NCA model for me with these parameters, because like

I wouldn't know how to test it. But anyway, I'm mean long wanted to hear so AGI, we're gonna get AGI, or at least we're going to get someone calling something AGI soon. Right, Artificial super intelligence where it's doing things much better than human beings. I think this report takes

for granted or not it takes for granted. It has lots of documentation about its assumptions, but it's saying, Okay, this trajectory has been very robust so far, and people make all types of bullshit predictions of the fact that these guys in particular have made accurate predictions of the past is certainly worth something, I think, right, But they're like, okay, you kind of follow the scaling law, and before too much longer, you know, AI starts to be more intelligent

than human beings. You can debate what intelligent means if you want, but do superhuman of things or and or do them very fast? I think might be different, right, And NA that can do things very fast is and maybe it's certainly a component of intelligence. Right, But like, but I don't take for granted that like quote unquote, AI can reliably extrapolate beyond the data set. I just

think that, like it's not an absurdly pavlogic. It may even be like the base case are close to the base case, but like that's not assumable from first principles. I don't think we've all seen lots of trend charts

of thing. You know, if you look at a chart of Japan's GDP in the nineteen eighties, he might have said, Okay, well, Japan's gonna take over the world and be and people bought this and now it's kind of hasn't grown for like forty years basically, right, And so like we've all seen lots of curves that go up and then it's actually an estra where the fuck you call it? Where like it begins to bend the other way at some

point and and we can't tell until later. Right. The other thing is like the a billy of AI to plan and manipulate the physical world. I mean some of these things where they're talking about, like you know, brain uploading and dice and swarms and nanobots, like you know, there, I would literally wager money against this happening on the time scales that they're talking about, right, they double the timescale. Okay, then I might start to give some more probability. And look,

I'm willing to be wrong about that. I guess we'll all be dead anyway, fifty percent likely in this scenario.

but like they're very good at manufacturing and machinery. So like, what if we have the brains and they have the brawn so to speak, right, and they have maybe a

more authoritarian but functional infrastructure. So I don't know what happens then, right, Like, but the abilitiy of AIS to comment to your resources to control the physical world to me, seems far fetched on these timelines in part because of politics, right, I mean the fact that it takes so long to build a new building in the US, or a new subway station or a new highway, and the fact that

our politics is kind of sclerotic. Right. And look, I mean I don't want to sound like too pessimistic, but if you read the book I recommended last week, fight, I mean, we basically did not have a fully competent president for the last two years, and I would argue that we don't have one for the next four years. Right, So, like all these kind of things that we have to plan for, like who's doing that fucking planning? Our government's kind of dysfunctional. And you know, maybe that means we

just lose to China. Right, Maybe that means we lose to China, at least we'll have like nice cars. I guess.

The thing that I think they rely on is that that AIS are capable of designing just this incredible technology much more quickly so that our building process and all of that gets sped up one hundredfold from what it is right now. But it's still, at least at this point, needs humans to implement it right, and needs all of these different workers. And so yeah, I think there are some assumptions built into here that I hope, like I hope that that timeline isn't feasible, and I do think

that there are things that are holding us back. All the same, I think it's really I think it's interesting. One of the reasons I like this report is that it forces you to think about these things right and try to game out some of these worst case scenarios to try to prevent them, which I think is always an important thought exercise. I do want to go back to kind of their good scenario, which just so bad scenario is, you know, we're all wiped out by a

chemical warfare that the AI is release on us. Good scenario is that you know, everyone gets a universal basic income and AI does everything and no one has to do anything and we can just you know, at play poker. Yeah, and that just as as you suggested, that seems actually like a very dystopian scenario where people can become much

more easy to brainwash, control, et cetera, et cetera. It's like a dumbing down, right where where we're not challenged to produce good art, to advance in any sort of way. Just to me, it does not seem like a very meaningful.

you read it. There's also another post by a pseudonymous poster called el Rudolph L who wrote something called A History of the Future twenty twenty five to twenty forty, which is very detailed but goes through kind of like what this looks like at like more of a human level, how society evolves, how the economy evolves, how work evolves, right, and like very detail, just like AI twenty twenty seven is they're kind of focused on the parts of A

twenty seven, then I think it kind of deliberately ignores maybe can call them mild blind spots or whatever, right, But like, but that's interesting because that kind of thinks about what types of jobs are there in the future. There are probably lots of lawyers actually, right, because you know, the law is very sluggish to change, especially in a constitutional system where there are lots of vitail points, right,

probably high end service sector. You know, you go to a restaurant because everyone's for a lot of people are rich now, right, and you're flattered by the attractive young server and things like that is kind of highly kind of like catered and curated experiences. I guess I have some faith in humanities beilieved to fight back quote unquote against like two scenarios that it might not really like

either one, you know what I mean. And like the scenario where like AI is producing like ten percent GDP growth or or whatever. Right, man, it's great a few own stocks that are exposed to AI and tech companies probably, right, but it's also making that money on the backs of

mass job displacement. And like, you know, it kind of a sert confident in the long run that human beings fine productive things to do, and and you know, massive employment has been predicted many times and never really occurred, right, But like, but it's not occurring this fast where they think the world ends in six years of whether they're predicting or we have utopia in six years, and like just the ability of like human society to like deal with that change at these time scales leads to like

more chaos than I think they're more predicting. I think also, and I told them this too, right, I think also leads to more constraints. Right that you hit bottlenecks if you have five things you have to do, right, and you have the world's fastest computer, et cetera, et cetera, But there's like a power outage in your neighborhood, right, and that's a that's a bottle deck. Right, Maybe there are ways around it if you're like go to home depot and buy a generator or you know what I mean.

But like, but the point is that, like you're often defined like the slowest link, and politics are sometimes the slowest link. But also but like also, you know, I think the report maybe under states, and I think kind of in general, the a safty community like maybe understates the ability of like human beings to cause harm to other human beings with AI, right, that concern kind of gets brushed off as like to pedestrian or like like to.

it's a really important place to end. And I think that that's a very valid kind of way of reflecting on this name.

are outside of its domain yep, or there could be espionage. Anyway, we will have plenty of time, although maybe only seven more years actually to like explore, explore these scenarios.