The AI Arms Race Part Two

on the AI arms race, and the last episode. If you haven't heard that yet, you should go back and listen to that one first, where we lay all the groundwork for the stuff we're gonna be talking about today. Last time, what do we do? We talked about sort of some definitions of different concepts in artificial intelligence, how likely we are we think to achieve them, what ways they might be achieved, some potential stumbling blocks to achieving them.

But in the end we want to say, today, okay, let's assume that people create what's known as an artificial general intelligence. Is that going to lead to a worldwide arms race for intelligent machines? Boy, that's spoiled it. No, We've got a lot to say about this, and this is if you listen to the last episode you heard. At the end, I mentioned that we have some interesting characters to talk about, people who have perspectives on this idea, and the first person we're going to chat about falls

into that interesting character category quite handily. Yeah, so this is just one example. There have been lots of people actually who have written about the idea of a looming AI arms race. But one example of a guy with this prediction is a guy named Sultan Istvan who is the presidential candidate in the United States for the Transhumanist Party ticket. He's a trans humanist journalist and I guess, now, politician I suppose, yes, enthusiasts seems too modest a word.

He actually he had on his website a a suggestion of a trans humanist bill of rights, and it included things like the right to extend one's life and stuff like that. But anyway, so, uh so, Istabon has a thesis that he published in a Vice Motherboard article. Uh that was I guess a year or two ago, I think, and he he said, you know what artificial general intelligence is not going to arise in the lab of a tech company like Google, or a university research program or

some genius kids garage. A g I is going to arise through the work of state actors defending their interests and as so, speaking of coming superhuman artificial intelligence, Zoltan rights quote, politicians and military commanders around the world will want this super intelligent machine mind for their countries and defensive forces, and they'll want it exclusively. Using AI's potential power and might for national security strategy is more than obvious.

It's essential to retain leadership in the future world. Inevitably, a worldwide AI arms race is set to begin. So so here's the thesis that we can explore for the rest of today's episode. Now, Ralph, the face of it, I think at least part of Sultan's argument is is wrong. Yeah, I don't necessarily think the AI arms race part is wrong, And I don't think necessarily that nations wouldn't want exclusive use of some sort of super intelligent entity to work

on their behalf. I think that's apparent right, Like it would be crazy to suggest otherwise that if there was any given nation were to have the opportunity to have a super intelligent entity on their side. It would be insane to say, now we're good, go help someone else that then what's the part you disagree with? I disagree with his statement that says that AI is not going to show up in the lab of a tech company, but instead will become the will arise because of the

the work of state actors. Yeah, okay, so, and I know that's not the only thing we're going to disagree with, but that's the first thing here. Yeah, that's the first thing I disagree with, because I don't know if you've ever worked for any government agency, Joe, have you? Have you ever worked for I've worked for a university's state universe.

That's close. Yeah, Because all right, so you know about bureaucracy, You know about the barriers that are there and for progress, because well I know about bureaucracy and barriers to progress in private industry. Sure, sure, it's just in government, and I think it tends to be on a on a level that's exacerbated prodigious, right, And I mean, let's be fair. Okay,

So bureaucracy exists for a reason. Bureaucracy exists so that p did tasks have a streamlined process to go through, right, so that that way you have designed a system that is really good at handling very specific things. The problem is anything that's outside of that specificity doesn't fit the system, and then getting any progress on that is laborious at

best or possibly impossible, depending upon the degree. Now, I would say that companies are far more nimble and have a greater incentive to invest in research and development of artificial intelligence on a scale that dwarfs anything that any government is able to do, unless you're talking about a massive government that just decides to turn its full attention

on this problem. So king yes, yeah, some sort of very authoritarian approach saying all right, we're just turning all of our like if you're playing the game Civilization, you switch all your cities so that they're just producing science that kind of thing. Where uh, you know, here in the United States, for example, yes, we have organizations like DARPA that's part of the Department of Defense. They are very heavily into administering projects that guide research and development

in AI and other areas of technology. They're not the ones who do it themselves. Private companies or even publicly traded companies or research organizations do the work. So I think maybe state actors will play a part in the sense that there might be some funding. But I would argue that we're far more likely to see the emergence of something approaching general artificial intelligence from one of these companies, not from like a specific state sponsored program, because the

companies are more nimble. They're able to respond and change their course of action much more quickly than any government agency can. There's a lot of inertia and momentum involved in government agencies, and trying to change course is very

hard to do. You know. I would say, if you are a government and you're interested in funding AI research to win this AI arms race, to beat all the other countries and international competitors to beat, to establish this a g I that you can call upon to do your bidding, I think that the best strategy to go about that would probably be to set some kind of huge prize for it. Uh and and governments have done

things like this before. Do yeah, research prizes, So some gigantic pool of money, you say, first person to produce a computer that can meet these five requirements gets all of this, yes and uh and so again, then you're still talking on indirect kind of influence, right, like that you've you've created the incentive, but it's not the agency that actually creates the a g I that has been

the Resulten has been argueing for. Also in another article at Sultan wrote there's a bit where uh he he says, well, this would be unlike the space industry, where we see international cooperation like on the International Space Station, which to me seems to be very convenient in the fact that there he's ignoring the origin of the space race, like he's saying, AI would be different from the space race because with the space race, or rather the space industry,

because the space industry, we see this relatively peaceful cooperation between nations to advance our understanding of science and technology. Right, But the space race started as a branch of the Cold War. I mean it was, it was a product

of the Cold War. If it weren't for this this our space program, could could you argue that it's essentially something that was like hastily cobbled together after sput Nick because they were like, oh crap, well yeah, I mean, and spot Nick did exactly what the Russians wanted it to do, which was to indicate to the United states, Hey, we can build a rocket that can reach you, right,

that was really the Soviet Union's primary goal. Now, the people who worked on the spot Nik project, they had their other like their their own individual motivations and their own individual goals for that project, but the state goal was to say, we can launch a rocket that can reach you. You America, And then America said, well, we have to make sure that we have the same capability. Um and they funded it in a way where it was in the name of science, and there were legitimate

scientists doing legitimate science under this program. But it wouldn't have existed without the Cold War, right, without this this competition between two massive world powers. And so really I would argue that the reason why the space industry is the way it is is because this history that predates it of of competition and kind of uh well again a branch of that Cold War, it was, it was acts of passive aggression, I guess, uh so, I think

an AI arms race. I agree with this point that an AI arms race would be uh something we would expect should we reach that level of AI sophistication. And you could even argue that we're seeing an AI arms race right now. It's just we're seeing it in the context of narrow AI as opposed to general AI. But I a lot of the premises he sets up to support that I disagree with. I think that that those would need to be revisited. I do agree with his end conclusion, but his argument is not I don't think

it's supportable like the actual premises of his argument. That is okay, Well, what are some of the basic arguments than about whether the world is about to be entering an artificial intelligence arms race? Well, like Sultan says, he says, an AI a really useful AI. So the furthest extension we would put to that is the super intelligent general artificial intelligence. Although I argue that's not necessary for it

to be an issue. Uh. He says, it would be such a valuable tool that no one would be able to ignore that possibility, like to to to say, we're not going to go down that road. Well, I'd agree that AI is a tool too powerful to ignore. I mean, in general, intelligence is the most valuable thing we have.

Intelligence is the thing that you know, not just that empowers us, that makes human life worth living, I mean, and and increasing your share of intelligence capability is probably the most important thing you could do to ensure your future success. Right, and now he makes the argument that whichever nation out there created the creates the first super intelligent a g I wins the game. Like, imagine that that geo political uh influences and forces are all reduced

to a board game. Whoever hits that last square is the one who hit first, is the one who creates super intelligent a g I, and they win the whole thing automatically, hands down. Um. I don't necessarily believe that, because I think that access to a super intelligent a g I doesn't immediately solve other issues that a nation must address, at least not not so quickly that other nations can't also react to it. Yeah, I I think I'd agree with you there. I came up with a

little thought experiment to sort of service an analogy. So imagine you've got two ships trying to win a race across the Atlantic Ocean. One ship is a nineteenth century style steamship, is capable of about eight knots, and the captain is assisted by a team of the world's most brilliant scientists with access to powerful supercomputers and all the

tools they could want. The other ship is a modern ocean liner powered by geared steam turbines capable of thirty knots, but it is captained by a rather dim guy named Todd, who has no scientists or supercomputers at his disposal at all. Todd, Now, unfortunately, I think the fact is given what I've stated, Todd is still going to win the race, right unless unless Todd is so dumb as to cause damage to his ship. Yes, then, assuming that Todd is is at least of the level

of intelligence necessary to follow a specific route, Yeah, it's so. Yeah, it's good to have very smart people at your disposal to help optimize your chances of winning. But if you're stuck with a ship that's more than three times slower and and that's how you start the race, you're just

probably not gonna win. But I would say maybe if the captain of the first ship is able to consult with the scientists and the supercomputers over a period of several years leading up to the race, well then you you could maybe go in with a clear advantage and win it. But then again, if if dim Todd knows this, he may get some scientists and supercomputers of his own and then you've got a right, you've got an arms

race on your hands. But I would say that just having this intelligence advantage does not automatically mean you immediately become the runaway winner for all time, right though. It though it is a very strong advantage. And ce Zoltan's point is that, or the argument he makes is that whichever nation develops at first will have a dominant hand from that point forward, will become the dominant force on

the face of the planet. Yeah, I know. He's also of the opinion that I think, if you have the power, so if you have a g I on your side, you have the power to shut down and prevent other nations from achieving a g I. And I think unless you're willing to take incredibly extreme measures, that's not necessarily

the case either. It also presupposes that that a g I is going to have those capabilities, and it may be that it can accomplish things that we cannot easily do on our own, but doesn't necessarily mean it can magically accomplish anything, right Like I there's I think he's giving it so much credit. And to be fair, we don't know, right we we don't have anything that's super intelligent, because we've not been capable of creating such a thing.

And maybe he's right, Maybe a super intelligent entity would be able to accomplish things that most of us would think of as being, if not impossible, impractical because it would be so difficult to do. But I agree with you. I think that it would be something that would give a country a distinct advantage, but not guarantee what Zoltan seems to think of as victory. Uh. He he seems

to to, at least in his arguments. I think that what we're heading toward is a world that's united because whichever country comes up with a super intelligent a g I will be able to subjugate all other countries do its will. And he thinks, gosh, darn it, the good old US of A should be the country to do that. Ah. There's some elements to Sultan's philosophy that I find particularly troubling.

Might be might be too strong a word. Well, I mean to defend I I don't think I agree with his philosophy either, but to defend what he says, I mean, he he does say at least what he has in mind is a sort of benevolent hegemon e. It's not like we're going to subjugate you and oppress you and do what we tell you. I think what he has in mind is that, you know, it's going to be

the Federation from Star Trek. Everybody's happy, and you know, it's a utopian kind of I think that's what he sees if the United States is the entity that creates this. But I think he also imagines a world that would be in the realm of the dystopian future should some other country do it, like, for example, China or Russia. Not that he specifically names these countries, but that would be example, or North Korea. I think he had actually

does in North Korea. No, I will admit I can imagine if North Korea were able to develop an artificial general intelligence long before anyone else. I think that could have disastrous consequences for the planet. Absolutely. But I think again saying that in those scenarios, you don't get that Starfleet future, you also get some other science fiction future that's probably written by Burgess or something. I also don't

think that's very likely though. Yeah. Uh, Now we have other people that have weighed in on this issue, who have some really interesting points to Yeah. One thing we read is by Anya Cosperson, writing for the World Policy Journal blog about the idea of an AI arms race, and what does she say, Jonathan. She says that, first of all, AI is it's undeniably intrinsically valuable. And Yeah, I think that. I think there is no way to dispute that. I think we both agree that AI is

definitely valuable. Um, even in even at the extent that we have it right now, the the narrow AI that is our reality, we see that it's valuable because it's doing stuff for us. I mean that that kind of defines value. Uh. So there's an incentive for countries to

pursue the development of AI. I think that's undeniable too. Now, even if those countries agree that weaponized AI is unethical, that's not enough of a deterrent, she says, to make sure that this does not, you know, blossom into a weaponized AI future, because Yeah, there's always gonna be rogue

operators that may not be state sponsored. You may have independent rogue operators that are willing to either develop or perhaps more likely appropriate AI technology and then convert it for weaponized purposes, which creates pressure on all nations to consider developing weaponized AI as uh, just as a preventive measure. So in other Mr President, we cannot allow an AI gap exactly exactly. Uh. There are a lot of doctor Strange love moments I had while researching this this particular topic.

You can't fight here. This is the war room. So one of the examples that you could think of is that the United Nations agrees that weaponized DAI is a bad idea. This is this is a hYP hypothetical situation. I actually had more to say about this in a bit. Yeah, well we'll we'll cover it in greater detail in a second. So in this hypothetical situation, the U n says, all right, uh, we as an international community agree that weaponized DAI is a bad idea. It's gonna lead us down a destructive road.

Once we head down there, there's no turning back, and the outcome is going to be negative. There's not a way of envisioning this future where there's like a net positive outcomes. So we're gonna agree no weaponized DAI. But then you've got those rogue operators like a terrorist cell or organized crime that see this as an opportunity to

use AI in a weaponized way, and they don't. They're not beholden to this international agreement, they have no stake in that, right, They're they're not getting they're not worried about international sanctions from the u N. They're trying to accomplish whatever goals their particular organization has and they're using

whatever tools are at their disposal to do so. Yeah, I mean you're invoking here the concept that and this is true generally even peace loving people who are not looking to start a fight don't tend to opt for unilateral disarmament. Yeah, because because if you were to do that and there's that one person who hasn't agreed, then that one person just runs rampant over everybody else. Right, Like if you if you had a group of kids,

let's go Lord of the Flies. You have a group of kids, and each kid is given a whiffleball bat, and it's just to keep this from being like too violent for for even for a hypothetical situation. Each kid has a whiffleball bat, and you tell all the kids, like, well, you can either have that whiffleball bat and wail on your classmates, or you can put the whiffleball bat down and we'll have a fun activity. But you all need to put the whiffleball bat down in order for that

to happen. And one by one they start dropping their whiffleball bats until there's that one bully in the back. It's like, no, I know what my fun activity is, hit all the kids who don't have whiffleball bats. That's

the same sort of argument here and up. So you know, I made a little funny note in here saying, next thing, you know, we've got gangster drones shaking down local businesses for extortion money, Like you know, it's a you gotta you got a nice pizza joint here, and be a shame as someone I don't know assimilated it into what's

growing technological layout, you know. So so you know, I think that's a legitimate argument, and I agree very much with those those assertions that if in fact, you know there's a possibility to go down this, then someone, someone, whether state sponsor or not, is going to do it. I would also argue we're already at a point where this can happen. You don't need a g I for

this to be a problem. Narrow AI is enough exactly, So you don't need a general intelligence that can you know, be super intelligent and produce the next next generation of technology and innovations and everything. You just need like a really good weapon that's smart. Yeah. Or even if you're going to go the semi autonomous route right like, you could argue we're there with drones. Were there, we're using technology that removes human operatives from conflict and replaces them

with robotic ones. We'll have a lot more to say about about that concept a little bit further too, because that that ties into one of the other big concerns about weaponized AI in general. Sure, okay, so let's uh, let's speculate little more. Sure, if there is an AI arms race, what's it gonna look like. What are some scenarios that could actually happen? Okay, So we have the state sponsored approach where countries decide not to essue weabinized AI.

Whether that ends up being a direct implementation of like a robotic soldier or AI that's assisting in some military capacity for strategical or tactical planning, it doesn't really matter. You could have topped down AI or bottom up AI exactly, or both you know, you could have some integrated approach that's both. Uh. So one scenario is that you have established nations like the United States, Russia, China developing weaponized

AI like webinized robots is a good example. So there's still some they're already examples like drones that are kind of in this category, whether they are autonomous or semi autonomous or remote controlled. Uh, there are various types. Obviously remote controlled you wouldn't call it AI because it's under the operation of a of a human, but semi autonomous or autonomous would fall into the AI category, even the

narrow AI band. So these future robots, uh that these hypothetical future robots that are weaponized AI, they take the place of our soldiers, or they augment the presence of soldiers. So let's say that you would normally be able to field a couple of thousand soldiers. Well now you've got a couple of thousand soldiers and ten thousand robots. Or you get to a point where you just have the robotic soldiers and you don't put any humans in the field at all. It starts to sound like I'm playing

war gaming here, but this is serious business. Uh. They those robots then use narrow or general AI, depending upon the level sophistication we've reached in this hypothetical future to identify targets and differentiate them from non targets. They can enter hostile zones and make snap decisions without any emotional considerations. So they're very cold and calculating in that sense. Because they are things and not people. They remove some of the checks that would be in place for a nation

before entering into conflict. This is one of those common arguments about if we go down the road of weapon iszed ai, it makes war so easy. Yeah, war becomes such an easy decision because I mean yes, because you're you are not committing human life. You're committing resources, so

it's still costs money and time and resources. But you're not You're not saying, well, we're gonna you know, we're not gonna enter into this because human lives will be lost, and we will lose citizens and families will be affected. You're like, well, the robots, you're putting more layers of abstraction between yourself and the actual carnage too. I mean, I'm sure you've read that old idea. I can't remember

who it was, who suggested it. But somebody who suggested, you know what, Uh, you've got the person who follows around the president with a briefcase for the you know, nuclear launch materials that you would require in order to initiate a nuclear strike. And the suggestion was, you know, the president really, uh shouldn't be able to just initiate

the strike. The president he or she should have to kill that person physically with their bare hands or with a cleaver of some kind in order to get the code out of some kind of implant in the person's body in order to use it to launch the nukes, because it show a level of commitment that that is necessary considering the outcome of the choice. Yeah, it sounds insane, like that's that's insanity. Why would you ever institute a policy like that, But you're talking about a policy of

launching a nuclear strike killing millions of people. Yeah, and it just shows you that like you're not you're not really considering those deaths. You just you've got them sort of like a step away from you. Those those are numbers and their numbers that exist far way away from where you are. So the exactly that that abstraction is what makes that action easier to undertake. When you make it a real thing that you have to confront, it's harder.

And again, like you're saying, you know, with the robotic soldiers, you've got that one concern that's taken away, the idea that well, our soldiers won't be put in danger. But then you you should start asking your question. Yeah, but the other side, that's that's people. These robots, we would be killing human beings. They're not our human beings, but

they're human beings. And that's where you know, depending upon the the UH motivations and the psychology psychology of the nations involved and the military personnel involved, you get very different outcomes. Um. So there is that argument that we would become more warlike in this future because we would have fewer impediments to entering into a war um which

also kind of sounds like the Terminator scenario. If you remember, Terminators were robotic soldiers that were meant to um be in the service of humans, but because of the emergent general intelligence, the super intelligence of Skynet, they turn on humans. Well, they don't have to possess general AI to be a threat. They could have narrow AI. They could even be semi autonomous where there they're consistently being guided by human intervention for it to still be an issue, to still be

a very real escalation AI arms race problem. Um, the super intelligent general AI think makes it absolutely terrifying, but it's not necessary for it to be a problem, right, Like,

that's that's just the the degree of the scenario. Uh So, if you do get to that general AI super intelligent, then you start entering into other possible disasters, the classic science fiction one being that the robots themselves decide that they have to unite against humans in general, not just work for one group of humans against a different group of humans. For some reason, I've never found the robots decide to exterminate humanity scenario all that plausible. I have

found very destructive scenarios plausible. Like I could see how artificial intelligence could cause catastrophic damage to human civilization, but the I don't see it coming in the form of

exterminate all humans. You have to you have to have some form of you know, the robots would have to have some motivation, right as opposed to just the directive, Like I can easily understand imagine scenarios where robots through a directive pursue it in a way that was not anticipated by the people who there are disastrous side effects

of their behavior. The classic example being you've got a super intelligent computer and you say I want to bring about world peace, and it decides to eliminate all humans because by eliminating all humans, you've eliminated the potential for conflict. Right. That's that's the classic example of Oh gosh, I wish I had thought of that. Uh. It also falls very much into a realm that I am I love very dearly,

which is dungeons and dragons. Whenever your character receives a wish, you will see players spend hours trying to craft the perfect wish to to create a a full proof scenario where the dungeon master can't misinterpret the wish and create a terrible outcome for the players. I've never seen this, so they're like genies or someone. Yes, So so you might say like, like, uh, you know the Simpsons version make me a sandwich and turns them into a sandwich

because it's misinterpreted the wish on purpose. But same sort of thing with the general AI, Like there there are disaster scenarios where it's not that the general AI was setting out to cause harm. It's just that through the process of trying to achieve whatever the goal was set, there were unintended consequences that were harmful. Um. There are

other scenario as though. There's the one that I think is the most likely, which is that companies, some sort of corporations, maybe multiple corporations, developed more competent AI that ends up having a negative impact through unintended consequences. Kind

of similar to what I was just saying. UH, governments will end up relying on these companies, contracting with them or even just buying off the shelf components to put toward UH military use again either top down or bottom up, essentially saying like, well, I know that this this piece of software was intended to do this, but with a little modification, we can have it do this other thing. We see this all the time in all sorts of

technology industries. The VR industry is a great way of pointing out how this happens in VR before we got to some of the consumer headsets and and controls that are out on the market. Now you had researchers who were they were they were appropriating video game technology working with it programming new interfaces for it, and using that as tools to develop VR applications, And it was all it was taking something that was in a related field

but changing it, transforming it to do something else. We could see that, and I think that's the more likely future that we see various types of AI developed specifically to do some non military function but then get adopted into into into uses applications that it was not necessarily

intended for at the beginning. I think that's very likely simply because companies have a lot of incentive to continuously innovate in this space, and we're going to see a lot more rapid development there than we would in some state sponsored program where uh, you know you've got you've got a top down pressure that work harder, do do more innovation. Um. And also we could just see AI cause massive amounts of harm just in its normal operation,

not even when it's being used for something else. So we've talked about this in the past too, Like the stock trading AI, like the various simple algorithms that just look at the stock market and make very small but

very rapid transactions. Uh, once you get a bunch of those acting at the same time, you don't know how they affect one another, and there can become a cascading effect where you create market and stability simply because of the collective activity of all these where if maybe it was one AI or one simple algorithm that's acting on the stock market, no one would notice. It wouldn't really

make a big difference. But when you have thousands of them all working to try and make a profit for whichever company is employing them, there could be these unintended consequences leading to like a market crash, which we've um We've seen market crashes that last minutes and then in return normal, but it could be catastrophic. Uh. Then we

have the rogue actors scenario. This is the one where the countries, the various nations in the world have agreed, Okay, we're gonna take a hands off approach on weaponizing AI. We don't want to do that. The rogue actors one is that these other entities, whether they're terrorist cells, organized crime. Uh, you know, maybe it's a a group within a nation that wants to break free of whatever the the government

in power is, they might be willing to take those steps. Uh. One of the examples kiss person actually brought up is to imagine a weaponized drone with facial recognition capabilities flying over a crowd, and it analyzes the crowd and it looks for a specific person, and if it identifies that person, that person is a target. It's weaponized drone, so it fires on the target and you essentially have assassination by robot. This means that doesn't sound implausible to me at all.

I mean, you could you could rig something up like that today. It would not be terribly sophisticated, it would not be terribly reliable. But the other point that Casperson makes is for rogue nations, they don't not even just rogue nations, rogue actors, they don't necessarily have a high threshold of reliability that they must meet in order for them to be comfortable investing in this technology or utilizing

this technology. Right like if if there if their device ends up killing someone mistakenly, well that's unfortunate, but it's they don't necessarily care, like, that's not they don't have anyone else to answer to apart from their own organization. And if their organization doesn't value that humans life, there's from their perspective, there's no problem nations is different. Right, you have a lot of people to answer to, and

you can't just go killing people indiscriminately. There huge problems with that obviously, I mean you know everything everything well, like like not just the not just the obvious ethical issues. But then you've got things like the Geneva Convention to contend with two. Right, So you've got this this possibility of UH agents that are outside of state control that could potentially be using this technology, which therefore puts the pressure on all the states to at least figure out

a way to counteract that, if not keep pace with it. So, by the way, I also really love the word rogue and also uge, which some people well interchange because they make a type of So I'm not talking about yeah, Mulan rogue are rouge actors. As an actor, I have a warn rouge before. I'm not ashamed to admit it.

I'm sure it look good on you. Okay. So one more thing I want to point out is that very often the ideas that get tossed around when discussing the concept of an AI arms race are specifically about weaponized AI, so it has military connotations. You're talking about military robots or drones or at the top level, a g I that's some kind of military commander or creating new weapons

or something like that. I was thinking about how there could very much be an AI arms race that's not necessarily explicitly military in nature, or at least not at first, right, I mean, so developing an a g I. If you have a superhuman general intelligence in a machine, that's not just useful for military advantage, it's useful for every possible competition, right right, any any application of intelligence, it would by

definition be useful toward that. Right, So in market competition and trade wars and stuff like that between countries, and a g I would be a supreme advantage. I mean in any in any place you're looking to solve a problem or excel, the a g I is is an advantage. So it almost seems like an overly confined way of envisioning the transformation of geopolitics by AI to just think about AI robot soldiers and AI military commanders and stuff

like that. Yeah. No, I agree entirely, which is why one of those scenarios I was talking about companies like can you imagine the first company to provide a a computer or a smartphone or a tablet or some piece of personal electronics that incorporate a g I into it in some way, whether the device itself possesses it or it has uh an Internet link to a cloud based a g I solution that would be enormous, right, that would be that would be a game changer. That would

be a killer app a killer product. And so assuming it's it's super intelligent, yeah, well even if it's not super intelligent, just just imagine having a pretty intelligent person you could rely upon to do stuff on your behalf. Like think of all the narrow AI applications we have right now, but you have all of them wrapped up

in a single format. Factor, So you would have something that could do everything from helping you keep track of your activity throughout the day, to making sure that you are on time for all your scheduled appointments, to following up with you about concerns, all this kind of stuff like like that that ideal personal assistant that really creates your reality. Uh as far as how you interact with technology,

that would be a huge product. And you can bet that any company that works in consumer technology is looking to incorporate more and more sophisticated AI into their products. So it's there already is an AI arms race in that sense. It doesn't necessarily mean an arms race that

is going to lead toward violent confrontation. But there is this competition already in the market for companies to try and develop in that space, and maybe several generations down the line, that evolves into something that is more akin to a weapon's arms race, But it doesn't necessarily have to start out that way. That's one of the scary things about this, right that we might not even intend this to be used in any way that would be

remote related to military applications or acts of hostility. But generations down the line, and by generations, I'm talking about technology generations, which happened way faster than human generations. That might end up being the case. All right, So what happens? What? What are the what are the outcomes of the AI arms race? Like, what are the various things we see this? How how do we see this turning out? People stop

making terminator jokes on every article about AI. I mean they would certainly think twice about it before the AI the AI kneecappers come by. I saw that joke you made real funny. You know what else is funny? This battling clamps. Now they don't need bats, but you know, sometimes you just got to do stuff for style. Okay, no, what we're what are some things? So? What are some things? So one outcome is that we get this uh competitive approach.

This is what we're seeing right now. Everyone's competing to make more robust, useful AI. Eventually someone creates a super intelligent AI that's harmful to humans, either intentionally or otherwise, and then we have to deal with that reality. Ultimately, dealing with that might mean that we don't succeed, that we as a species are eliminated, because if you're talking about super intelligent AI, by definition, it is smarter than

we are. It's more capable than we are at processing information, and thus, any strategy we might come up with to try and take down that AI will be something it's already considered and figured out a way to counteract um at least that's that's the general way we we frame that, Like, if it is truly super intelligent, nothing we think of will be something it hasn't already considered by definition, because it's more intelligent than we are capable of being unless

we also become super intelligent. Yeah, well then there's a totally different AI arms race, right. Uh So that's one outcome that's kind of depressing. You know, everyone dies. I don't really like that, big fan. I mean, I love Shakespeare, but you know, we don't have to have it be a tragedy. Were canna have a be a comedy. Maybe everyone gets married, we get married to robots. How about that one. That's a good outcome. Uh So another outcome is super intelligent AI ends up being impossible or so

impractical to create that we never really implemented. And in this scenario, we would discover either that we lack the capability to achieve super intelligent machines or we could do it, but it requires so much energy and it's so resource heavy that no one's willing to actually go to the

step of making it. Yeah. I think I stick by the conclusion we talked about it in the last episode that it is pretty much demonstrated that super intelligence is in principle possible, but that doesn't mean that it is in practice feasible. Right, it might be possible in principle,

but we just can't ever do it. Yeah, if it if it turned out to be something where we just don't have the physical resources to design the system, or maybe we do, but it would be to to do so would create a hardship because we're diverting so many resources from other things, or we're just too locked into patterns of stupidity to ever figure out how to do it. Right. Yeah, then we never have to worry about the super intelligent

part at all. But here's the problem with that. We don't need it to be super intelligent for it to be a problem. Right. You don't need a driverless car to be super intelligent for it to be able to deliver an explosive, for example, which is terrifying, but a reality that we could see. Um. So then we would enter an era of rapid development to both create and

prevent harmful uses of AI. UM These would be human guided, right, not necessary fairly AI guided, because if we aren't able to make superhuman intelligent or super intelligent rather AI, then we would mostly be relying on human ingenuity in that case. Zoltan's outcome was that idea that we would become a united species under one government, one flag, one nation, indivisible. His goal would be to have America be the dominant

force that would then bring the rest of the world together. Uh. So, essentially the US would annex everybody else and then we would just have to worry about the possibility of super intelligent AI that we had created turning against us, and not worry so much about each other. Um. So are you voting for Zultan President? No? I I don't think that's a realistic outcome. Um. Either. Well, first of all, I don't think it's realistic that we would have unity

on any on any global level. We don't even have unity on a national level, so unity on a global level to me seems idealistic to a point of being implausible. Um. But beyond that, I think again, as I've said many times, the the AI being an issue will happen long before we get to any super intelligent type of AI or

general intelligence. Um. Another outcome is that we create a series of guidelines to shape AI so that it works for the benefit of humans, not to the detriment, and that involves creating a set of practices that allow us to guide AIS development a way that doesn't lead to the terminator scenario. This is where you have the people arguing we need to come up with ways to instill human value into machines problem yeah, and the and of course this leads to other problems too, right, like who

defines what human value is? Under whose definition do we create this AI? And like who writes the guidelines? And who wants to stop someone from developing AI that doesn't you know, abide by them? These are all tough questions I don't have answers to. Well, I guess the last major question is is there any way to avoid an AI arms race? Well? Can it? Can it be averted? We first, we first have to think about that you in, right, Yes, the international agreements the AI equivalent of the Nuclear Test

Band treaties. So the international community, Uh, they've done stuff like this before, with varying degrees of success. They've attempted to halt or limit the proliferation of nuclear weapons and the acceleration of the twentieth century at least nuclear arms race by coming up with these International Nuclear Test Band Treaties. And these treaties banned various types of nuclear weapons testing

among ratified parties. For example, the Partial Test Band Treaty of nineteen sixty three was signed by nuclear powers like the US, the UK, and Soviet Union, which mutually agreed to prohibit all testing of thermonuclear weapons except for underground tests. Right. The same thing is true with chemical weapons, like they were essentially international agreement that that is a as a no go. Right, So there's a question, I guess, could

we do something similar with AI? People recognize the potential for danger and make AI development illegal everywhere in the world. First of all, I'd say this seems unenforceable even by international treaty, Like, even assuming you could get every country in the world to sign on as a party to that treaty, which I don't necessarily think you could. For example, the Comprehensive Nuclear Test Band Treaty of nineteen six that attempted to extend the you know, the scope of the

nuclear test band treaties that existed. They couldn't get that signed or ratified by key countries. And by my estimation, I think it's much easier to hide the development of artificial intelligence than it is to hide the testing of nuclear weapons. You can probably perform key AI research in ways that can be contained in a single nondescript warehouse. It's just not that hard. There's no radiological evidence to sniff out with an atmospheric collection aircraft or anything like that.

There'd be no seismographic signature. All these things we can use to check and see if people are performing nuclear tests they're not supposed to be doing. You could have decentralized development, Yes, how would you? How would you do anything like that with AI? So in the end, I think even if you're able to get all the member states of the u N to sign a comprehensive AI Test band treaty, there would be no way whatsoever to

enforce it. It would just be paper. Right, And we have to keep in mind that, like we said before, there are a lot of potential uses for a GI. The scary, right, But do we even want this? Mean, let's not forget how useful AI could be if it's a G. I's promise this, this almost certainly yields major breakthroughs in medicine, energy, all kinds of fields that would, if implemented properly, improve human life, increase wealth for the

whole world, and protect our environment. But I guess the question. You know that that key phrase, if implemented properly is a big if. Okay, So would there be any other ways to prevent it? I would say nothing that I can think of. I mean, my intuitive ruling is that if a g I is technologically feasible, a different question than whether it's possible in principle, which I agree that it is. It's just not possible to prevent the development

of it. You can't put anything in place to stop people from doing it, unless you're talking about some something extreme like just well, wipe out the whole human race or destroy all computers or something right like like the human race as a as an entire species gets wiped out before we have they to create an a g I, which which case that's like trying to prevent I mean, or that some other thing caused it and it wasn't It wasn't even and the the a g I wasn't

even remotely connected to the reason for for the extermination of the species. Assuming human progress continues as normal without major catastrophe, if there's a way for us to do it, then we're going to do it. Someone will do it. Yeah, it doesn't. I mean, if it's possible to create it, someone will create it, because that's what we are as a species. It's it's not necessarily that someone would create it. In an effort to create to do malicious activity, but

someone will do it. If it's possible, then we will get around to doing it sooner or later. The question is how hard is it? Right? Like if it's if it's so hard that it's possible, but we're not going to be there for another century, Well we can have this conversation, but it's not really really going to matter until a century from now. That's it's still going to matter, but it will be a hundred years later. This whole discussion does seem to make that control problems seem of

of imminent importance. Like even if the development of a g I is not imminent, which I don't think it is, I don't get the feeling that it's going to happen anytime real soon. We still do need to be thinking about these control problems, like what what do you do to UH to you know, to limit the unintended consequences of a g I to UH to understand how we're going to use it and prevent people from using it in a destructive way. UH. They're they're really hard problems,

but we need to be working on them. I think what we're gonna see is a new dawn of electromagnetic pulse technology to take out take out robo soldiers and computers. Is I mean that's I'm I'm kind of making a joke, but I'm kind of not because as if you get to a point where you're thinking, well, I can't stop anyone from trying to pursue this line of development, UM, and even if it takes a long time, people are

going to go after it. What do I do in order to prepare for the eventuality that someone is incorporating this on some scale, whether it's a bunch of narrow AI implementations, a general AI into implementation, or super intelligent a g I. You gotta have you gotta have a plan in place. I'm I've kind of made a joke about e m p s, but I think that's gonna

be a big thing. I really do think that, uh, those that technology would be an important element because if nothing else, you could shut stuff down, anything that's electronic, You could shut it down long enough for you to have a different plan to go in place too, UM potentially take care of things. Obviously, if you get to a point where it's a distributed intelligence, that's a lot harder because there's no place to aim at. It's everywhere, and at that point we're having we're having a totally

different conversation. Maybe that's the point where humans and machines merge and we we truly do, become transhumanist and Zoltan rules over us all well said, yeah, um, and I know we had a lot of fun with this, but I mean, it's it is one of those things where you've got to kind of stretch your mind to think about because obviously we're talking about situations that aren't in the realm of possibility right now, but we can see kind of where we're headed, so it's easy to imagine it,

even if it's not something that is pressing right now. Uh. I kind of curious what you guys think. If you have any thoughts on the subject of the AI arms race, you should write in and let us know. Our email addresses FW Thinking at how Stuff Works dot com, or drop us a line on Facebook or Twitter. You can go to Facebook search FW Thinking, our profile will pop up. You can leave us a message or send us a tweet over on the Twitters. Our handle is f W

Thinking and we will talk to you again. Really see for more on this topic in the future. Of technology. Visit forward thinking dot Com, brought to you by Toyota Let's Go Places,