AI Intentionality: Who’s Really in Control?

And to help us do that , we have two really great guests today . Roni Abovitz is a founder of Mako Surgical , the gold standard in robotic surgical systems .

He then founded Magic Leap , one of the innovators in augmented reality , and most recently founded a company called SynthB , which talks about developing collaboration in intelligence , which we're going to learn a little bit more about what that actually means , rony , and also Dr Uri Maoz . Professor Maoz is a professor of neuroscience at Chapman .

He's also a visiting professor at UCLA and Caltech and a member of the Chapman Interdisciplinary Brain Institute , and that's really , really important when we think about AI intentions . So , gentlemen , welcome , thank you .

So the answer is there are certain species that we can create and almost breed and harness and tune that could be highly compatible with people benevolent and help us do great things . There's others which could turn into like the nightmare you know , velociraptor , t-rex , you know kind of scary things at a Jurassic Park level .

It depends what you're breeding and what you're trying to build and what kind of not just like policy controls , but the actual creation of this sort of new kind of intelligence or almost life form . You have to be very deliberate . And are we creating golden retrievers or , like you know , insane vipers the size of aircraft carriers ?

So I think it's like like it's not one thing , it's an entire field with many variants . So I'd love to hear Dr Moses take on that . But I think some of them can be controlled . Uri , what do you think ?

I see his point about different biological systems being more controllable than others , but I also worry about the fact that even if we try to limit what people make we don't have , we can't control everyone and it could be some rogue player somewhere that makes this what was it ? Super tanker , Viper or something like that .

So like the clones that were made in Korea , something along those lines . So if that's the case , I think we need to think about methods that would control potentially anything , even the mega viper or something like that . How can we know what it's about to do before it does it and stop it ?

Are those templates that we use to develop in those controls and those kinds of disciplines transferable to AI , or is it a completely different kettle of fish ?

The difference that actually frightens me here is that you can take an AI system , put it on the internet and within a few hours it's in a rogue nation where you don't have access to it . It's not the same way with nuclear weapons centrifuges . Engineers , you need physical things here .

I mean , yeah , you need some things like a computer , but honestly , to run ChatGPT you need $30,000 , $40,000 worth of equipment or whatever . I don't know ChatGPT or whatever whichever one . So I mean , any country , not let alone a business , can invest $30,000 in a computer .

Right , we're trying to aggregate knowledge and wisdom and data and information and know-how and skills , and at the same time , there's others in the space exploring consciousness and synthetic life , and you know these emergent properties .

So we're really playing with the , you know , primal kettle of fish , that sort of , you know , depending on what you believe got us kicked out of the Garden of Eden , into wherever the heck we are right now , but it sounds like we're about to do that again .

So I don't know if humanity has a playbook for getting itself out of the sticky situation other than we . If you look backwards , we just kept making it worse for ourselves . So we have to be very present and aware and collectively united , I think , to put our arms around this . I agree with Uri . I'll use Uri instead of the doctor , if you want that .

This is very different from , like you know , bioweapons or nukes or things like that . You can't control amounts of uranium Like a 16-year-old kid hacking an old GPU in a garage could do God knows what at this point . So you need like societal ethics and all sorts of people collectively coming together to dampen what's going on .

Like if you had a farm in Iowa that was breeding toxic velociraptors and everyone else is breeding golden retrievers and dairy cows , somebody would say , wouldn't God's name , why are you doing that ? You know , and somehow the society would form around that person and go why are you breeding these poisonous snakes that can kill us ?

So you have to create this feeling of like golden retriever is good , dairy cows good . The guy in my next door breeding thousands of cobras to release them in my neighborhood bad . We don't want that . So people need to be able to identify these things and deal with it . And right now the level of ignorance is high .

The power concentration in small hands is insanely high . We have a real problem . I'm glad we're talking about it , but I don't know if I have a collective answer . I have a local answer .

You know , local , responsible people can create and breed the safe forms , the dairy cows , the golden retrievers that we want , the cancer curing things , but I don't know how to deal with rogue people right now .

That's a government level and almost like local level , like in your city , in your town , if someone was doing something horrible , what would you do about it ? You know it can't just be . The federal government has to come in and solve it . You do need some like local level intervention .

It's like if someone was had a meth lab , two you know , two doors down , what would you be doing ? Yeah , so if some kid has some illegal , crazy AI toxin producing computer farm , what are you going to do ? You know what's the local , you know , is there an AI police ? I don't know . I mean , it would be great to get your take and your guys on this .

yes and no . The systems right now , the biggest , baddest generative AI is still . I mean , gpt is for pre-trained , so they are pre-trained , and that still costs many , many millions of dollars . After that , there is I mean there are some types of learning that happen , but it's not the big learning actually takes quite a bit of effort and time .

But who is to say that the next iteration of something or other would not be ?

Hey , we can now actually make it learn on the spot , and I mean that's , I think , for us a good thing , because if we get on some solutions right now , while the big learning still is frozen after the training phase , then once these things become more like us , in the sense that all of us are going to be different after this conversation assuming we are not

in a coma and we're learning from each other we're going to be different . Our brains are going to be different at the end . If you talk to Chad GPT at the end of the conversation , I mean there are some things that are different in the transformer , but the big thing inside the brain quote , unquote is actually not any different .

So if we don't act now when it's not different , how are we going to chase it when it morphs all the time ? And the other thing to remember here is we live in geological time for these machines , right ? They work on the nanosecond scale .

Our action potentials in neurons maybe the shortest time we can think about is about a millisecond , so we are like six orders of magnitude slower than they are . A second to us is two weeks to them , right ? So if we don't get ahead of it now , it's going to be a problem . I'm sorry I'm not having a lot of .

I don't have a lot of good news here but that's I mean yeah .

It's like AI becomes the new Mount Sinai , you know , and I worry a great deal A about what are the intentions of the developers and how much it's sort of skewed to go into a certain direction and the fact that you have so few alternatives .

I mean , if I want to do search , you know I could do Google , google or Google , you know , I mean the other engines are just not there . So I think that that's what worries me . Is that a degree of intentionality where you have a group of companies all of whom have the same worldview , all whom come from the same culture .

Basically , bay Area , puget Sound , you know , advanced engineering schools , and I worry a great deal that the kind of sort of vibrant debate just will not emerge because it's all sort of the same mind meld .

I think if you've got malintentioned or unintentioned meaning if you have no feeling at all about what you're building or you have negative intentions , somehow that will seep into the work . And I'll give you an example . This is like , backwards looking , what we did at Mako .

We asked how could we keep something safe that could potentially cause a lot of harm to someone in an operating room ? You know , a robot plus AI , and we integrated Asimov's three laws into the system . So we were like , what is the ? What is sort of like the , the rules that you'd want not just to be policy on paper , but in its DNA , in its behavior ?

How do you make the behavior benign so that it is inherently safe , the way you know , if you had a hawk in your hand ?

It's much , much less safe than petting a bunny rabbit Like you want to make a bunny rabbit because that's going to be safer on people , but a hawk could rip your eyes out , and right now I think people are building hawks and velociraptors and things like that .

So the intention to me translates into design architecture that has genetically built in behavior and safety , that incorporates things like Asimov's laws of robotics and adapts them not just to be legal things , but literally the control , your behavior , the way , like in an animal and we're going to talk about this like the brains of certain creatures are tuned for

certain benign behavior and others are not . And I think this is where you have to get to the intention of the companies making things . Now , if you educate the populace and say you have to start rejecting people who are not caring about that , you know like , basically , you have to vote with your feet , because people are really ignorant .

They think there's no choices . That's not true . There can be many choices and you can outright reject people that have malintentions , many of the you know . I don't want to name some of the big companies , but they do not have your best intentions in mind .

I hear what you're saying about let's build benevolence into the DNA , right ? So we're going to create the golden retrievers instead of velociraptors . I get that right . But somebody will do gain-of-function research to obviate the golden retriever gene and morph it into a velociraptor . Whether you like it or not , is there some disincentive to use it ?

I mean there were some European companies , I think it was Minstrel and some others , but the US really had the , you know , the lead . And then there was the deep seek moment right which is like whoa right like the chinese have kind of potentially caught up with us .

This is like nick , it's like yeah , well , sputnik was different in the sense that the soviets were ahead right here it was like oh , stalin , stalin is just . You know , joe just exploded the atomic bomb , right . So there is that moment in the deep seek . Of course , here it was also a financial . The difference is the world is more integrated .

But I just I think that's an interesting thing to some extent creates this oh , if you're going to I mean it's not AI has many uses other than the military ones of let's kill as many people or something like that . But there is something in the sense of oh , if you AI us in some bad sense , then we can AI you back whatever that means .

you cannot let a near superpower peer get overmatch over you . Like , if you got air power overmatch , you win . The US has had air power overmatch for a long time . It's just basically you won and that's the end of it . Ai overmatch is the next major thing .

That , if you didn't understand that , so it went from aviation to space , so the whole space race was God forbid . You know , someone has space overmatch over us . Now it's AI overmatch . That is like this intense race . So it's weird . It's like the space race fueled all kinds of technologies , but it was the fear that they would win militarily .

This is what's driving , I think , a lot of behavior right now .

Are there other dimensions that we're working on overmatch with ? Oh , I mean many .

You'll run rings around everybody . I mean , you'll make it's like being what we are today and dealing with someone from like 1800 years ago .

So once we had a quantum computer with enough bits , our credit cards would no longer be safe on the internet , and the first country , the first organization that does that would probably make billions to trillions before they would let anybody know or could . But I mean I think quantum computers are still . So the AI thing is happening right now .

Right , yes , I mean Rony's , without quantum . Yeah , without quantum . It could be propelled by that . But I'll say one thing that I think is important in this aspect that Rony mentioned , which is the race . In a race , your job is to win , right , and what I think Rony and I keep on saying is I mean , winning here could be really bad .

If you win by having the huge viper that's going to eat everybody up , then you've won the race but lost everything else .

having spent time in China and reading about what's going on there , it seems to me that this would be a tremendous weapon for accomplishing what China wants to accomplish , which is complete control of basically of every activity that human beings do , starting with the Chinese people and then extending it to the rest of us , Because if we're ending up using Chinese AI

, it may have already invented you know , Xi Jinping's values Forlorn of robotics . And that , I think , is really terrifying in my mind .

Maybe Modi does , because he's there was a really interesting podcast with him a couple of weeks back with Lex Friedman , where he talks about India's perspective of we don't need to beat everybody , we have to take care of ourselves but there's also this need to collectively figure things out .

You don't hear a lot of people saying that , but if the intention of countries is to say one of us can dominate the globe , everyone will actually lose , sort of like that early movie War Games from the 80s that anyone who wins loses . So you have to figure out a different kind of equilibrium .

It's kind of like you know , and if you ever see it like , go back to Jurassic Park , like the T-Rex fighting the Allosaurus or whatever , they just trash everything . Nobody's happy with that outcome . So I think there needs to be a different , intentional goal .

They could think 100 years out , they could think 1,000 years out and go . All of these outcomes you guys are planning , they're all bad . Here's a narrow set of outcomes that do not destroy the planet , ruin everybody , ruin the economy . People make moves . They just turn knobs and they like blow up the economy .

They create chaos without thinking because they have short-term twitchy brains . One human brain cannot lead anymore . I'll just make that bold statement .

Think through the consequences of our actions , which you got to do now , because you turn the wrong knobs and there's like no recovery from it . It's like one-way doors that you can't come back from .

They're just harmonizing the world . Exactly so it's . I mean , you'd have to agree on what benign is and you know , between us here we say , yeah , like a golden retriever is kind of benign , but if you're a US commander

of an army , then no , you need vipers .

Now that you know and you can make that into say that's a positive compared to , let's say , the chaos and conflict that we have in the US .

So well , if you're talking very philosophically , I'll give you an answer which is and it goes I forget the name of the movie , but there is a movie where it's at some point I think it's Harrison Ford there and he's on trial and at some point the judge is some like your mother has taught you what being okay is , or I forget the word that they use .

But basically I think that most people deep inside kind of know , and when they don't , then we tend to sometimes lock them up . But maybe I'm wrong . But back to what you were saying , I think the way to work here is to involve various players , and I mean , who am I to decide for everyone ?

And when you mentioned the center , part of the idea so we want to develop a center that focuses on AI and intentions , and the point there is not that , you know , the three of us just sit there and make the decisions for the rest of the world is that we would involve different players from different fields and really try to have different stakeholders come there

and I don't yet have an answer for you , and it could be that they all come together and we realize , no , the conflict is such that there is no way , they're just one . I mean just like you had the Second World War . There was no way to say , ok , we're just going to cohabitate with the Nazis , it's going to be fine .

No one party here seems to have to destroy the other , and I'm really hoping it's not that . But until we really have a frank conversation , I don't know how we define these things that work for everyone .

Once people come to grasp and I love the idea of what you guys talk about like you pull all the different parties and if you can get to that shared understanding that if we don't figure this out , we will all cease to exist , it's not like one of you will and the others . There's too much collective integration that one can't really win anymore .

There's too many nuclear weapons , too many bioweapons . Ai could do things that just wipe everyone out . Once you figure that out and go we do If you pulled everyone on the planet , 99.69 says we would like to continue to exist . If our leaders don't agree with that , get rid of those leaders .

I think that's a shared understanding and then you're like , from there , just the universal document that we would like to exist going forward . I think most humans would agree on that .

And when you point that out , I think it's going to freak a lot of people out and they go . Well , why is this happening ? So now you begin . If you do what you guys are talking about that intention , it's like get the attentions at the top aligned with the wants of , you know , seven , eight billion people . I mean literally .

If you polled seven or eight billion people , how many are going to say we think it's okay if half of us don't exist anymore ?

That is the biggest disconnect right now what the will of the planet wants and a handful of leaders who are not aligned with that will .

You know what ? What do we actually have to do to get people to , to start getting aligned ? Do you think it's going to take some kind of cataclysmic meltdown that people will then have to recognize oh okay , if we don't get our shit together , we're not going to be able to exist as a species .

By the time you go , we'll just unplug it . Well , again , the three seconds that it takes you to think about unplugging it for it , it's a month and a half of planning . By the time you do that , it's going to be plugged Again , assuming this is not the ai that we have right now but a future one almost like a sky , exactly , so well a skynet .

I mean , yes , but if you look at some of the things that ais right now are doing , they are , they are able to , and they do they scheme .

You tell them things like what would you do if I try to , if you have a purpose and something is trying to shut you down so you don't do what it is that you're supposed to do , and the AI tells you things like I will try to . I don't know . We talked about this once .

I could try to alert the media and then , if that doesn't work , I'll try to exfiltrate my parameters out of the company that created me so that I could survive .

And , of course , if you ask it directly and I tried it actually just yesterday so I mean , what they say is I'm like , oh , I don't have self-preservation , but you understand that that was a Google engineer typing that in so that if you ask that question going to come back with that nice answer .

But if you make it act in a certain way , you can see that that if you make , if you create a situation where , well , your self-preservation is important for the things that that that you need to do , then it takes some action .

The pterodactyl is not flying over the ocean

yet . This is that moment where you know what you wanted to do build this intention institute can actually do something . You , what you wanted to do , build this intention institute can actually do something . You still have that moment where you can't . By the way , the League of Nations , un , why aren't they addressing these things ?

They were built a long time ago to do something else . Right ? It's like asking why is my weird two-wheeled bicycle with a giant wheel a little ? It's not able to go as fast as my mountain bike .

Do they see that they have a vested interest in being ?

So it doesn't really matter what they think independently . The push is to get the next model that is acting in more cheaply and that can get the same thing done better than the competitors in some way , because it's a race . And I think again , roni's point about the race is critical and I think that's part of the big issue that's driving it .

And I will say one more thing here . I mean , there are things we can control . I don't think that out of this podcast we can somehow create a world government , right , or something like that .

So this is I mean , if you think about the T-Rex , I mean this would let us know that , oh , this T-Rex is going to try to break the fence , but that golden retriever is not . So the golden retriever is fine , the T-Rex , we have to shut down and reanalyze and understand what's going on .

They have no moral and they just want to play the biggest baddest computer , to build the biggest baddest T-Rex thing . And there is an overwhelmingly larger number of people like that that we do not hire at what we're doing .

But there's a sizable minority I just think of them as the Jedi Rebel Alliance that are really happy to find some subgroup , that are not bought into the like sort of the Death Star view . And again , I don't know if it's like 70 , 80 percent on the Death Star side , but maybe 20 percent , maybe can grow Don't want it .

Now I'll tell you what happens when I'm talking to CEOs and institutional leaders , the overwhelmingly large number of them do not want a Death Star at all . So this is the mismatch . They want a controllable , secure , protected golden retriever . You know , I've probably talked to 150 Fortune 500 CEOs in the last few years .

From that sample size I would say , like 99% do not want what's being offered to them . They don't want this thing . So the mismatch is between they want something that protects their company , grows it , does not decimate their employees , does not decimate humanity .

That's a very positive signal for me that you could influence Fortune 500 , you could influence government institutions , one at a time , and at that local level they actually are aligned with the minority of developers .

But the majority of developers are very happy to work on the big giant desktop because it's a lot of fun , massive computing and crazy stuff , but it's not aligned with what the world or people really want . It's like here , go build this gigantic thing . That's scary , and they don't have to think about it because they're inside the bowels .

You're just grabbing people so security numbers and reading their social media feeds to sell them garbage and influence their behavior . And reading their social media feeds to sell them garbage and influence their behavior . Why don't you actually go cure cancer ? Because that system can be stripped of all the malintended things .

If your intention is cure cancer , put that huge GPU cluster on curing cancer and take all the tens of billions you're raising and put that on the cancer research folks . But they're saying that that's not what they're doing . They're actually using these systems to do other things .

Those other things are just sort of like mine you for your data and mine you for your revenue from you and then not cure cancer .

If you put that whole firepower on nothing but curing cancer and all the money and fund all the cancer researchers , check , problem solved . That's not what they're doing . So that's an intention check . If the intention is what Sam says cure cancer go do that , put all your firepower and you'd do a great thing for humanity .

I mean , when we talk about AI now , we usually talk about these

large language models , the chat , gpts or whatever , and if you just trained something to cure cancer on its own , it would never have gotten as intelligent as that . I'm not saying I completely support this , but I think this would be the argument .

But I think I get your point in saying well , okay , let's say you train it on all this data , what are you using it for ? Are you using it to cure cancer or are you using it to sign for more data ?

So the actions and the intentions are misaligned . Like , if you really want to be doing positive things for science and medicine , you can build a very intelligent system that's boxed in on science and be much more intelligent than something that needs to like . Take Miyazaki's animation work that has nothing to do with curing cancer .

Reading all of our social media feeds and like creating like spam blogs and like you know , replacing the New York Times with like right wing stuff is not going to cure cancer . So again , it's like they're using these systems for other reasons and they have the superficial thing on the top , but most of the effort is not there .

So I think the intention is really important . If you look at , what are the GPUs doing per cycle ? What is the data inside these things actually churning around ? So much garbage is there . So much misinformation is in there . So if you really put your arms around it . I think you can push these companies . Who's the you ?

I think it's the 7 billion people that want the good outcome . Uh well , you know , how do you harness that ? How do you harness that ?

We have a system that seems to be broken in that particular aspect at the moment . Right , the recycling effort , the redirection effort . It's not something that is getting a lot of support from our government right now , but ultimately , I don't know that private industry has the incentive to cure cancer in the way you're thinking about it .

I think we will put other pathways out on the table for the public and for government and for businesses to see that have very the destructive ratio versus the positive ratio is very different .

You're going to show systems that can be controlled , that don't have to hallucinate , that do not have to have all the traits and behaviors of what most people think of as AI . It's just describing one particularly malevolent view of it and it does not have to be the case for a lot of other species .

I think once the public is more and more aware of it , you might see the free market align at what it really wants , like its own self-preservation . People don't want to be eradicated . They don't want all jobs to be lost . There's a number of reasons that people will start to push back .

Clear direction forward in the AI industry now is this focus on AI agents , the idea being that it's no longer you're just going to sit there and ask the I don't know LLM to go over your speech , but you're actually going to ask it to plan your vacation in this or that city . So now it actually has to start interacting with the real world .

It has to buy you a plane ticket and all those things . The real world . It has to buy you a plane ticket and all those things . So that's where a lot of the money is going right now and it seems like that's going to be a direction .

But of course , that means these systems get more and more autonomous and get more and more able to create problems in the real world . It's no longer like a system that's going to tell you that it's going to do something , but it can actually do it .

So my hope is that a year from now , we'll be sitting and doing maybe the next podcast in the center that we're trying to do , that we're trying to make , and we will have this conversation . My fear is that there will have to be some really big bad thing happening before people go , oh shoot , we should have .

And now let's try to think how to make this not happen again . And if you want to finish with a nuclear , well , I was going the other way and saying you want to finish with a nuclear weapons . Look what had to happen before . You were starting to think how do we start to control this , and and so on .

But I really think there is and I agree with roni that right now the t-rex is still it's . We're still kind of growing it . It's not going to run out of the gate .

You know , people are always willing to listen to a story about how to alleviate pain , and I think that the answer to how we get people tuned into all of this right and motivated to think about it is lies in there , somehow .

Right , we may not need the cataclysmic event happening , but we may need the fear of the cataclysmic event happening in order to be able to get people to focus on the pain that would be very obvious to them . So who knows where this is going to go ? I hope we can continue this conversation on the Feudal Future podcast as we move forward .

Gentlemen , thank you so much for a very enriching conversation .

Thank you for inviting us , yeah .