EP 4: Speedrun

Hi, everyone. This is Andy. And before we get to today's episode, I just wanted to say a quick thank you to everybody out there who has been sharing the series. It is because of you that we have been a success. We're very grateful. And I wanted to talk a little bit about our hope for the future of the press and share a recommendation with you. The field of journalism, it is no secret, is one with a checkered history.

You can go back 200 years ago, and Thomas Jefferson was saying that the man who never looks in the newspaper is better informed than he who reads them. And the truth itself, he said. becomes suspicious by being put into that polluted vehicle. Pretty tough stuff, President Jefferson. And we know from polling that right now, again, much of the country shares a version of that sentiment.

with trust in the media hitting a new historic low year after year after year. Here at Longview, we are working hard to earn back that trust. We aim to build a reputation of fairness. and openness, and to do our part to forge a more honest and trustworthy press in the future. And one of the organizations that we look up to, because they're already paving the way forward for that,

are our friends over at Tangle. Every day, Tangle publishes this newsletter and podcast where they summarize the most influential arguments coming from the right, coming from the left, coming from the center. And I think it's a genuine beacon for where our future industry can go. And this is not a paid advertisement. Tangle is legitimately someone who we have collaborated with in the past and we hope to in the future.

And as a part of that collaboration, Tangle is offering a 40% discount on a subscription to all of you who listen to the show. And so if you want to go check that out, please go to readtangle.com. slash invention. Again, that is readtangle.com slash invention. I think you're going to love what they're doing over there. And again, thank you so much for listening. Now, on to the show.

The progressive development of man is vitally dependent on invention. It is the most important product of his creative brain. Its ultimate purpose is the complete mastery of mind over the material world, the harnessing of the forces of nature to human needs. This is the difficult task of the inventor. who is often misunderstood and unrewarded. Does anybody know who wrote that passage? Nikola Tesla.

This is The Last Invention. I'm Gregory Warner. So now for the main event. There are some like Tesla, Edison, the Wright brothers, Ford, Jobs. At the 2017 meeting of the National Governors Association. Rare entrepreneurs who make the impossible possible. All these governors, from red states and blue, came together in a room.

to find out what they could do to prepare for the future. From a man who seemed to be ushering in the future. You know, I'm really thrilled to introduce a man who's arguably the personification of technological innovation. Please join me in welcoming Elon Musk. And the governors are eager to ask him about his plans for Tesla, about

electric car infrastructure, about how to get ready for autonomous vehicles, and even SpaceX flights. They want to know, what does Elon see as the next big tech on the horizon? What would you want? But... No one seems prepared for where Elon wants to take this conversation. I have exposure to the most cutting edge AI, and I think people should be really concerned about it. I keep sounding the long bell, but, you know, until...

People see robots going down the street killing people. They don't know how to react. He tells them the best thing that lawmakers can do to prepare for the future is make sure that humanity has a future. AI is a fundamental risk to the existence of human civilization. AI is a rare case where I think we need to be proactive in regulation instead of reactive, because I think by the time we are reactive in AI...

regulation, it's too late. Because what's going to happen is robots will be able to do everything better than us. I mean all of us, you know. Yeah, I'm not sure exactly what to do about this. And at first, it seems like the governors maybe think he's pulling their leg, but he just keeps going. But when I say everything, like the robots will be able to do everything, bar nothing. Let's move back to your rolling out the Model 3 this year, right? How many orders would it be?

And after a moment of uncomfortable silence, the lawmakers eagerly move on. They never return to the subject. Now, it would not be long...

after this very public warning, that Elon Musk himself was accelerating to build that very technology he seemed so alarmed about. And he wasn't alone. Today... how some of the very people most concerned about artificial superintelligence came to decide, one after another, that the best way to protect the world from this technology was for them to build it.

First. And build it fast. The AI race was started by the people who warned about it. It was started by the exact people. Sam Altman, Elon Musk. Demis Hassabis, Dario Amadei, who said, at least nominally, that they were the most concerned and they wanted to prevent this to happen. They are the exact people who actually brought us into the situation we are in now. And they're still doing it.

So, Andy, walk me through how we get from Elon Musk warning about AI to him trying to build AI, like going from Doomer to Accelerationist. All right, so this all started with a meeting. between Elon Musk and Demis Hassabis. The Demis Hassabis of DeepMind, the child prodigy. Right, the gamer of gamers, the child genius. Back in 2012, Peter Thiel set up this meeting between these two men and...

In the years since, that meeting has become like a Silicon Valley folktale. Like I heard about it from dozens of people who I spoke to for the series. And some of them were saying that if AI becomes even half as powerful. as people think that it's going to, the future will look back at this meeting and see it as some kind of turning point.

You know, it was the Demis meeting with Elon that we sort of brokered. Peter Thiel himself recently told a version of the story to my old colleague, Ross Douthat. The rough conversation was, you know, Demis tells Elon, I'm working on the most. As the story goes, Musk says...

I'm sending us to Mars so that if anything terrible happens here on planet Earth, you know, nuclear war, some kind of civilization ending pandemic, we've got this escape valve. We can actually travel to other planets. Our species can survive. And then Demas said, you know, my AI will be able to follow you to Mars. And then Elon sort of went quiet. And this is a huge trigger for Musk, where he is like...

Who is this guy? What is he trying to do? Why is he telling me that he is ultimately doing something that might kill us all?

that he wants to keep his eye on Demis. And so he invests in DeepMind to keep tabs on the company. So his reaction is to be concerned and maybe a little freaked out and then to say, here's some money. So I know exactly what you're up to. Yeah, exactly. So Muskie becomes an early investor in DeepMind, and then a few years later after the impressive Atari demo. The AI that mastered Space Invaders. Right, yes, without any training. Yes.

Google quickly jumps in wanting to acquire DeepMind. Elon actually tries to get in the way of that and buy DeepMind himself, but it doesn't work. And when Google acquires DeepMind, that's the moment where suddenly... Elon starts going out in public and really sounding the alarm about what he believes are the existential dangers of AI. I don't think most people understand just how quickly machine intelligence is advancing. Mark my words.

AI is far more dangerous than nukes. I think that's the single biggest existential crisis that we face. Keech Hagee from The Wall Street Journal, she says that this is what inspired Elon to start going out and trying to lobby... lawmakers, and President Barack Obama. Elon even has a meeting with Obama, and it was kind of interesting because there was a sense that, yes, Obama understood the risks, and yes, he also understood...

how important AI was going to be for the economic development of the country. It promises to create a vastly more productive and efficient economy. He gave an interview to Wired around this time, you know, saying if properly harnessed, can generate enormous prosperity for people, opportunity for people, can cure diseases that we haven't seen before. But it could increase inequality. It can suppress wages. And so we're going to have to develop new social constructs in order to embrace fully.

And yet, Elon left that meeting with a sense that Obama wasn't really going to do anything about the existential risk piece. Elon, he doesn't just strike out with... President Obama, he's also talking to Vanity Fair. He's talking at colleges. He's speaking at conferences. Remember, this is the Tony Stark era, Elon Musk we're talking about here. Right. This is a time when Elon Musk had...

pretty broad market appeal to lots of audiences, especially on the subject of technology. This is Elon at his peak celebrity pre-controversies that would follow. But even for him... He feels like no one's taking him seriously. And so he starts to host these dinners where he would invite other tech leaders, sometimes other billionaires, and they would get together and try to brainstorm away.

that they could stop Demis Hassabis in Google from making some sort of civilization-ending AI. All right, so first he tries to buy DeepMind, fails at that. Then he... Goes to the leader of the free world, tries to warn him. That doesn't work. Goes to the press, gives some speeches. Then it's themed dinners, where basically the theme is, how do we save the world from superintelligence?

That's pretty much the story, yes. And one of those dinner guests was none other than Sam Altman. It is my belief that in the next few decades, someone will build a software system that is smarter. and more capable than humans in every way. And that very quickly it will go from being a little bit more capable than humans to something that is like a million or a billion times more capable than humans.

Who was Sam Altman at this time? Sam Altman was the president of Y Combinator, which basically meant he was like the king of Silicon Valley. Keech Hagee actually wrote a biography of Sam Altman called The Optimist, which is really good. I recommend people check it out. And she says that by 2015, Altman was already almost this mythical figure in Silicon Valley.

helped to turn companies like DoorDash, Instacart, Airbnb into household names. What's Sam Altman's superpower? How would you sum up what he's so good at? Sam Altman is... a once-in-a-generation fundraising talent. He's an incredible storyteller. He can convince people that he can see the future. He can sort of summon companies into being just by persuasion.

He is also kind of a fixer with lots of relationships all around Silicon Valley, people who owe him favors and could sort of make anything happen in Silicon Valley that anyone wanted. It turns out that since Altman was young, he had always been enamored with this idea of making a true AI thinking machine. But by the time he's having this meeting with Elon Musk...

He had read the book Superintelligence by Nick Bostrom, and he had come to believe that if AI was made irresponsibly, that it could possibly lead to the end of the human race. And he even began blogging about this idea that if AI happens, it could be the most consequential thing that ever happened to humanity, but it could also be dangerous. So when he goes to this meeting with Musk... Altman starts...

talking to him about this idea that he is also very, very worried about AI potentially going wrong and becoming an existential threat to humanity. He pitches him on the idea. that if you want to stop a dangerous AI, if you want to stop Demesis Abbas, if you want to stop Google, then what we need to do is make a safe AI before they make a dangerous one.

What do you think about the idea of us creating a lab that counters Google? Why don't we make a lab that... would create the same technology, the same AGI technology as a counterweight to Google, except it would be non-profit, it would be open source, and it would be for the benefit of humanity. And Elon says, great, let's do that. Sam basically convinces Elon to bankroll this thing. And by the end of the year, they have created OpenAI. We started a group called OpenAI.

It is a nonprofit. The goal is to build general super AI for the benefit of humanity. OpenAI is structured as a 501c3 nonprofit to help spread out AI technology so it doesn't get concentrated in the hands of a few. What is going to be your sort of biggest differentiator then, like OpenAI versus like the megacorps? I hope that our biggest differentiator is, number one, we do the best research in the world. And number two, we care the most about...

how it gets deployed. Okay, so OpenAI, the company behind ChatGPT, came out of this plan to stop Google by beating Google at their own game. But do it...

in a totally different way than Google was doing it because it was going to be non-profit? Right. The idea is to create almost like an anti-Google. They called it a non-profit research lab. They didn't even call it a tech company. And at the core of that lab... is this mission that not only are they going to make the super mind, the AGI, but that they are going to ensure that this thing is good for the entire planet. Right. Okay, so when they start OpenAI...

What's it like at first? So the first thing that's quite interesting is in order to pull off what they wanted to do, they needed to recruit talent. So they needed to break up Google's monopoly on AI. research talent. And they used their nonprofit ethos and this mission-driven idea. to very effectively poach a bunch of researchers from Google and then also bring a bunch of new PhD grads into the founding team. And I remember even with Google purchasing DeepMind.

Most people in technology still don't really buy into the idea that AGI is coming anytime soon. And so the people that primarily ended up joining OpenAI were self-selected. So-called AGI believers, people that were there for the crazy quest to try and recreate human intelligence. And they were of one of two.

There were the people who were AGI believers but doomers that were really focused on the AI safety orientation of we're ultimately trying to recreate this thing in order to prevent existential risk. And there were the accelerationists who were like, we believe in this thing because we think it's going to bring us to utopia.

They were both there together in this one lab working on this project. They were both there together in that one lab. And at the time, they philosophically did not seem that different because they were compared to the rest of the field, which just did not. think that this idea of creating AGI was really something that held water. The doomers and the accelerationists are just two sides of the same coin. They both...

belief in AGI. There were only so many of them. So they were all kind of banded together based on that shared belief and excitement and fear around doing this journey together. Essentially, they were sending out this signal to the world of technology. And in response, they end up actually bringing together a really fascinating mix of people. They were able to poach Ilya Suskever.

one of the guys behind the ImageNet win with Hinton, from his job at Google. They get Greg Brockman to join them from Stripe. They eventually bring in this guy, Dario Amadei, who had also worked at Google. And these were people leaving... big paying, very stable jobs in the world of technology to come work at this new research lab because, as they said it, they truly believed in this mission and how important it was.

Okay, so they walk away from these big paychecks, these stable jobs, and they build what? What do they begin to make at OpenAI? Well, at first, Elon is very excited about the idea that they should make an AI that's going to go head-to-head. in some kind of game against Demis. Legendarily, Elon Musk is a gamer, and so he's pushing him down that path, but...

There's also just this kind of looseness, right? They're a research lab, so there's this sense of, like, let a thousand flowers bloom. Like, what path might lead to AGI? We don't know. Let's try this one. Let's try that one. But after months and months of this...

Without any real meaningful progress, suddenly Dimas Hassabis strikes again. Now the wait is almost over. In less than one hour from now, Man will face off against Machine in an epic game of Go. The competitors are Grandmaster Issei Doll. In 2016, Dimas and the team at DeepMind, they thundered back onto the public stage again.

This time to play the game Go. The game of Go is the holy grail of artificial intelligence. For many years, people have looked at this game and they've thought, wow, this is just too hard. Everything we've ever tried in AI, it just falls over when you try the game of Go. And so that's why it feels like a real litmus test of progress. If we can crack Go, we know we've done something special. Are you familiar with Go? I was obsessed with Go as a kid.

I remember, because I didn't really get into chess, although I got into it briefly, but Go was like, the stones were very beautiful. The pieces are just these black and white stones. What I think is cool about it is that it is an ancient Chinese game. And I was looking it up and it appears as if like we don't even know how old it is. Like there's records of people playing Go.

Like 2,000 years ago. It's older than chess, though, right? Way older than chess. And what's crazy about it, too, is that you can be playing for a while and not even know who's winning. It's that complex. Jasmine Sun, who was one of the tech writers that I spoke to about this. She told me you could play go every day of your life.

And you would never play the same game twice. The game of Go is like an ancient Chinese game that is known for having more possible board positions than the number of atoms in the universe. You're saying you can't even calculate. the positions. It's unfathomable. It's literally unfathomable, right? There's no way through some sort of brute force search that you can just search every possible move and compare them all against each other. You can't do that.

memorize the strategy. This was a game that pretty much no expert system could have hoped to truly master, which is exactly why Demis Hassabis wanted to create an AI. But another way that it's really different from chess is that... Go players, because there is no way for them to really calculate their way to victory, the ones who become masters of this game are often described as having some sort of deep instinct.

Or often they use the word intuition. Champion Go players are known for being really intuitive, I suppose, for having some sort of like deep feel of like strategy and the board that you cannot learn by like memorizing any sort of rule book.

If you ask a great Go player why they played a particular move, sometimes they'll just tell you it felt right. So the one way you can think of it is that Go is a much more intuitive game, whereas chess is a much more logic-based game. A lot of games players call Go a very... quote-unquote, human game. So Demis and DeepMind, they create this AI system called AlphaGo. And the way that they train it sounds like sci-fi. And it actually...

plays into one of the big fears that the Doomers have about how AGI might one day turn into ASI and, you know, replace us all. And it starts off like this. So at first they just... loaded up with a whole bunch of data of human beings playing Go so that it can find its own patterns and see, oh, that's working for this person, that's working for that person. But again, like...

Go has more possible board positions than the number of atoms in the universe. There are so many moves and positions that no one has ever thought of yet. But then what they do is they make a identical copy of the AI system. So that the AI can play against itself millions and millions of times, each time learning new strategies and gathering more data and learning new strategies and gathering more data.

The self-play is what they call it. This is like the thing that really makes the system not just like quite good but superhuman at playing Go because it's able to put in so many reps, like an infinite amount of practice that no like ordinary player ever could. Interesting. So that makes me think of that idea from Malcolm Gladwell, the 10,000 hours thing, like it takes 10,000 hours to become an expert in something. But this thing can...

basically log the equivalent of 10,000 hours of practice in like a month or something. It's even crazier than that because AlphaGo can play itself so quickly that in the span of a week, it could play more... than a human could in centuries. Wow. Like hundreds of years worth of nonstop playing in a week. Hello and welcome to the DeepMind Challenge Game 1, Round 1, live from the Four Seasons here in Seoul, Korea.

In March of 2016, they set up this showdown against the player Lee Sedol, who is often described as the greatest player of his generation. It garners all this attention, more than 100 million viewers. All these journalists are there. Everyone in Korea is watching. The game is huge there.

Everyone here is very excited for the match of the century. Journalists from Asia and around the world, about 350 members of the press, are here to see if artificial intelligence can really beat human intelligence. Now, I should say just a level set here that pretty much everyone thinks that Lee is going to win. Even Google believes that their system is very likely to lose.

Dimas Hassabis himself, he later said that the team was given just a 5% chance of pulling this off. Which is interesting because even though chess had fallen to AI and Jeopardy had fallen to AI, Go was seen as just too hard or maybe too human. Yeah, it's too something. Lee is too good.

The game is too complex for an AI. And maybe even just like it's not ready yet. Maybe this like one day it could happen, but not today. And in the middle of one of the matches during Move 37. as it's called, AlphaGo ended up doing this weird thing that seemed to prove the doubters may have been right all along. Interesting. AlphaGo played this move, which I want to hear more about in a second, but...

Lee has left the room. It makes a move on the board that to all of the spectators, to Lee Sedol, looks almost like a mistake. It's a very surprising move. It's a surprising move. I wasn't expecting that. I don't really know if it's a good or bad move at this point. Like the DeepMind AlphaGo team, they were watching it and even they thought it had made a mistake because it seemed weird. It seemed bad. It was something that...

No human player would ever do, had ever done in a position like that. So while the cameras are rolling, while the world is watching, the DeepMind team is like, damn, did this thing just malfunction? Like it glitched. Yeah, like a glitch. The professional commentators almost unanimously said that not a single human player would have chosen U37. I can't believe what I see right now. And yet in the end... I think he resigned.

Oh my gosh. AlphaGo wins. Yeah, Lee has, I'm getting word, Lee has resigned. In the battle between man versus machine, a computer just came out the picture. Each mind put its computer program to the test against one of the brightest minds in the world and won. AlphaGo beat a professional player who has... 18 GO! World Championships under his belt. And then suddenly people start to go back and re-examine that move. Move 37. It went beyond as human.

guide and it came up with something new and creative and different. And they realized that it was... the turning point in that match. That move was what got everyone to say in the Go community, in the broader community, even Lee himself to say, oh, the computer, like this machine.

can be creative, it can be intuitive, it can sort of like master this thing that I always thought was a human task. The more I see this move, I feel something changed. Maybe he just can show humans something we never discovered. Maybe it's beautiful. So the AI played a move that no human had ever made in any at least recorded Go game. And that means it discovered its own original strategy?

Some people would go as far as to say that it had something like an original thought, an original idea. And what do we know about how it did that? How did Demis Sabas and his team at DeepMind explain Move 37? Well, they really wanted to know. And so they spent time digging through the code and looking inside the guts of the system. They wrote a paper about it.

And while they were able to, you know, glean some information, because it's one of those connectionist neural net AI toddler styles of AI. Right. Remember the trade-off that Yoshua Bengio was talking to us about to get this kind of... impressive performance, this level of intelligence. You just have to accept that you're not going to get satisfying answers to these kinds of questions. You have to accept some level of mystery.

This is the black box. And when I asked Yahshua Bengio about this AlphaGo moment. With AlphaGo, I thought, ooh. Now we're getting close to something important. He said this is when he realized that AI was now entering a whole new era. And he wasn't the only one. All across Silicon Valley, across the world of technology. People were singing Demis' praises. People were abuzz about AlphaGo. And of course, yet again, Elon Musk doesn't like this one bit.

DeepMind's show of force in AlphaGo freaked Elon out a lot. made this sort of ambling approach that OpenAI had at the beginning of, you know, let a thousand flowers bloom. All the researchers can kind of pursue their own different areas. It made him have little tolerance for that. We now know in some detail what happened inside of OpenAI during this time because a number of their internal emails were revealed as a part of a lawsuit. And so you can see in these emails, Elon Musk

telling Altman and the leaders at OpenAI just how frustrated he is that they're losing to Dimas still. In one of them he says, OpenAI is on a path of certain failure relative to Google. There obviously needs to be... immediate and dramatic action or else everyone except for Google will be consigned to irrelevance. What does immediate and dramatic action mean? Well, according to Keech Hagee, this is when...

Musk was saying, where is our game player? Elon wanted to drive like fighting tit for tat with DeepMind. And he really wanted to respond to it by... showing an even cooler and harder game that AI could beat. We need to challenge DeepMind in some kind of public display of our dominance in a game with an AI game player. And Altman...

and the team at OpenAI, they were saying to Musk, well, we do have a way that we think we could beat DeepMind. We've got this strategy, but for us to implement that strategy, we're going to need way more compute power and we're going to need... more money. So they pitch him on this idea that the nonprofit OpenAI could have a for-profit arm, and then that way...

Altman could go out and do the thing that he's best at, right? He could go get investor money that they can use to up their compute power. They start discussing, how are we going to convert the nonprofit into a for-profit? And all of a sudden... Musk and Altman start butting heads, because when they go to form the for-profit, the question becomes, who will be the official CEO of the for-profit? And both of them want to be the CEO, and they cannot agree.

And Musk comes back and basically says, no way. I gave you all this money to start a nonprofit. If you're going to turn that nonprofit into a for-profit, then I should be the head of that. He even thought about... folding it into Tesla and just making it an arm of a for-profit company he was already running. He wanted to be CEO and have controlling voting power. So Elon says if it's going to be for-profit, then he's going to be in charge. Yes, or he's going to walk.

And so now OpenAI has a decision to make. Lose Elon Musk and his celebrity and his money and his tech prowess or change the structure of their company that they designed. specifically not to have one person be the ultimate controller of this technology that they think is going to be so powerful that no one man should wield it. Right? Right. One OpenAI co-founder, Ilya Seskova.

He wrote to Elon in this email and he says, quote, The goal of OpenAI is to make the future good and avoid an AGI dictatorship. You are concerned. that Dimas could create an AGI dictatorship. So are we. So it is a bad idea to create a structure where you could become a dictator if you choose to. And so what does Elon do? Well, he responds to this email, guys, I've had enough. This is the final straw. And then Musk decides in a huff.

If this is not going to stay a non-profit and it's converting to a for-profit where I am not in total control, I am leaving. And not long after, he quits OpenAI. But within a few months... He starts going around on a very different kind of campaign, essentially telling people that he has also quit trying to sound the alarm about AGI and what it's going to give rise to.

Four, three, two, one. Boom. Thank you. Thanks for doing this, man. Really appreciate it. You're welcome. Very good to meet you. Nice to meet you, too. And thanks for not lighting this place on fire. You're welcome. And it's at this time that Elon goes and makes his first appearance on the Joe Rogan experience. This is the infamous episode where Musk smoked pot on camera. Yes. I mean, it's legal, right? Totally legal. Okay.

There's tobacco and marijuana in there. How does that work? Do people get upset at you if you do certain things? And as you remember, this became like a whole big thing. The stock value of electric car manufacturer Tesla tumbled 9% Friday morning. Billionaire Tesla head Elon Musk. What is he up to? Shares in Tesla took a hit today shortly after video was posted of CEO Elon Musk.

apparently smoking pot. But of all the spectacle in this moment, there was this other part of the podcast that even I didn't really notice until I went back recently and listened to it. You scared the shit out of me when you talk about AI, between you and Sam Harris. I realized, like, oh, well, this is a genie that once it's out of the bottle, you're never getting it back in. That's true. Are you honestly, legitimately concerned about this? Is, like, AI...

one of your main worries in regards to the future? It's less of a worry than it used to be, mostly due to taking more of a fatalistic attitude. Musk basically tells Rogan, hey, I did my best to warn people. I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. This seems like a scene in a movie where the robots are going to fucking take over and you're freaking me out. Nobody listened. Nobody listened. No one.

You met with Obama and just for one reason. Just to talk about AI. Yes. I met with Congress. I was at a meeting of all 50 governors and talked about just AI danger. And I talked to everyone I could. No one seemed to realize where this was going. After a short break, OpenAI, now without Musk, stumbles into a breakthrough that will transform the industry and, yet again, make the people most concerned about building AI safely.

decide that they need to build this even faster. Stay with us. The last invention is sponsored by Ground News. Ground News is one of the most helpful tools that I use to avoid the echo chambers and media bias online, especially when it comes to shining a light on our blind spots.

So whether you're politically on the left or the right or somewhere in the center, the blind spot feature from Ground News highlights the stories that tend to be disproportionately covered by one side or the other. As an example, take these two stories about President Donald Trump. One, which had low coverage among left-leaning outlets, It's a very important relationship. We're going to get along good with China. reported that

Trump says U.S. will accept 600,000 Chinese students as part of a trade deal. I hear so many stories about we're not going to allow their students. We're going to allow, it's very important, 600,000 students. And another largely uncovered by right-leaning outlets. Trump's social media company is using Crypto.com's... Trump family crypto empire expands with Crypto.com partnership.

That's our transactional Trump family. Make some money when you can. By seeing which stories are amplified or ignored, depending on the outlet, Ground News helps you step outside the filter bubbles that shape most people's news diets, giving you a fuller picture of what's actually happening. I really think that if you like this podcast, you're gonna like their mission. Go to groundnews.com slash invent to get 40% off the same unlimited access vantage plan that we use.

You can even sign up to stay up to date with the biases in your coverage proactively with the weekly blind spot report delivered directly to your inbox. This is a great way to support them and the work that they do because Ground News is a subscriber supported platform. We appreciate what they're up to. We appreciate their support for this podcast. So go check them out and make sure to use our link groundnews.com slash invent so they know we sent you.

This episode of The Last Invention is brought to you by FIRE, the foundation for individual rights and expression. There's a pattern that you can trace throughout history. In ancient Athens, Socrates was put to death for asking tough questions of the powerful. Centuries later, monarchs banned and burned books they considered dangerous. And in the last century, authoritarian governments shut down newspapers, censored broadcasts, even jailed their critics. The struggle was always the same.

Who gets to decide what people can know? Today, that struggle is playing out in a new arena, and the risk now is subtler. Search results that quietly vanish. Recommendation engines that steer us towards safe and comfortable answers. and AI filters that can suppress ideas before we ever even see them. That's where FIRE comes in. FIRE has spent decades defending free inquiry on our campuses, in the courts, and in our culture.

And now through a $1 million grant program in collaboration with the Cosmos Institute, they're supporting projects that keep free thought alive in the era of AI. Join us today at thefire.org slash thelastinvention. By supporting FIRE, you are protecting the future of free inquiry in America and ensuring that tomorrow's most important questions can still be asked. Once again, visit thefire.org slash thelastinvention. And thanks.

The Last Invention is sponsored by Cozy Earth. Around the holidays, I always look forward to the same moment. When everything slows down, the lights are twinkling, there's a blanket, a movie, and some time where no one is rushing anywhere. It's my favorite part of the season. And this year, Cozy Earth has basically turned that moment into a gift. Their bubble cuddle blanket is exactly what it sounds like. Incredibly soft, textured, and luxurious without being fussy.

it'll become the go-to blanket for all your cozy moments. For fireside nights, for lounging on the couch, or for curling up and watching The Grinch with your kids for the 10,000th time. They also make these bamboo pajama sets that are lightweight and sleep much cooler than cotton. It's one of those gifts that people actually end up using every day, long after Christmas morning has passed. For the people in your life who are impossible to shop for, that don't really need anything,

you're giving them an excuse to slow down and enjoy the moment. And one of the best parts, it's risk-free. Cozy Earth gives you 100 nights to try their products and a 10-year warranty. Although, once you feel them, you won't be sending them back. Right now, Black Friday has come early at Cozy Earth, and you can stack our code INVENTION on top of their site-wide sales for up to 40% off. These deals won't last, so start your holiday shopping today.

And if you get a post-purchase survey, tell them you heard about Cozy Earth right here on The Last Invention. Wrap the ones you love in luxury with Cozy Earth. Okay, so once Elon Musk walks out, what happens at OpenAI?

Well, it turns out that even though this was a nightmare for everyone at OpenAI and they were worried that this might spell the end of the company, it kind of turned out to be a blessing in disguise. I talked to Andre Karpathy. who worked for both Elon at one point and OpenAI. And he said, you know, in the beginning, OpenAI was trying to copy paste DeepMind. And in the end, it turned out that DeepMind had to copy paste OpenAI.

As Keech Hagee was saying it to me, pretty much the whole time that Elon was at OpenAI, he was pushing them towards this AI game player strategy. And after Elon left in 2018... Over in the corner, a completely different researcher had a breakthrough with a completely different technology, a language model. It was only after he was gone that instead they focused eventually on language.

And that is how by 2022, they flip everything and have Demis and Google chasing after them instead of the other way around. The next generation of artificial intelligence is here. It's called ChatGPT. But before that would happen, there was another split inside this company. And it's a split that some people in Silicon Valley think... may end up being far more consequential even than Elon Musk's. And this is the paradox of Dario Amadei. Who is Dario Amadei?

Why does he end up at OpenAI? And what is it that he really contributes to the team there? Dario is, like Demis actually, has a neuroscience background, which is one interesting thing. He is also... What he was really known for at OpenAI was his emphasis on safety. Jasmine Sun and Kevin Roos. They're both working on a book right now about AI, and Dario is one of its central characters.

He is, by his own admission, kind of a nervous person. And he was really a pioneer, not just in developing AI systems, but in worrying about them and how they might go wrong. He does end up being extremely concerned about AI risk and the potential for systems much smarter than us to develop their own goals become...

unaligned with, opposed to, or just sort of not caring about human goals and then to sort of end up taking over or screwing humans up. So Dario was one of the people who came to open AI as someone whose motivation was more about stopping a dangerous AI. Yeah, he's very worried about superintelligence. He's associated with this group called the Effect of Altruists, and he says that he comes to open AI in large part because of this altruistic, safety-focused mission that they have.

And how do you sum up what it means for someone like Dario to study AI safety? Like, what exactly is AI safety? So, AI safety is a big field that contains a bunch of different subfields. One of them that... Dario and his colleagues have been very instrumental in is called mechanistic interpretability. That's a very long name. I've told them they should rebrand it to something people can actually pronounce if they listen to me. But basically...

Mechanistic interpretability is the science of figuring out how AI models... make decisions, why they behave like they do, what is going on inside the guts of the system. So he's studying the mysterious AI black box. Yes, making that interpretable to humans. Okay, so he's trying to probe the AI to figure out why it's doing what it's doing, and is it going to do anything we don't want it to do? Yes, that's one part of AI safety. There's other parts of it too.

The other thing that Dario is really into is size. Early on in his career, Dario Amadei had worked on a... project at Baidu, the Chinese internet conglomerate, that dealt with these so-called scaling laws. And this was a theory at that time, it was an unproven theory, that Basically, the key to making an AI system more intelligent was just making it bigger and training it on more data.

countercultural in AI research at the time. Lots of people were theorizing that you needed some clever new algorithm or some very different architecture to make these models smarter. But Dario and his colleagues sort of had this... idea that you could actually just make them bigger and the systems would get smarter. And so the idea here is just that if we take an already promising neural network AI system and we just make it bigger, then maybe...

like the human brain, which is bigger than the bird brain or the cat brain, is smarter, this thing will also get smarter and smarter and maybe one day will even become a general intelligence. Yes. And essentially, this is sort of his best guess at how... companies like OpenAI are going to get more intelligent systems. It's not by training them on more specialized data. It's not by coming up with clever efficiency.

hacks. They are just going to make the models bigger, and that is going to take care of a lot of the problems. So his theory is that if you just take a promising AI model for OpenAI that... became their language model that is a neural net that looks for patterns in text and language. Then what you just need is a massive amount of text and data to pump into it.

as well as a massive amount of GPU computer chips, which are, of course, very expensive. And this is why they're pretty sad to lose Elon Musk and his money. Yes, this is one of the reasons that it was sad to see Elon walk out the door. But not long after he does, Sam Altman goes out. He does the thing that he's so good at. He knows that they need lots of money, lots of GPUs. So he goes out and strikes up a partnership with one of the biggest companies of all time, Microsoft.

So Microsoft ends up fulfilling both of these things. They become the largest investor in OpenAI, and they partner to build the supercomputers that OpenAI needs. Greg, this is a detail of the story that's especially wild to me. When Dario and his team at OpenAI, when they get access to these Microsoft supercomputers, they decide to take their scaling theory as far as they possibly can. Dario Amadei was...

really pushing for the idea of, no, we really go big or go home. So, for example, at DeepMind, when they did that Atari demo, they were using just one GPU. For the richest universities in the world, like MIT, Stanford. It would be a big deal to have a few dozen chips. And in places like India, you would have grad students, multiple grad students sharing one computer chip. So they're...

trying to do their research on fractional amounts of GPUs. But with this Microsoft partnership, OpenAI now has access to these supercomputers with thousands and thousands of GPUs. And so as the story goes, Dario approaches the leadership team at OpenAI and says, guys, what if next time, what if we take this model and we crank it up?

to 10,000 GPUs. So 10,000 computer chips. I mean, no one had ever thought about that before. Like, that's bananas. Actually, like, within OpenAI, this was a contentious... decision because some people were like, is that even possible? That just seems improbable. Other people were like, no one has ever done this before. And if we think that AI could go badly.

Maybe we should more gradually scale it, not just do this dramatic step change. But his philosophy was we need to accelerate the development of this technology so that we can then. retain hold of it and figure out how to perfect it in the lead time that we have over other potentially bad actors getting a hold of it.

And Sam Altman also really liked the idea because his entire career has been adding zeros to things. So he was like, let's do it. And Ilya Sutskever also philosophically was always more in the camp of scaling will. potentially bring wondrous and potentially terrifying things but we should not be afraid to go in that direction and so

the main people that were running OpenAI all converged on, yeah, let's give it a go. However, if you are going to massively scale up your GPUs, your compute, you also have to massively... scale up the data that it is searching for patterns inside of. Because just think of it like, no matter how smart and powerful it is, if it only has access to a limited amount of data...

It's never going to truly become like an artificial general intelligence. Or it's almost like if Einstein, as smart as he was, if he'd only ever read one book. He would know that book really well. Right. But he wouldn't be that smart. Yes. And so they're going to need a massive amount of data to match the massive amount of compute. But here's the problem. There's only so many open source free databases on the internet. And so here's where OpenAI does something.

that right now has a lot of people, a lot of different corporations, suing them, including the New York Times, because it appears, and some former OpenAI employees have leaked some details about this, that they just started dumping big chunks of the internet into their AI. I believe this is going to be the pinnacle scene in the inevitable Hollywood depiction of this story one day where they're just...

ramping up all the stuff that they're throwing into their system. Like, ooh, here's a free database. Let's put that in. Oh, look, let's put some Reddit in there. Yeah, and then, hey, how about Wikipedia? And then, hey, man, these researchers over at the University of Toronto just scraped all these books off the internet. I'm sure they wouldn't care if we just took all them copyrighted books and fed that to the LLM. No problem, right? What else we got?

Yeah, that's pretty much what happened. Allegedly, they started throwing in scientific journals, news articles, blogs, transcripts from YouTube videos. They just kept going and going. And how far does this go? Do they feed it the whole internet? Like I said, there is a lawsuit that's happening right now. So we're going to learn more details, I think, as information comes out from those suits. But it's been reported.

That basically, if a website or if some text online didn't explicitly have a label on it saying, do not use this to train your AI, they adopted a stance of... better to ask for forgiveness instead of permission. So all we know is that they just dumped a lot of the internet in the system. We don't know how much or what exactly they put in there. Yes. And we know that this is eventually...

the strategy that would give birth to what we now call ChatGPT. And so Dario is both the guy who is saying, let's... scale this thing up further and faster than anyone has before. Let's crank this up to 10,000 GPUs. But he's also the safety guy. Isn't there a tension? between those two, between let's crank the knob up to 11 and, ooh, we really need to make sure this is safe? Yeah, this is sort of the classic paradox of Dario Amadei and of AI safety in general.

is that they, on one hand, fear the effects and implications of these very large, very powerful models, and they're trying to... build them and stay on the cutting edge of AI capabilities. And I've asked Dario about this before, and he says, In order to be able to study the safety challenges of very powerful AI systems, you have to have very powerful AI systems to use as your testing grounds. You can't sort of...

learn about safety on a Formula One car by practicing on like a jalopy of a 10-year-old Honda Civic. It just won't teach you that much about what kinds of risks are going to take place. when AI is very powerful. And so the argument here is that to make a powerful AI that is safe, that is good for humanity, you are going to need to learn about...

powerful AI systems and test powerful AI systems. And so therefore, you're going to have to make one. Yes. If you want to do cutting-edge AI safety research on very powerful systems, you need to actually build those powerful systems. I think the other thing that like a Dario Amadei would say is like, whichever system is the best, it's going to be embedded in every part of society. Like we're going to use it to make.

decisions about who to give a loan to. We're going to use it to plan our cities. We're going to use the super intelligence to maybe even figure out our military strategy. The only way to have the impact that we want to have in the world to ensure that we have superintelligence and that the superintelligence does things like curing cancer instead of screwing us all over and self-sabotaging ourselves is by...

having both the safest and the best model. Because if ours is safest, but it's not actually a very good model, then the unsafe model is going to be the one that's going to be widely deployed, and that's a much worse world. And the last one I've heard him make is... the only way to stop a bad guy with a powerful AI is a good guy with a powerful AI. Essentially, this is the argument that this technology, it's so powerful and so lucrative that someone is going to build it.

In Dario's mind, that someone could be an authoritarian government. It could be a rival AI company that doesn't care as much about safety. It could be a... terrorist group. And so the ethical thing to do in his mind, if you are concerned about the power of these AI systems is

for you to be the one who builds it and keeps it safe and kind of sets the high bar of safety that the rest of the industry will have to follow. And it was this mindset, this idea that to make AGI safe, you need to make it fast. This is part of what drew Dario to open AI in the first place. This is a part of the mission that he loved. However, one day, late in 2020, he just up and quit.

along with several members of the AI safety team, and they went out and pretty much immediately started a rival AI company called Anthropic. All right, so... Kevin Dario Amade has yet to accept my interview request, although the people that he works with are very nice. And I had a nice meeting with him. And maybe one day he will come on the show. But in the meantime, I know you've spoken to him.

What do you understand is the reason that he leaves OpenAI? What does he see there? What is it he doesn't like? So the official story of Why Dario... is that they had philosophical differences about AI safety approaches and priorities that Dario and his team wanted the company to... put more emphasis on safety and that others at the company were less interested in that. I think the real story is more complicated and involves a lot of not only philosophical differences, but also like real.

I've heard about in reporting about specific decisions they were making, whether they were taking safety seriously enough, whether they were... becoming too commercial. You have to remember that when Dario joined OpenAI, it was a research nonprofit. It was specifically set up not to be and act like a normal AI company.

By the time he left, it had started this for-profit subsidiary. It had struck this deal with Microsoft. It was starting to look more and more like a kind of normal tech startup. And I think that made him and his colleagues very uncomfortable. And there are some other juicier stories that I'm going to save for my book. So we don't know what Dario saw that scared him? No, we don't know. I mean, maybe Kevin knows something and we'll just...

have to wait for his book. All we know is that he quits. He says that it's connected in some way with AI safety and that he opens a competitor claiming that now he's going to be the one.

to make AI truly safe. And so now we have more competitors in the race. Yes, and this pushes everyone to work even faster. And really where you see that most dramatically is around ChatGPT. Because OpenAI had already decided that eventually they wanted to release a version of chat GPT to the public. But they didn't think it was quite ready.

It had gone from GPT-3 to GPT-3.5, but it was still buggy. It still regularly had these hallucinations that they didn't understand. So they were trying to hit their benchmark of GPT-4. before going public with their chatbot. But suddenly this rumor starts to spread within the company that Anthropic also has a chatbot and they might release it soon.

Allegedly, this rumor starts to go around OpenAI that Dario, Amadei, and Anthropic are planning to release their own chatbot and to do it before OpenAI can. And so OpenAI executives make a decision. We are not going to wait for the GPT-4 launch because the model's just not ready. But we have the chat interface and we have GPT 3.5. And so they're nervous that if Anthropic beats them to market with their chatbot...

Then open AI is going to seem like they're behind the ball. They're going to come off like a copycat. They're operating under, you know, a very Silicon Valley belief of winner takes most. So you need to be the number one. You need to be the one that has. the name recognition, the one who invented this kind of chatbot. And so they just decide, you know, let's do a low-key, you know, no press release, no advertising, no social media blitz release of...

chat GPT 3.5. And Keech Hagee and Karen Howe, they were telling me that supposedly back at OpenAI, The team did not think that this was going to be a very big deal outside of Silicon Valley. Like, they didn't think it was going to make a very big public splash. And so why release it if they didn't think it was going to be a hit? Well, in some ways it was like insider signaling.

To just say to the world of technology, we were here first. It doesn't matter if the public uses it or not. It mattered that the world of technology doesn't think that they are just copying off of their rival's anthropic.

Inside the company, it was a low-key research preview is how they described it. Let's just release this model with this new interface that is just like a chatbot and see what people think. The night before, they were like... making bets on how many people would actually start using the model in I think the first weekend. And the highest bet was 100,000. So that's how many users they provisioned their servers for.

And so on November 30th, Sam Altman goes onto Twitter and he just writes, today we launched ChatGPT. Try talking with it here. And he pastes a link. The next generation of artificial intelligence is here. The future is now. The internet's going crazy over new artificial intelligence called ChatGPT. A new artificial intelligence chatbot. ChatGPT is like a Google you can ask to do things. They can answer essay questions, write songs. Who knows what companies and...

And ethical issues. It already has more than a million users. Very creepy. A new artificial intelligence has gone viral. Probably the area. And I caught the eye of Wall Street this week. Or questioning whether chat GPT might be the threat to some established big companies. Next time on The Last Invention. I'd love it if you could just take me back to this time period in your life where after years of being on the fringes, as you've described it, being rejected.

You and Hinton and your fellow connectionist AI researchers, your contrarian views are proved right. And then all of these actual AI systems, these promising new technologies are born. out of you guys' determination to chase after this idea, despite all that they sang, how did that feel? Like, I imagine it felt really good. Oh, yeah. I mean, it was great. Let me share something emotional. So shortly after AlphaGo, I don't know, maybe 2018 or something.

Oh, I guess that's when I got the Turing Award with Jeff and Jan. I thought, I've achieved the greatest prize that a computer scientist can expect in their life. And I've accomplished so much. My career has been so rewarding and successful. What else is there to do? I felt like if I die tomorrow, I'll go with serenity. You did it. But, wait, but there's a but. November 22, chat GPT. It dawned on me, yes, but like, look, this has been a really big step. How far are we from?

human level? Maybe just a few years, maybe a decade, maybe two? And then what? Like, what's going to happen with this kind of technology?

Aren't we going to build machines that we don't control and could potentially destroy us? How do we make sure this doesn't happen? And I didn't have an answer. The Last Invention is produced by Longview, home to the curious and open-minded. We are an independent outlet focused on giving people the backstory to the debates shaping our future. To support our work, click on the link in our show notes or visit us at longviewinvestigations.com and become a subscriber.

And as always, it really helps us if you leave a rating and a review on Apple or Spotify or wherever you listen to your podcasts. One last thing to mention, audio from the documentary AlphaGo was used in this episode. Thank you for listening, and we'll see you soon. the app that helps you spot media bias and see a broader picture of the news shaping our world. Get 40% off their vantage plan at ground.news slash invent. This episode is sponsored by FIRE.

defending free thought in the age of AI. You can learn more at thefire.org slash thelastinvention.