Mythos Goes Myth

I mean, I guess let's just dig into it. There's still a bunch of other stuff that is that is also in the offing, including, you know, rumors that we're gonna get 5.6, uh, Meta's AI team apparently in tremendous disarray with uh reports that You're uh you're not allowed to to not report to duty and that some employees feel like they're being drafted into this unit, uh, but nothing touches what has happened for the last seven days at Anthropic.

We have the release of the model, what it appears to have done, then a story about the guardrails on the model and the backlash on that. And then come Friday night. It is uh export restricted, meaning that Anthropic has no way to keep it online with the the assurances that it would not be touched by either foreign nationals or foreign

uh uh people. And so it's it's currently offline as of now and all the ensuing political drama. Where do we want to start? Do we want to go uh back to front or front to back?

There's a lot of, you know, he said he said kind of going back and forth, but I I wanna kinda call attention to a couple of things. I think that haven't got as much signals, I think probably should have and might help shape the discussion. After the anthropic Department of War situation, um there was

an effort within the current administration to try to bring anthropic back to the table. And Axios reported on this. They talked about this, that there was a meeting with Scott Bessent and Susan Wiles. Susie Wiles, who is basically, you know, is Trump's chief of staff, being control of the calendar and in the way to what's it?

we can we can role play at what's going on, or we can just kind of actually go deeper into the coverage and understand a lot of what who a lot of the more players are. And and I've seen some people who I like and whatnot kind of like

Yeah, I'm not gonna offer to take, you know, was the right decision wrong decision. I just wanna kind of surface sort of like what what's sort of known. Um, and I've seen some people who've offered some takes that I think are very naive about what's actually kind of going on in the sense of like, They just want to go, they'll just use the administration or the government or this. It's like yeah, there are hundreds of personalities involved and and people with many different agendas.

What it was apparent when Scott Bessett brought Anthropic back, this was back in April, said, hey, uh, went apparently went behind Trump's back, you know, Susan Wiles to sort of bring this meeting together to try to just, we need to sort of sort this out. And, you know, the the sense was that, you know, Anthropic's an important company. It's a very key company. They're a pioneering company. They're building incredible technology. Corporate America.

uses this every day. It's become vital, like Google, like OpenAI, like Microsoft, very, very important and vital to the economy and and and a big driver there. And that's why, you know, Bessant wanted to say, hey, We we need to be involved this because this is just it's they're not just a thing that goes away. You know, they're not they're not just, you know, some crazy SF lunatics that just you can ignore.

Um yeah. They might be crazy SF lunatics, but they can't ignore it because they they build very useful technologies. So that was the effort was trying to go there. When this new wave came out of reporting on what happened. it appeared very much that when you read, you know, who was in the room and whatnot, it sounded very much like Scott Bessett was like trying to get, you know, so what happens is fast forward, anthropic, taught hype Smithos, and I think that they way, way, way overhyped, um

I I think it's incredibly powerful technology. I think that we have to be very mindful what these things can do. I I think it is a very capable, absolutely 100% very capable model. I do think it's potentially cat capabilities and cybersecurity and bio and all that. I think all that stuff is legit. I think that if your plan is to talk about it that way, then release it as a product.

You better have everything locked down in a way that just not is gonna put everybody at ease. Cause if your job, if you first spent your time putting everybody ill at ease, using uh literally using language like nuclear secrets, nuclear weapons like this, through the mythos thing, turned out

That means you can't just brush this off. And there was a story that they had like foreign actors may have had access for up to two weeks of the model that was supposed to be too important to keep from there. Didn't get much attention. Model, and then then we're like,

I'd said Mythos sounds like an early checkpoint. And and I will tell you, Google has early checkpoints of high much more capable models than the metrics they showed a couple weeks ago. OpenAI has very capable models that that than the that than their current state of the art. And we've seen, we'll hear the phrase internal model. You know, when you listen to the open eye podcast, you'll often hear the term an internal model. It means something there.

So, but they just don't come out and announce them until they're ready to be announced in theory. Anthropic was the one who said, Oh, you shouldn't pre-announce the stuff, but then it felt like that's it they did with Mythos and hyped it up.

So they come out with Fable, they put out Fable, Fable Smith those class, whatever. This is the one that's supposed to be the one that does this. First, it turns out that they were doing these weird safeguards. Like if you're trying to use it for machine learning research or AI research, it wouldn't just refuse. It would actively give you wrong answers by some accounts.

It would even change your work by some accounts and say that like change things in you and make the thing worse for you. And that irritated the entire developer community because it wasn't a matter of the model saying I can't do it, but they literally was getting dick.

Dick and Anthropics said, No, we told you this. They said, like we thought it degraded. We didn't really mean that you like would literally it would make the thing worse from doing that. And they had a lot of pushback from that and they had to like reverse themselves on that and say, Oh, we're gonna Try to change this policy there. So that that irritated a lot of the developer community because

Great model. Absolutely a great model. But all of a sudden people were trying to do certain kinds of research. Uh Daria's uh Daria Unmets, you know, he's a cancer reason he's a researcher, works on cancer. He was talking about all the safety guards, like he couldn't even use the model because it said, Oh, you work on cancer, you're bot you're in biology, whatever. He was livid. He

Researchers at Amazon, mind you, Amazon is a major anthropic investor. They're also partners with OpenAI. You could read whatever conspiracy theory you want, but they are heavily invested in anthropic. Couple weeks ago the conspiracy was that re reportedly Amazon somebody at Amazon had used like half a billion dollars worth of tokens for Claude.

And people like, aha, they're manipulating the market to increase their share value in anthropic because this just helps bump up the c you know, the anthropic market. Well this week a bunch of researchers, you know, some excuse me, researchers are people at anthropic excuse me, at op at Amazon, pardon me, at Amazon.

Yeah. Said there was a there was a a a vulnerability in the form of a prompt that could get could defeat safeguards or certain safeguards within Fable. And then now people are saying, ah, it's because Anthropics partner with OpenAI is why there's it's just literally.

Yeah. You then have to run it by your legal, you run it by your trust and safety team and you have to decide, okay, what we do. It's not, it's not a YOLO thing. It literally people go, Well, you have to report this. You have to do this thing, and that is that is a diligence thing. And it may not have been an easy call for them to make. And I think that they deserve some credit because

This is not in Amazon's best interest right now. It is really not. It is it is you can come up with a very convoluted conspiracy theory to make it fit that, but it's really not. Not in their their their best interest for this right now. So anyhow, you go ahead, sir.

uh a status on uh you know not just voluntarily putting models out there uh but You know, Amazon puts the word in, and now it's Scott Bessett and Howard Lutnick that are making these phones.

He's not doing it because of the tech stack. And he's not doing it because, I mean, with the way it was reported at the time, he was the kind of cooler head that was trying to prevail within the administration because the Pentagon was. furious. Like they were just they had absolutely had it with Dario and they didn't care if anthropic, you know, burn.

Uh uh and so Scott Besson comes in as the as the treasury secretary, somebody who has very deep ties in Wall Street. And not only, as you mentioned, Andrew, is Anthropic and Claw just a utility that is used throughout all Fortune 500 companies. There is an element of the American economy depending on a triplet of IPOs that are going to come out over the next probably eight months. One of them already came out in SpaceX and did well. Uh, and now you have anthropic and then open AI in some order.

And that... Scott Bessant wants all three of those to go well. This is a signal for the next 15 years of American technological dominance. These are going to be companies that you will hear about every single day. And they want people to have faith. that these are good to invest in. These are good for retirement funds. They are these are good for uh everything that goes along on the more financial quant side.

He tried to step in and you know we get into what he said uh uh well she said when it comes to the security researcher that Anthropa quoted, but like It was a personality thing. And that's, you know, that eventually leads to an export control.

More to it there. The stories that Chinese might have had access to it. They try to get hold of Dario. There are different accounts by this. We've heard wellness retreat, whatever, and whatnot, but we know that it was a Yeah, the thing that what's different about anthropic than open AI is open I is actually a much flatter organization. And you can get to a person, you can get to a person that can make a decision or Sam will delegate a decision to somebody else. Sam will say,

This person can handle it, whatever, and shut down the model. Like we've seen this in other ways, like Tebo, you know, resetting limits and other things. There's a lot more people on the switch with hands on the button to say, okay, let's stop this for right now. I would say that my experience at OpenAI was in a in its

That culture was way more, way more willing to discuss things internally, way more, I would say, almost cantankerous internally. And I think learned that lesson and also sort of said, like, okay, let's

let's let's let's turn the switch on this, whatever, and then sort it out, whatever. I would say here apparently was they were wanting to get hold of Dario. There was different stories about this. Why it was it how long it took to get hold of him, whatever. And people I was ninety minutes of this, whatever.

And then the apparently was like I think Bessett was trying to say, like, can we can we just shut down access to it right now and then figure this out? And that I don't know if you've seen Dario answer questions.

I've been to DC, I've been sitting there on behalf of opening. I I've talked to these people and stuff in my in in limited capacities. And I understand that I can't go into that room and be like, Well, guys, uh chill, I got this. That because

Sometimes you don't. There's not a high amount of trust between DC and the tech world because there has been numerous reasons when tech has not been forthcoming or has downplayed stuff, whatever. And because at the end of the day, it's a business. And when a business says, No, no, no, everything's fine, whatever. You got to be more proactive. You've got to give better signal than that. And I think that here it seemed like there was.

There was a frustration. And then the outcome was, and I would say, here's the two things that outcome was put the X work controls on this, whatever. Very quick to do it. And the people that brought this in were not the DOW people. This was not the same decision. This was different. There were national security people involved. I understand a lot of the NATSEC people are actually

Outside the administration. These are sometimes career people. And I think that's what can overlook too, is it's easy to go, oh, it's a political disagreement. Oh, they didn't like the researcher. I'm like, that's not how some of these other people operated. Um so we're in a sticky situation now.

interesting that that those researchers were named. Um, you know, I think Anthropic's pretty particular about who they put out there to talk about things. They have a few voices that they want to kind of be their um Their communication channel. So it was notable that there were non-Dario uh names that were that were sent to kind of talk about this, but I I I I have a lot of I have a lot of questions. And weirdly, none of them are technical. Like I I believe every security researcher that says

Hey, this is this is something that is replicable on other models. And you can do versions of this with Opus. You can do versions of this with 5.5. I I believe all that. Really, it only underscores again the same problem I had with the supply chain designation, which is like. When you're in these rooms

You gotta be able to get get through these conversations because there's they're only gonna get more intense. This is training wheels for for for where we're going. And uh You know, a lot of the temperature around this was turned up by one company. And it's anthropic, the one that's the one that's the that that that's complaining about government intervention now.

Like I'm sitting here going, well, yes, what do you think was going to happen the first time there was a security issue or something? One, when you say use the word nuclear materials, which are, you know, what we have created, we've created it, you know, based with a nuclear regulatory commission, which I think is just

derailed our energy policy and our climate policy in such negative ways people can't even understand. And so part of it is it's like like because just way, way over cautious. And when you're the guy out there Saying, oh, we need to be careful. We gotta be this could be, you know, this is the impact here. And then you

Do rush it to market, do this, whatever is one thing. And then also when you're the first safeguard is can they can the government call somebody and say, Hey, can you just shut this down? And they they can't feel comfortable that you're and like

By the way, on a you know, uh background reporting says government said we don't have this problem with any other lab. They said they're they're not thinking this is gonna jeopardize the release of anything else because you have Gemini 3.5 Pro coming out, there's GPD 5.6 coming out. They've said, no, this is a unique problem with this. And I will say part of it is that Andhropic is very much top down. very much uh and and you know uh Dario the guy calling the shots And I would say

They have built he is he he and he's got a he's got a great team of people, very clever people that they have built an incredible company. But I would say that one of the things that I realized that I've made this argument about why you why people say, Oh, what about the first billion dollar solo run company? I said, You know what? Technically you might be able to do it, but you're gonna need a lot of lawyers and anything else like this because the bigger you are, the more surface area.

The same thing with AI companies, you know, early days of open AI, the idea was, oh, could this company stay around 400 employees? Is that possible? And I'm like, I don't think you can, not because it's a product thing, it's your surface area increases. And as you are talking about being very, very, you know, the driving force of technology and advancement and the thing that scares government. I've said this, you know, I've said this internally to other people that

you know, when you say that you have nuclear power to an actual nuclear power, they will take you seriously. And when they take you seriously, be concerned. And so, you know, I I think there's a world where they figure out the right balance. They understand that, you know, not every not every pushback from the government is because somebody has a personal dislike for them. That's part of the problem is that

There are people there who are trying to under like Anthropic has said, Hey, we don't understand the full ramifications, what these things could do. And then the government's like, Well, if we don't understand that, maybe we should shut it down. Like, no, no, no, this one we got. It's like,

But you didn't really see that with this one. You know, you you you had a group of uh researchers from various different companies that signed a letter saying that uh this was an over punishment for something that exists in in other models and everything.

Uh, but I've been kind of surprised that the reaction has been more muted, both from the public and also from kind of baseline employees at a lot of these other companies. And and I wonder if it You know, this is now a pattern of behavior that that they get in these spats with the government and they can't get themselves out of it and and The reporting on the response time.

Of calling Dario. And let me also tie back to previous reporting from the Pentagon fight, when apparently one of the questions that was asked to Dario was We were dealing with incoming nuclear missile fire from a foreign power. Could we use autonomous targeting via Anthropic to fire those down? And Dario said, well, in that case, you can call.

Let's remember that. That was one of the things that allegedly uh infuriated the Pentagon. According to both sides of this, When DC, when the administration came calling about this. The White House says it took ninety minutes? Anthropic says it took an hour. But the fact that one of them is looked at is like, we got back as fast as we could, only an hour. And the, you know, uh White House is like, they didn't get back to us in 90 minutes.

This is a panic that we need to, you know, have an export control. Number one says a lot about East Coast versus West Coast time values, but is is, you know, that's the difference. Like their their public thing was like we got back to them as soon as we could in an hour. It's like, All right, this was you know, now how much

there for them to be like, Hey, what's going on? We need to talk about this, whatever and they finally get'em on there and it it's like, I don't I I don't know. I I think that you have to be you ha I I think that the challenges You have to be willing to empower people, not just hire a bunch of lobbyists, which, you know, Anthropic did that. They hired a bunch of conservative think tank lobbyists and whatnot and people to sort of make their case for them.

you know, which like I like I but it's like when you're talking to people deeply embedded and who have accountability, you know, whether it's head exeth, whether it's best it with the economy, whether it's people in the national security you know, organizations like you've you've gotta be a bit more embedded with them.

And and'cause that creates confidence because then it creates this idea of okay, I know them, I know the guy they're gonna send, I know this, who can I talk to over there? Can I call your I oh I already know your guy over there that I'll call and talk to, whatever. And that's I think that's the concerning thing. I think the next step is the anthro I'm glad that Anthropic's sending the team there. I think they need to think about like culturally being a bit more um you know be connected to that

'Cause it's here's the thing too, it's gonna transcend administrations. You know, we're gonna get a different administration at some point. And and I don't think all of this can be pointed out to a political difference. I think it really comes down to a structural difference.

And Dario wrote this whole essay about hobbits talking to Tree Beard, you know, the Ints in Lord of the Rings and how the Ints are sometimes too slow and to react and woo this, whatever. And I'm like, great way to reach DC. Start with a Lord of the Rings reference, bro.

To lie to you intentionally and maliciously destroy code, that is that's that that is that is that is a line that I think is is too far. Well.

China is real. China's a real place. It's not just a thing we hear about on Twitter or YouTube and people say and then then it disappears when we're not thinking about it. And they have they have spent more money than any other country on the planet trying to exfiltrate American technology and are building their own defense systems or counteroffenses to our entire military apparatus from everything from

bombers to aircraft carriers to cyber. And so and part of that is making sure they they know that US has this advantage. They don't want that advantage to to stay around and they want to do whatever they can to to to eliminate that. So one thing Xanthropic put in there was that

You know, if you were trying to do things with Claude Code that looked like distillation, like you were trying to copy the model, get the model to enough outputs that you could simulate, whatever, it would create extra tool calls. It would do a lot of other things. That was funny is that that when you looked at other models that were doing unnecessary tool calls.

Uh, they were the Chinese models in Groc. And that was a thing where it was like, well, that's a weird weird coincidence there. The OpenA models didn't the anthropics when it was trying to really do a thing didn't. So anyhow. Things like that were there. But then we got to the point where they put out mythos and then mythos is

Hey, help me figure out how to train do a training run on this. It's like great. This is the best method. And people like, this is terrible. Like it actively found the right method, then decided to do a different thing or change something somewhere else. It was disturbing because what you want, the ethical thing is a refusal, is is you have a classifier that says.

I can't do that. Like, hey, this is a limit there. And I think their fear was they didn't want to have a ton of refusals for doing tasks like working on code, whatever, but it put them in an interesting situation where C customers were paying for a thing and the model was actively making things worse or lying to them. And I don't think that's a good look. And and

You know, what are the what are the answers to it? The other answer is you just do a refusal and you just take, you just get the heat for it. You do a refusal. I can't do this. I can't be used for ML. I can't do that. Um, and give you an example of, you know, the ideological difference between OpenAI and Anthropic. OpenAI stated repeatedly that they think the the democracy, United States government, you know, democracy should be the determination of what you can and cannot do with these things.

that is not their job to say, you can't do this, you can do this, whatever. Ultimately they leave it to the democracy to decide how these models will be used. And that's controversial, you know, in in AI world in that

You know, Anthropic said, well, we think we should ultimately be the arbitrators of how these technologies are used. And I think you as a creator of a technology, you can say, I want to decide how my thing is used. That's fine. Will you create a product? I think you could, you can, you know, do Atlas Shrugged on that.

But it is an ideological difference that open eye is like, Hey, we're we will we prefer the government to come in, we prefer the dematic democratically elected government, elected officials to determine this policy, what should be done.

You don't really know until about a week in when people have a kind of a fuller fleshed idea. Um, and you can see kind of a clearer path of the jagged frontier of the models and some things it's Brilliant and some things it's not great. You know, it's it seemed like there were some drawbacks where it was very argumentative in the way that many of the anthropic models have more increasingly become. But yet a lot of the back end stuff, it was apparently.

Pure magic. You know, some of the the one-shot demos of uh video game stuff and and videos that were created were were were really, really, really cool.

And then we kind of got the rest of the noise. And that's where I uh it would be really, really, really sad. It'd be really a bummer if if this kind of wild ride of you know, technology drag racing uh that we have seen in AI was was kind of over either for governmental intrusion or because there's one company that can't keep their car on the track.

Google, I I mean I think the thing is like I believe the vibes on I believe the five vibes on Fable. I believe that I think I think there's good I think I think that I will say that like I've seen some things like oh I got GPT five to do that but then I like I you know, I'm I I know I'm like insane about knowing how to prompt weird stuff and to get models to kinda do things against the way they normally want to do it, but

I believe it. I believe that people I think are sincere like really loved it and think it's a fantastic model. Um, and I'd say that we because like we know is nobody's mentioned. Gemini three point five flash right now. Like that that's that's that's the problem that Google has is Google will announce a model with really good benchmarks and then just nobody uses it because it's

They brought in Eric Schmidt, because when Google was very, very powerful, super, super quick, they brought in, you know. And I think that's the key is they need, I think they need to bring in uh, you know, maybe somebody we've known, a known name or somebody like this, and just say, hey, this guy

It it doesn't it it seems like it would be very, very hard to put anybody higher on the org structure than some of the people that they have now. And it doesn't seem like any of those people can talk to the government.

There are there are a few people like like, you know, if if Greg Brockman told Sam, Hey, we need to do this, we have to stop or we have to do this, Sam would do it. I think there's a couple of other personalities that open eye that can say, Sam, can we just do it? And Sam like go for it, do it, we'll figure it out later, whatever.

I don't I don't know that that happens at at Anthropic. And I think that's concerning is the idea that this the stories about the wellness retreat, whether or not it was just intentional, just trying to make them look more wacky and weird, but it also shows a frustration to say that.

that that is the that is the fear of the DC co you know, the the the you know, of the East Coast is that yeah, these people are weird and they're gonna off doing stuff we need when the the fit hits the Shan and like Dario said, call me and they called him and then they couldn't get'em to come up with a complete answer.

You know, so I think that if you give somebody, give somebody say, hey, you know, you've you've you've got the authority to sort of the government says you need to do this, then pull the, you know, pull the plug on it, do whatever, say yes to whatever you need to do in an emergency situation, and then solve it.

Maybe even the exact same conversations that they wound up having with the government about what this jailbreak was and blah, blah, blah, blah, blah. Like if there was some token of goodwill to say, Hey, let's assuage your fears and then let's have a conversation. The model might be running right.

That we were that we were too protective, that'd be air on the other side, and be not protective enough. And here you had a case of where. Not so wasn't somebody it wasn't like Pete Hegseth said, Hey, I've decided I found I hacked this. It was Amazon, not in a a a you know, of of the big tech companies is about as

equally involved with everybody as you can imagine. It's not incentivized to F this up, says, Hey, we see a serious issue, whatever. And Anthropics like, nah, that's it's it's nothing. You know, it's like Ye you might be right, but that does not make people feel comfortable.

Uh, with the the the feds. Uh uh, you know, you don't normally see companies that live to tell the tale after. after two of'em. But Anthropic's massive. And again, I mean that there's a lot tied into that IPO. There's a lot tied into this company. Uh uh I think there's a lot of people rooting for it. So

I hope that they get it. I hope that they get it right because I don't I don't want to see I want to see these two companies compete. I think that these two companies compete has been one of the most phenomenal stories of the last 20 years. I think it's been great. Uh and and uh I it's made me excited about tech in a way that I have not been in in a in a in a long time. And I don't think that it's it's Ma my my interest is not in having one of them shoot themselves in the foot repeatedly.

Uh we they they we're not quite sure if they will pull our model or not. Um and hey, what do you mean? We're we're can we get them to just give us an agreement? Well, no, they won't. Sam Altman says, ask it's asked, like, can open AI guarantee us access to the model? And he's like, as long as it's lawful and legal with what is allowed, you know, and is not making kill decisions and not doing mass surveillance, we're okay with that. Uh And they're like, cool. And then Sam got got ripped apart.

Yeah. For literally, you know, stepping up when the country asked, you know, to say, you know, can are we secure? Do we or do we have do we have some do we know that AI won't go away overnight if China does something crazy or whatever?

And I got criticized. I think there's I think there's an attitude at open eye. It's kind of like sometimes you just gotta st we'll see. I don't know. No idea what'll happen to me. I think me and it'd be great if they release it, but also wouldn't surprise if they're like, We don't want to jump in here.

All right, do we hold it for another few days just to you know get a little bit get further away from this? But then again, also in this game, I mean

We shall see. Did you read the meta report? The bad vibes inside their new uh their new AI lab.

I can have like 30 teams, 40 teams working on something. Somebody has to keep an eye on how these things kind of integrate together like this, but they can kind of go off and do their own thing for six months, come back. You can throw more people at it. And in a time Microsoft learned the hard way multiple times, so did Apple. You will, that will just prolong the time that it takes. There's a communications effort.

When you're making an AI system, it is all the communications effort. It is a small group of people. Who have empowered to make a certain amount of decisions, run a certain number of experiments, figure out how to use compute, et cetera. One of the stories that that that we've talked about in the OpenAI podcast, Jakob Bahatsky, who is the head of uh research for OpenAI, the chief scientist for OpenAI.

One of the things that Yalka brilliant many things was pioneering in though was predicting what compute with a certain algorithm would reform, like what would happen. Like, you know, he was very good at predicting. Weird GPD forward land, etc. like this. When you're an organization like Meta, which is not a research org, they have funded research, they've had Fair, they've had a lot of different stuff like this. They didn't buy a research org like uh DeepMind.

You know, nor do they have like Google Brain, which, you know, uh they didn't buy a research org. It was a very great collection of talent like Deep mind had when they're trying they had Fair, which they brought in Don McCoon and other people there. And it was very much run like an academic program. You had to be a PhD, all this. The recipe for me was like this is

That was not gonna yield the results they thought it was. Reinvented it, brought in Alex Wayne, who I think is brilliant, built up scale AI, super, super smart. This is a completely different thing. You know, telling a bunch of engineers and researchers they gotta label data or do whatever.

That is that is going back to what you know and not how these things tend to work, in my opinion. We saw this at Microsoft at Silamon, where they brought in him to try to build up a whole competitor to, you know, a a hedge against open AI. And I'm like, I was, I was like, I think he's great at certain kinds of companies. I don't know that he's a guy that's going to run an entire research led organization. And what I want watched happen at Meta, I have friends there now that I I

I think that there will be things that they will be great at. Like I bet you that when Meta starts getting starts really built, you know, starts scaling up their data centers and start thinking about how to operate that stuff.

I think they're gonna be probably be state of the art because they think those are skill sets within theirs. When it comes into research and doing stuff, they have built some amazing stuff. I love like their segment editing models. Some of their smaller models were great, but the bigger effort.

Man, it's hard. And the mistake that the mistake that I think Zuckerberg made, and again, this man has done far more for the world than I ever will have done. So I'll make that very clear. Um Started trying to build an AI team.

Great miracles. And it's like, no, you need a team. You need to build a team. And You know, we've seen repeatedly.

on each step that you walk forward. And so you gotta kind of have people that really have it all in their head uh to do it. And you gotta trust them. And it doesn't seem like they have that right now.

But I do think it kinda gets hard because when you sit down You know, opening and I spent who knows how much money training GPT 4.5. And 4.5 was a strange model, big model, very capable in some ways, but other ways wasn't. And they ended up doing a different path for the five series. And

As far as I know, no heads rolled. Nobody got fired. Nobody's like, oh, you screwed this one one up. They there probably was a very good thesis on why they thought that would work and it didn't, but also wasn't the only plan. There were other things to do that.

I don't know what that's like at Meta. I don't know if somebody comes into Meta and says, I have this grand vision to go do something that, you know, if it fails or whatever, or they're not gonna be like, hey, what we learned along the way.

Don't worry for certain tasks will let you turn it off temporarily. It's like, you know, we we were talking to somebody who works at one of these lab what works at one of the the top labs, and they were like, They should have just made it optional. Just said, hey, you don't get our great computer use models that we're building if you don't opt in. Like let them let them let them opt in.

that uh, you know, Dario wrote something a couple of years ago saying that by the about this point in history that the models that have kind of broken the uh uh you know uh uh uh launch trajectory that have that have uh made it out uh will be the leaders forever and it's gonna be impossible for other models to catch up and that right now you can say anthropics there, you can say open AI's there.

We'll see what Google's cooking up. They certainly have the money and the talent to do something. We'll see whether or not they can get it together. But that everyone else pretty much It's gonna be nearly impossible to be able to compete at the frontier past now.

Um a lot of the research stuff is downhill from what goes goes from open I to other labs. So a lot of stuff like there have been some pioneering stuff out of the lab, but a lot of the the the exodus of people from open eye to other labs.

Has been sort of a biggest thing. You know, Anthropic comes out of OpenAI. A lot of people go from OpenAI, bringing techniques and methods over to Anthropic. Um, there's some stuff, you know, coming from Google, whatever. But OpenAI, we knew, you know, early on, had a great advantage.

Um when you look at also when you look at overall capabilities of the models outside stuff and medical and math and whatever, you realize like open I really has this advantage, it kind of doesn't get talked about as much because outside of like code and whatnot, it's hard to sort of evaluate that. Um I think anthropics. In a great position.

Microsoft was because relationship with OpenI was going to be in a sort of a strong position. We knew Google. Google is really a management issue. Like Google has all the incredible talent in the world, but management management issue. The question I had asked was you know, what were Amazon and Apple going to do? Yeah. Apple turned out nothing. Finally may have built a really good on device with the the latest Siri, if it's true.

Which they could have had two years ago, uh, easily. Um, Amazon said, you know what, we're because I did I knew the problem Amazon was always going to be. Amazon culturally does not like to pay huge pac compensation packages for researchers anybody. And they were always I knew they were always gonna be in a competitive disadvantage because they were just not going to write over like hundred million dollar checks and stuff like that to bring in researchers.

Anthropical excuse me, Anthro Amazon would invest hugely in whatever. And I think Amazon, you know, has their new train you know, training processors, whatever. I think Amazon has an egg benefits from the ecosystem. XAI. Uh I just knew that that I I think that Elon's the greatest in engineer of our time, maybe one of the greatest engineers of all time, incredible businessman, what he's built there. Um I think that though every time he started

uh in its debut on Wall Street. We'll see where it goes over the next several months, but muzzle top to everybody uh for for that initial that initial pop. So maybe he was just having a good time uh, you know, watching this uh astronomical stock. Uh situation.

you know, there's the the talk of like do they try to merge Tesla and SpaceX because part of the problem is is Tesla's the one that's supposed to be building the robots, but like powered by what? Um I don't I don't but yeah I I don't I was kind of like, unless you're Anthropic or OpenAI are already like a trillion dollar company, I don't know that there was gonna five years ago that there was really gonna be a way in for you in the game. Um, and I think part of it comes down to too is that

Both open AI and anthropic are extremely culture driven. Extreme, even I would say the culture to open I has changed a lot. But I would say that it's kind of an alignment though with the idea of, you know, well, let democracy decide kind of views on this. Um

Yeah, I mean who you know, I I I know there are people working on stuff, but there are also people like, Oh, this guy got a few billion dollars and I'm like, Yeah, that guy spends a lot of time high on a beach right now. So, you know, um maybe, I don't know. Um, but you never know. I

GPD-4 comes out unofficially, how big it is, hasn't been announced. Huge model, a big model, comparatively, right? Big, big model. At the time that I worked on the release of that, if you told me that Three years later. I would be able to run a model maybe slower, but as capable roughly as GPD 4 on my MacBook at the time, a 2023 MacBook. I would be like, That's just crazy. That ain't gonna happen.

Fast forward, we have the GEMA models from Google. We have, you know, Microsoft's worked on their five models. OpenEye comes out with GPT OSS model, the OSS twenty B model. Which I think is pretty close to like, you know, running like a GPT four mini or whatever, you know, or like an it's capable of reasoning. Very capable model. It's actually the GPD OSS one twenty B model is the one that if you wanna use Cerebrus or Grock, they'll use.

One of the reasons is it's distill majorly the major reasons distillation is you just take a big model and say, okay, give me the right data set or thing that I need to sort of train my smaller model instead of me wasting my time on reading 5,000 versions of the Canterbury Tales. Um There are things within machine learning that I think that there's a lot of stuff on how people try to optimize training on type of text.

We've only well only kind of the really big improvements we've seen in training on text is improving the quality of text and creating and changing the order in which you train on stuff and the vocabulary size of the stuff. To my knowledge, there hasn't been a lot of fundamental stuff that we've heard about. Really conduct thinking more intelligently about the way you look at tokens, how you train upon them and trying to make more efficient ways.

I think there could be way more efficiencies out there. And could it be done with a smaller team just saying we're going to approach this very differently? Could be, you know. But you know, I've I get pitched all the time by really clever researchers say, hey, we figured out this way to be more, we're going to be more efficient than blank. And I'm like, cool. You know, the that you know, GPD for

GPD five mini is like a thousand times cheaper than GPD three. You know, you're competing with that already. There's already this trajectory, these models of of just using transformers getting better. But I don't want to be the guy that says, no, it can't happen. I'd be like, I would have to it would have to be, I'd have to be like, I can't see how it does, but that doesn't mean it can't be done.

Well uh you know, you know, two things. One is we could have put America could have put a thing in space first, but we didn't want the mi we didn't want the army to get the credit. We're waiting for NACA before the Vietnamese to do it. Like literally Werner von Braun and his team of Americans, um, we're looking at a rocket that could have done it and told no, don't do it. Second thing is is that

There was a lot of stuff that Deep Seek did that were well known but secreted of the labs and then somehow showed up there. Uh and brilliant things there, but I'd say that there are All right.