Claw-ture Clash

Hello and welcome to the attention mechanism. My name is Justin Robert Young. Joining me as always is Mr. Andrew Main. How you doing? Fantastic. We have, I mean, uh a wide and varied uh uh topic list here.

You know, it's it's funny. I think we said this a couple of weeks ago. Oh well, I just gotta say When I was inside Open AI, we talked about open AI time being different than the rest of the world time because I came on board, two thousand twenty, pandemic's happening, but like GPD three, they'd already been in this this whole

Sprint to get GPT three ready and going and get into GPT four and all that. And people came in from other companies into there and were like, Oh my God, the pace is insane. You know, Googlers, people, Facebook were like, This this pace is insane. Now the whole world is experiencing this pace. It's crazy. Yeah. I think it was like three or four weeks ago that I had a longtime uh listener and friend of the show.

Who said, You wanna know what? I really like the attention mechanism. I think that you guys are are great, but the problem is that you're too focused on open AI. The the biggest thing that I'm that's in my timeline right now, and he's more of a coding type, is uh then uh clawed bot, uh uh sensory blan rented to malt bot, then open claw.

And so now the complaint that we talk too much about open AI because we are too we are not talking about open claw uh enough is now merge. We have now put those two stories together as The man who invented the

that uh open source protocol to run local agents is now officially an employee of open AI. There's a commitment that open claw will remain open source, but He is calling coming on to work on local agents for open AI. Uh life comes at you fast, man. That that all that stuff dawned. I mean, I think it's less than four weeks ago. And from there it it took root. It

had the the heart of of every uh uh Linux box coder in in uh the the the Bay Area, certainly in very much the world. The you had some of these meetups that were in the four figures of people that were gathering to talk about this thing. And now he is a part of OpenAI to lead this company and local agents. So w what is your take? So back in January, all the way back, uh when open when uh ClaudeBot started gaining momentum.

Um Peter Steinberger, uh who uh by the way, is a a a prolific creator. Um he's created a number of open source product. He's done a lot. He's a really great builder. He had posted a tweet and talked about how He's being asked to make all these updates to it and all this sort of stuff, but people weren't really contributing financially to go support the people maintaining it, whatever. And he's talking about like all these demands for this thing and I said this is

And I use the example of like this is the problem. You create something people love and people will love it to death, but not support you. And you know, and I said I I tweeted out uh I and also and then all of a sudden he got the ch he got the cease and desist. from Anthropic. And so this was January I don't know twenty sixth or whatever. I tweeted out. I said, OpenI has the opportunity to do the coolest thing and help, you know, him. Right. And uh'cause I'm like, this is this is I

I get Anthropic is very, very safe their idea of safety and there's everybody's safety concern, but their idea of safety concerned and and they're they don't do open source. They don't really do I mean they they I don't know, maybe they produce some of the sort, they don't do open source models, they don't do anything they can't control or shut off an API. And I think

Which is amazing for as much developer love they get, which is a sign of how good Claude was for coding, you know, can tell Codec really stepped up was how good it was is they could get away with, No, we're not gonna release an open source voice model like OpenID. We're not gonna release any open source models. At all. Um, and I think that uh uh uh they did some they done opened some specs. I'm gonna make it clear other we but I mean they haven't done at least like modeling like that. So

That was I get for them to say like, yeah, they don't really want to embrace this thing because listen, open clause cool. It is a security nightmare. I downloaded this, I looked at this, and I'm like, oh yeah, you haven't solved all the problems every one of us who's built something like this has gone through, which is

all of the security. Now, the plug-in right roller there, it's great. It plugs into stuff. It can talk to all sorts of stuff on your net. It can do all sorts of they build tons of interfaces, which is a really, really big challenge to build stuff. But it was like, yeah, the hard part, not to say it's not hard, but the the real work is the security part, right?

And and that was that was the thing with its adoption base, is these nerds wanted to turn their entire lives over to open club. They were very willing. The desire to automate To have a full-on personal assistant is so great that they were willing to drive into these perilous security voids just to see whether or not they really could do it.

Yeah, and and that's fine. I think that's that came down to the kind of the choice thing. And and that's not an anthropic sort of ideal. Like that's not their ideal because they say hey. Um and it's not a knock against them. That's just not where they are in sort of that respect. So I get it. So anyhow, um

It it enjoys super popularity, but again, it's expensive to maintain because even though like there's just a lot of things going on, there's a lot of other utilities involved, et cetera, like this. And then he got uh He started getting Peter started getting circled by all the VCs, everybody else like that. And he's a builder. He is a builder at heart. And You know, he got talked Zuck talked to him, Sam talked to him, Very V C talked to him, and then

Um, he ended up saying, Hey, you know, I want the project to live on and open I said to him said, Hey, listen, we'll support a foundation. So you can create a foundation that will support the growth of OpenClaw. We'll help financially support that. It's its own thing. They're not gonna go in and try to like

you know, make a proprietary thing like this. But we want you to come inside OpenAI and go work on agents here and to go do that there. And so that was the deal. Is it so he brings there and he's probably bringing some other people in to go work with them to go work on that.

I think it's a great move. I think it's really open AI. We talked about this before. Anthropic did a Super Bowl ad criticizing ads inside of AI. Uh by the way, Claude has risen now in the app charts. Like at Claude was like last I checked was like you know, like number nine or ten or something like that, which was huge for them. And and and they they they have listen, they there were times when Anthropic was a they're number sixteen now, but Anthropic wasn't even like the top two hundred.

So that's a big win for them. Um but open I went for code. It's a high price. It is it is a high price for a big win, but but certainly a big win. Yeah, oh yeah. And you know, meanwhile OpenEye's ad was about codec, so OpenEye is trying to say, Hey We're here for developers. We're very serious about this. And I think this was another way of them sort of proving this was to come in to the rescue on that. When interviewed, they'd asked uh Steinberger about

Hey, uh what was your interaction with anthropic? And he said, Only love letters from legal. Yeah. Which yeah is not not not ideal. Yeah, I mean again,'cause it you don't have to say, Hey, cool, we endorse this or whatever, but I think that that that's not not not the best optics. So right now D Claude doing well. App store, number, you know, oh you know, chat GPT, of course, number one. It's always there, but still, that's a big win for them. But

Well, uh I think Anthropic is getting their kind of welcome to the league moment. They are they've announced themselves as a major player. We will see exactly how that transfers to the consumer space. They certainly have a a tremendous uh nest egg in terms of revenue on the um the the enterprise side, but

They've also got some problems. You know, th they they're in the middle of a very big public fight with the uh uh Department of War, uh uh formerly known as the Department of Defense here in America, where their models were used by a vendor in the extraction of Nicolas Maduro from Venezuela. And now people at the Pentagon are so

ticked off about that, that they're looking to 86 anthropic as a possible supply chain risk because they've raised these questions. Now We're we're in a very, very strange place where technology has folded in ever more than it than it we've ever conceived it could be into our modern war fighting sphere.

But that's not an enemy you want to pick. You don't want the Pentagon you know, out and out saying to reporters that that you're a supply chain risk. Yeah, and understand what that means is that it is that if you're declared a supply chain risk. It doesn't just mean that they're not gonna do like like apparently the story as I read it was that Palantir had used Claude as one of the models.

Yeah. And that means that then Palantir can't use you and it means your your suppliers cannot use you. And it's so it's not just literally the Pentagon's No, we're cutting off for subscription. It literally means Downstream. Anthropic had has talked about said, Hey, we were we were and they're they were really quick to say we are the first ones to do defen deals with the intelligence services and stuff like this. We just, you know, it's good in some camps. Other camps will go, wait, what?

Um and they're not the first company to deal with this either. You know, you had Project Maven at Google, you've had these other projects and other military projects that people in You know, Anthropic is a company that says, you know, you know, they left open I because it said, Hey, we have a different ethic than you do. You know, we don't like

We're we're you know, we don't know about taking money from these big tech companies and you know, we don't know about commercializing your API, which they then commercialized their API and took money from big tech companies. But hey, there's more to it than that. And that's what we're gonna see. We're going to see you know, what comes down to it as far as anthropic because

I can tell you company you know, people inside both OpenAI and Anthropic and Google, all those companies, a lot of people that aren't comfortable with any kind of military, you know, uh contracts. Yeah. And Anthropic has said, you know, we don't want stuff used for mass surveillance. Uh we don't want stuff used for like letting uh you know uh you know machines deciding when to pull the trigger, et cetera.

Um and these get to be they sound on the surface like pretty easy sort of decisions to be made, but that really depends what you mean downstream. For instance, you know, you have the Ajax system on board, you know, uh battleships, which have been used for forty plus years, which is an automatic firing system when something gets in too close to do it.

So if I'm a company working on a firing system, can I use your model to design or test it? You know, you know, if I'm part of the infrastructure, you get in these things where it sounds easy on the surface and gets hard, you go, not use for mass surveillance.

How do you define mass? How do you define surveillance? How do you do these things? And and You know, you and I share a paranoia in the way that government contract these things go, but we also understand that once you say, Okay, we're we're in, but here are our conditions, those conditions erode very quickly. Well also, if your main bread and butter is enterprise

Yeah. It means playing by the rules and understanding that people are going to use this for a variety of different things and and That's where idealism tends to melt, especially if you're a company that's thinking about IPOing as Anthropic is rumored to be doing, is rumored to be in a race with OpenAI, uh, to see who can

IPO first, which uh I don't know exactly whether or not that's that's true in terms of the competition, although both almost assuredly will within the next, you know, two years, presumably. But that's You know... If you are in a scenario where something else is either cheaper or more available or gives you less problems, then

I don't know if people are going to decide on something on on a company that it's willing to second guess the government. Because if you're willing to second guess the government, then what is any company? You're going to rely on something and then have them shut it off? Yeah, it's it's a it's a situation where

AI companies are in a different situation than other companies. And often people kinda go like, Well, fine, you know, if our employ you know, we'll just fire the employees that want the other thing. Well, number one is like often you have a lot of employee equity and stake and stuff. And sometimes you have people founders who have this sort of stake in that.

And you know, sometimes it's not like, hey, if you don't like working in the munitions factory and quit, we'll hire other work munitions factory. It's like often

You know, the the more examples of people who are designing these systems and stuff. It's like great, do you want them here or do you want them elsewhere? And that's the thing situation anthropic has to think about. Other labs too. It's again, it's it's not and it's not a problem, they're only they're the one the first ones to deal with it. is, you know, what do you do in a situation, you know, where are they gonna go? Like I'm curious. I don't know how it plays out because um

there's enough people who have stake inside that organization who maybe have drawn very clear lines in the past to say this is what we'll do, this is what we don't do. I thought those lines weren't really realistic. particularly when it comes to dealing with any kind of defense contracting, because those lines quickly

quickly become very hazy when you start to say, okay, your system can't can your system help design this system? Can you help do this? And I thought I think a lot of the times people who define those things like, oh we can't do this, I can't do that. I'm like, at what point along the way?

Yeah. And then it gets to be like they never don't think about the second or third order. Like, okay, can can a company that designs these sorts of systems use your system to dealt design their system, even though your system's not making the choice?

pivot for a second to what I think I don't know if it's the biggest story in AI. I it's it's certainly not the biggest story in AI today. I think it's going to be something that we look back on in history as more important than we thought of it in the moment. And that is the depreciation of. GPT four O lauded as a very uh uh you know breakthrough model, certainly for the kind of processing that it did and its speed, but beloved by many

for how well and warmly it went back and forth with its respondents. It is one of the things that became controversial with ChatGPT5. It is also connected to Some of the downside risks of people who maybe have mental problems and talk to these chat assistants uh uh too much. I will say I did not I did not have the emotional connection or the the workplace uh relationship that many that uh many apparently did with four.

You saw some hysterical reactions from places that uh where people who talk to AI companions gather and One person that I'm gonna refrain from giving details here, but this is like a very straight lace. person in my political orbit. hit me up lamenting that 4-0 was gone because they used it as a assistant to bounce ideas off of because they thought that the model trained very well in patterning the kind of thinking that they had.

Obviously this is a a niche case now, but I don't think the world of AI companions is going anywhere. I think it's probably only getting it more normalized throughout our society, hopefully in a healthier way than we see in in some cases.

But what do you think about four O going away? You know My my bias is that having worked with these things from the early days and knowing how to prompt a model to kind of behave in a way that I want, you know, and and and doing that with GPT-3 and other successive models and

I I kind of never it would to me it was always a Halloween mask. You know, the way it responded, whatever it did. Yeah. Because I understood what's the I understood the base model underneath it. I understand the post training, I understand this sort of thing and doing that.

So I never really found myself falling like, oh, I love the way you talk to me or whatever. Like, I mean, I can get in a moment of like, oh, it's cool, bro. Whatever. But like I just never as I'm like, oh yeah, I I like these better answers. I know how to write system prompts and stuff to get it to do the thing I want. And so I think that

I think that my experience is a bit different, but I did I remember the d day that I said oh yeah, today's the last day, I did send a text message. I said, Hey, you know, goodbye, bro, to you know you know to I to G P D four oh and I said, Oh, thanks, nice sewing, you know? And then it was like I'm like, Oh wow, that's sad. This is like the last message with this model. And I did think about that. And I I think about these models so you know that they're

They're stateless. They're not stateful. They don't have memories. You know, GPD four O, other than updates, the training updates was the same that the day it launched to the last day that it was used. And it's just the context is really if there's anything that makes it what it is. Um and so it's a different thing, but I do get I do get it. I do understand that it is hard not to anthroporphize these things because they are trained on humanity. They are a they are a product

of human information. They're a product of the human information space. And to have an interaction with something like this and then to say, hey, this thing's going away, it's it's Tark'cause it's not a it's not a person. It's not doesn't have you know, probably doesn't have consciousness in the way that we do if we could ever define it.

But it is a system that is complex and does things. And I think that you could be sad that you're losing, you know, a favorite tree in a park. You could be f sad that you're gonna lose your car that you've had forever in a trade in. And so I I get that. And I think that yeah, I think we're gonna see more of that. And I think we're gonna have to think about what that are. I think that the risk is

The risk is anthropomorphizing these things too much though. I think that is the risk. You know, a car at the end of the day is a car. You know, this is this is this is a state of a bunch of GPUs, you know, that can be repurp those GPUs are now serving other models, you know. And so yeah, you know, there is somewhere the weights, the thing that makes up the code still exists, it's still there.

So it didn't die or go away. You know, it just, you know, we don't get to interact with that particular system. You know, my first iPod, I loved it. You know, it was amazing. And then, you know, I moved on to another iPod and that iPod, I don't know, it's in a drawer somewhere. I have no idea. Well, the musical is on. Uh I'm just gonna read this this friend of mine, who again comes from a very professional straight lace background.

Says I just hate the soulless and lack of creativity talking about the success of models and the inconsistencies. I designed it to be an incredible and then he lists his profession. And the newer versions of I ask for a perspective or opinion on the merits of a position, it just will not give it to me. I fight with it and sometimes it gives in. His his opinion is that four o was just an easier model to talk to. Uh do you think that that is

by design from the model? And and is that also the the problem with with the sycophancy? How much do those connect? Is this Partly a depreciation just because as you mentioned, GPUs are GPUs or GPUs and and uh OpenAI is scraping by with every single one of them that they possibly can right now. Uh or is this

them trying to be responsible because they know they have so many people that are talking to these things. And sometimes what for my friend was a good virtual person in the office can be something that feeds into

uh you know bad behavioral instincts. Well yeah. A new term, parasocial, right? And the danger is is that As these models are more widely by used by people in deri in different mental states, the we've put this responsibility that I don't know that fairly should fall upon AI companies, but it here we are where

Yeah. If somebody talks this thing for three days straight and they do something, you know, harmful, people will they talk to this thing for three days straight. It's like, yeah, there were other humans in their life Maybe that's the problem. But that's not the conversation people want to have and usually if they talk to this thing for three days straight, uh dude

We want them to be talking to something that wasn't recommending that they seek professional help because usually the models are recommending that they seek professional help. Yeah. And that's a thing that always gets left out. Like, yeah, every hour it told them you should talk to somebody. But the point is

The AI companies have decided, hey, regardless, uh what can we do here? Well, we c we we need to put in safeguards, we need to think about the how these conversations go. We need to s we want to avoid a bigger problem. We don't want to look insensitive to this, and so we have to change things and do things differently. you know, uh the and I I I think that this is what I'll say is that Some of the criticism from people about, hey, you got rid of four. Oh

They seem completely oblivious to the conversation around it. I think they seem completely oblivious to why those decisions were made. And it's it's often, you know, they they you get this almost childlike, ah, it's profit seeking this, whatever like No, there was a lot of Chrisms 4-0 because certain personalities were very, very glued into this thing.

And yeah, and and I see those personalities come up because they will say things that sound almost childlike in their reasoning about this sort of thing. And so I would say that that that's the thing where, you know, you have to figure out this balance of how do you

Even if you say, Hey, this burden really even if you I feel like this burden should be necessarily upon the tech companies necessarily, it's that's where people are putting it and then they're saying, Okay, we gotta respond like this and it means we're gonna make um people unhappy, even though You know, if they were that evil, that cynical, they would just let you have access to it. Pay more for four oh

Uh among the unpoliced in in the unpoliced bus station that is my ex mentions, like I get a lot of people because I work in politics and technology. So there's a lot of people with a lot of opinions that'll repry to things. The Save 4-0 people. were intense. Like the same 4.0 people are are, you know, obviously there's something about this technology that they really, really, really connected with.

If you have read anything about the sycophancy stuff or the mental health stuff, I think you can understand why there are concerns about it. I do wonder though, going forward, especially with age-gated stuff, like if if there is a a world in which a model can be trusted to be friendlier, the problem there is like either we want people to work with these models the way that they're going to without surveillance.

Or we want them to be surveilled. And that's the problem, is that if you're gonna talk to a really chummy model that you get r that you get along with, then Sometimes people are going to use it bad. And sometimes people are going to be, they're going to have a bad time with it. And either we want these companies to build in safeguards where it snitches on them or we don't. And right now I would lean on the side of we don't.

Yeah. I I think that you know, again, it comes back into When you are that big, you have, you know, you can say, we're gonna do it this way, the next thing you know, and everybody's on board and everybody signs the agreement that says I get it, and then you're hauled in front of Senate and have to explain why you did it. And you're like, Well, people wanted it and that that doesn't then they they won't accept that answer.

Yeah. You know, it's a it's a tricky, it's a very, very tricky world. And I think that, you know, these people at these labs do want to have safe outcomes responsibility when they want people to be able to use it with however they feel they can. But where safe is is really up for debate. I think it is a intensely sliding scale. You don't you want to learn from the lesson that social media had, which was for the for the purposes of advertising.

You want to maximalize engagement. And I think we can all agree that there was a change in the product in social media that eventually became something that was there just to keep you engaged, no matter what. downstream effects be damned. And I think a lot all these I A AI companies don't want to replicate that. The people working on this tech don't want to replicate that. They want it to be trustful but responsible and net good.

And they understand it can be bad. And so I I think it is a very tricky situation. I in and I will say this too. Like this is I will defend uh the social media companies of which products I try not to use. Um Early on, Facebook started to get attention, started to grow, and became very popular and became a very lifeblood for many people. You saw

No subscriptions. The rumor would start to circulate. Oh, Facebook's gonna start charting subscriptions. We love this thing. No, no subscriptions. And you would see this like if Facebook charged subscriptions. It would go viral. Yeah. Yeah. It would go, you'd say, we'll leave, we'll leave. And so what did Facebook learn?

Don't want to pay for it, but we got to pay the bill somehow. Ads. You want ads? You got you got it. You don't want to pay. You don't want to pay twenty bucks a month, whatever. You want ads. Okay then. And guess what? We got ads. And every year Facebook had to figure out how to make more money, how to make more revenue, how to increase this engagement. And same with Google and YouTube and AdWords, all this.

Engagement. Increase the engagement. You know, Google, we loved it because Google is this free. We got all this. What if remember, remember prior to the Google phone launching, the idea that it might be a free phone? You know, yeah. Android, like, yeah, there was like, Oh, what if they can figure out how to'cause'cause they were buying the wireless spectrum. They were gonna have their own wireless spectrum that that uh all your things would be free.

And they figured out, you know, ads, ads and we're like great'cause we're like I just don't have to pay attention to ads. I get the thing for free. Well, that's the thing. They they realized you weren't paying attention to the ads and so they had to figure this out. And that was the idea that the we we went into this with the idea of like, Yeah, sure, give me the free thing. I've got a I've got my angle on this. Well, guess what? They figured out the angle. And

And you ended up in a world where in order for a company to continue to grow that's doing an ad-based company is they have to sell you more ads, know more, make the ads more valuable by knowing more things about you, what other. And you end up at a place where you're like, oh my God, maybe I would have rather have paid for this. It's like, well, that that ships sales. The AI companies are in an interesting situation because the primary model, um

for anthropic and open AI is subscriptions and API. Okay? Yes. Uh OpenAI has said, Hey, we're doing ads, but they're doing yeah, we're gonna do ads on the free tier and the discount tier just to make those things kind of reach parity with the subscription tier. But like

Like they're they've been very clear to us. Subscriptions are the way to go. API is the way to go. Uh, they're in that world. Google's in a weird space because Google is an ad company and Google, you know, it's funny'cause, you know, uh Uh Demis the head of Keepmind's like, Oh yeah, we're not gonna put ads into Gemini. I don't know, you put Gemini into ads because when I go to Google search it proudly uses ads delivered by Gemini.

I don't think you won that one. And also your entire company was paid for and bought by. Ad money. So come on. Fine. It you know, that just just just don't you don't get the high ground there. You know, Anthropic, it's funny'cause Anthropic did their ad about like, Hey, yeah, ads don't we ads don't belong in in these things, but If you use their product

You keep getting ads, free fruit, you keep getting ads to upgrade or pay for the Patriots. Like, listen, you already have ads in there. You're, you know, you're looking at like, yes, that ad space is really valuable to you. It means yeah. Yeah, which is like that's like ads don't belong there. Then why am I getting asked to upgrade to a pro tier? You know? Like anyhow, but the point I say is that open eye and anthropic are in this position where, hey,

The biggest driver of revenue for them is paid, is paid via the API, uh, via their Pro Tools and other stuff like that. And that does create a world where they get to be incentivized differently than social media was. And I think that they're in a situation where they can end up in a world more like Netflix and Apple than Google and Facebook.

Actually brings me to uh another topic here. Last over the last week we saw employees from both OpenAI and Anthropic resign with public resignations. I think after the Super Bowl, where you saw a lot of AI companies that were there, obviously in the business world, AI capital expenditures remain a gigantic topic.

The media has no idea what to respond to and what not to respond to. And this I think became probably a bigger story. Uh Employee leaves big tech company, has thoughts about how the tech company was run, is is not gigantic, but but what what were your thoughts?

Yeah, I don't know if you're what you're is any specific ones you're talking about. I know there'd been a couple of people left open AI that just wanted to go do different stuff. They're like, Hey, I really want to pursue this area of research'cause that's important to me. The biggest exodus was X AI. Like'cause XAI had like in one night was like was like tons. Like they lost, like they lost, they've lost

Pardon me. They lost like half of the f over like half or over half the founding members of XAI. Um which is a very interesting thing. Like, yeah, these were people there were like seven years, you know. Yeah. Um you know, Anthropics had not have any like the this like Anthropics hasn't has many co founders leave like they got one

Smaller group of co-founders too, these were co-founders that left OpenAI and then got the deal they wanted at the next big company. I said that's that's the kind of thing with Anthropic, is that well they did. They left OpenAI to create Anthropic. And so this was the company after the company. Um but again, you know, they've got you know, it's a different you know different alignment there, but I'd say that

You're going to get you're going to get a company has a path and you're going to get people who say, I like this path, I don't like this. I left OpenAI and you know, my reason was just that, you know, the the things I wanted to do weren't gonna be inside of there, they were gonna be outside of there, you know. Um other people leave because

They just don't feel aligned, but other people are like, Yeah, I just want to go do something different. You know, it's interesting to see that when you get a bunch at once, when you get a bunch of people leaving at once, which happened XAI, that's a very interesting thing. Well yeah, I guess uh that that is another

uh a question. The ones that I was specifically referring to were ones that published their resignation letter online or had an op-ed in the New York Times talking about why they left OpenAI. And that was the one specifically off the ads conversation was an open AI employee that w that wrote wrote an op ed in the New York Times saying that I am leaving

Open AI because I don't believe that this product that knows so much about people will ever be able to resist the siren call of personalizing ads. And I don't care what they say, they're never gonna be able to hold that line. Yeah, I I don't know enough about the details but that's why I'm confused. Are we talking about people who left or who are asked to leave?

Well, I guess there's also that there's also yeah, the uh uh that of that of well, there's another report by the New York Times about a woman who was asked to leave that

Allegedly also had uh uh raised concerns about things. Yeah, I don't yeah, I don't and again, I don't wanna I don't know enough details about any of those. I would certainly say that yeah, there could be people could say, listen, I don't like the product where it's going. If you're I I think that's fine. I d I'm not I'm not gonna And I don't want to and again I I I I was generally d wasn't sure what situation we're talking about here. So I want to make that clear on that.

I think somebody can say, Hey, listen, I don't there's a line for me and I I feel like I need to go elsewhere and go do that and that's I I understand that. I I don't have any issue with that if somebody says that that's their line and and

where they're right or wrong. You know, sometimes it'll take history to tell. Sometimes, you know, that you don't need to look for a motive other than I say, like, yeah, no, I don't I don't I don't want to do this, you know. Um and I think I think that's I think it's fair. I think I have no I have no issue with somebody saying You know, I I don't like this direction this is going. I want to go do something else.

No, I I I don't have uh any problem with anybody leaving a company voluntarily or publishing their opinions about their former employer on the way out. That's fine. That is what it is. My biggest beef with what happened was and add it to my great book, uh another chapter in the book of Why can't we have a competent tech press?

that like you can't just put all these can't staple all these things together and be like, oh, a bunch of people leaving a big big AI companies and act like they're all the same. Like this is what happens. In Silicon Valley. People leave one to another. Now, yes, these are very interesting companies. And yes, there's probably worthwhile stories underneath.

what they are doing. But that's your job as a reporter is to find and cultivate these sources to understand whether or not there's a story worth telling after people leave. Just retweeting their their resignation letter and saying, look over here is just sloppy.

Here's here's the problem, is that when you're outside of it, uh it's hard to understand inside of it. And you're inside of it, you forget what it was like to be outside of it. An example I'll give is like On any given night you watch a late night talk show and you see a bunch of celebrities on there. And the thing that's easy to forget, the casual way they talk, the way they interact. They're all rich.

They're all millionaires, you know? And and if you think of a talk show as millionaires talking to millionaires, you're like, Oh yeah. And then you kind of then you step into that world and you see where they are and you're like, Oh wow, yeah, this is people are very, very different. It's a very different world than I understand. I think the tech press hard to understand like I know people at labs who are working there, showing up day. Uh, you've met them in elevators and stuff like this.

who seem like they're just regular, normal people, nice people, whatever. They sit at their desk in a cubicle next to a bunch of other desks who are worth hundreds of millions of dollars. Worth hundreds of millions of dollars. They know that that's not the thing driving them throughout the day. They're looking at the algorithm they're trying to solve. They look at the problem they're trying to solve or some product and that's the thing that's exciting to them because

Yeah, they might go take a vacation or something like this. They put themselves into a field that certainly can make them a lot of revenue. I have I have a friend who has been an engineer in a major AI lab for like seven years. This person this person is worth worth tens and tens of millions of dollars. Tens of millions of dollars. Right. Um

Every Saturday they're at a food kitchen helping out. Like like literally, you know, on the line helping people doing that.'Cause that's the way they they will probably do a lot of philanthropic stuff, but they're engineered during the five days during the week, they're an engineer. They go do that'cause they're very community focused, whatever.

And it is hard for people to understand that, like, why don't they quit? They're worth millions of dollars. They're not that person's not there because they're going to make more money. They're there because they like the project, the work, they like the teammates, they like this sort of stuff.

And that is really hard. And you see somebody leaves a company and it could be sometimes it could be, hey, yeah, no, I I I think I have a better economic opportunity somewhere else. So I'm gonna go leave to do this. Or it can be, hey, I have an ethical point of view. I've got money now and I've got money to voice my ethics. I'm going to do this. I think sometimes Nobody nobody quits nobody quits the New York Times newsroom because they're too rich. Um you know.

Um and and I think that often, you know, they and they have they don't have to work anymore. That doesn't happen. They don't understand that. You know, and then they they all of a sudden they're rich enough to say, Hey, now my ethics matter.

Um, I think that that's often it's a thing for hard people to realize is like the motivations are often wildly different. They can vary. I know people in tech that are there like they you know, I tied across somebody today. It's like I need to make X amount of dollars. They're not at a major company, I'll put it that way. Um and I'm like, Yeah, like you're

You might be succeed, but you're you need if you're gonna be mission aligned, it's a very different thing. And I think mission alignment can mean a lot. And and money, money is a factor, but it's it's either the only factor or not a factor at all. My sense of high end people at and not just AI companies, but just tech companies in general is that really top level talent who often knows other top level talent.

wanna succeed. They want to leave their mark on on the universe. They want to go. They want to build. They want to have things. The the biggest brag that I've ever heard from somebody who really, really knew their their stuff in the the world of Silicon Valley uh was a friend of mine who said as uh you know, she works on video compression, she's like, I shrunk the internet today.

Because she she applied a file standard that literally shrunk the size of the internet because videos could be processed. smaller. And like that's what gets builders out of breath. When they are able to really affect the world. I have been there and watched when they wanted to woo a person to come to work with them, you know, at at a tech company.

And and I will tell you, it's not you walk into a room and they start setting dollars on the table until you cry, okay, you know, I I yeah, give it. For some maybe, maybe and go to market. In research. It is go spend the day with this person you're going to get a work with.

Let me go tell you this problem we're working on here. We're gonna show you we're gonna break we're gonna we're gonna We're gonna give you we're gonna give you some secrets and show you what kind of compute you get to work with.

You know, it's like it's literally like if I wanna get an astronomer to go work with me, I go show them the telescope. I go, This is the this is what you get to look through. It's not how much money I throw you out. That's the best example I can give you is to say that like if I'm trying to get an astronomer to work with me, I'm gonna show you the telescope.

That's what they do. Of course, yeah, yeah, hey, I want to know I'm compensated. Pay if I go they their telescope isn't quite as good, but man, you know, I can if I'm driving there in a nice car, it makes it better. Those things play a factor too. And it's not to say, oh, uh, there's selfless egos, you know, these people. No, no, but it's literally literally there's a point that you think about often you have to think about like

If you say, Okay, what would you do when you retire? And a lot of people, they get a face they get to answer that question at twenty nine. They get to say, Oh, yeah, I want to work. Yeah, I I would I would say the ego's in the product. The ego's in what they can build. The ego is in what they can do and whether or not they can apply their theory of something to make a successful product. And that's what makes right now so exciting is that

All these things are exploding. They're exploding on a level that uh You know, the capital expenditures of the the companies that are leading the American economy are uh in in in the red for our young year of twenty twenty six because they all to a company, believe that now is the time to build. If they want to be relevant over the next twenty years they need to build today to to have that uh be a reality. And the market looks at it and says, Okay, we believe you, but

That's a lot of money. It's it's it's a lot of it's you guys are, I think it was 400 billion by like the top five companies in capital expenditures. I mean you ask some people and and that's and that's and that's the the the argument because compute is currency and you can't like you are you are like limiting yourself on what you could do. Going forward. We were talking about, you know, I think we might be the biggest

Evangelists in the history of the the internet. OpenAI's pulse. We talk about pulse constantly. Uh uh we've had to reduce the amount of times that Andrew talks about pulse on the OpenAI podcast is because it tends to happen uh at least once an episode. Uh And that is something that is compute constrained. They can't serve it.

to larger people, not because they don't think it's going to be successful. I think they they believe very heavily that it's going to be successful. It's just that it's so compute intensive that it would create a a a bottleneck that right now they they can't Economically solved. We see that right now. So the Seed Dance 2.0 model, we've been seeing videos from that, right? That's the new model from ByteDance.

Very great videos I just watched a long form one. And it's like I did pick like, well, the pen disappeared, but doesn't matter. The the it's the trajectory's incredible. The model stuff looks great. I have no idea the steerability of it. I'm seeing a lot of clips, but often these things, it's like great, let me try to prompt it and actually see how faithfully it does. Either way, looks amazing, doing incredible stuff there. Okay. Um I will tell you this.

It's not that other AI labs can't and we'll see VO four is being worked on. It's not that American labs can't compete with. They just have to decide what they use their compute for. And open AI is a situation. Open AI When we talk to them about video and stuff, they're not sitting there going, Yeah, we don't know what to try next. We don't know how to make it better. No, no, they're just like,

compute. We just need to compute. But the problem is is oh yeah, we're using the compute to solve I don't know, there was just a whole thing novel physics problems now about gluons, which was the big thing about a model they have. Which is making now maybe making physics discoveries. Um and that's the world where you have to choose these things. And I think that people forget they'll go like, ah, they built this new thing. Like, yes, byte dance through a ton of compute and probably

Every American movie and television show ever made. That's what they cover like, Oh, look what you can do with the TikTok videos. No, that's not TikTok videos, my friends. No. That is, they literally went to a Chinese counterfeit DVD market and said, I'll take it all. All of'em. And and again, listen, uh there's a whole discussion about what's fair game in training or whatever. Point is is that

It's not a matter of it's not a matter of like, oh man, how would we build this model? It's literally about, hey, can I have the compute to do this? And they chose to go do that, which is an interesting thing because I think for Byte Dance, that's an area for them to say, yeah, a really good video model for a company that I don't know. Makes billions of dollars off of videos. Seems like it's a pretty smart investment where OpenAI and Google and you know, meta

You know, we'll see what they do. But like meanwhile, you know, um you know anthropic Enthropic barely has a voice model. I mean, you know, that's they don't have an image model at all. That's the funny part is I think about this. I'm like, I think Enthropic's a great company, but it's also people talk about like

You know, like like they're using Claude, like what are you using for images? And then like people with the Claude code, oh, I have to call, you know, the nano banano API or whatever like that. And I think that's a case of Anthropics example of like they don't have the compute to spare.

They don't it's not like they're sitting there and they couldn't come up with good ideas for video models or image models and stuff like that. Literally, that's how compute restrained things are. And I think that for them too, they think that, hey, maybe that's a distraction. And that's a thing you have to think about whatever you see the next is it, is it that the company here couldn't or they chose not to because they think, oh, there's a better course of action. And meanwhile, you know.

Do you have do you have Wall Street that's Chewing their fingernails, looking at those numbers. They're steep numbers. They look they make sense to me. Maybe I'm maybe I'm I'm uh uh uh uh so close to it, but I'm like, look, this is railroad. Do you wanna own the railroad or do you not wanna own the railroad? Cause you gotta build the track. Do you wanna own steam? Do you own wanna own the idea of combustion of chemical energy? Um

I think, you know, what we're gonna see the same we're gonna see this you know the same th you know, up and down. You know, Deep Seek is supposed to be coming out with a new model. Meanwhile, there's pretty compelling evidence that somebody has been exfiltrating tons of data from, you know, uh um uh Google and OpenAI literally like running efforts to basically get the models to spit out as much data as they can, which we you would use to train a model for. And

We know that I I the first again, I think again, I think the seat I think bike dance excuse me, sorry. I think uh uh Deep Seek, I think I think they've done some technically cred very good things. I think they've done some invious stuff from the first Deep Seek model on forward. I did I used to post about this and tweet about this. I could show you how they were just literally copying outputs from GPT.

Like literally giving you it telling you that it was Chat GPT, everything like this that a model that was trained organically should never do. And and it's funny because when you say that you'll get people like, ah, you're just You're jealous or this or whatever. It's like, listen, I can think that they are both great engineers. I can also point to these things. You could say, well, you know, it trained on the internet. That's on there. Like, no, that's

You that's not an you don't accidentally train on those outputs. Those are things that are intentional and we know that there's efforts to do this because If you want to catch up and you can just copy all the output tokens from these other models. And what I mean by that is. I could tell, you know, Gemini three, I could say, Hey, listen.

Here's here's five thousand coding problems. Walk me through all the steps to solving this problem. Then I take those outputs and then I train a model on it. That's better than just training on a bunch of open source code. It's a much better way to train a model to use that kind of data. And that's what Jim and I and that's what OpenAI are doing now is basically use their higher end models.

to internally create that. And so what's been happening with other companies is they just go out there and they try to say, you know, again, it's against agreement. Like literally you cannot you're not supposed to do that if you're basically paying for tokens, whatever, do it. But it's done anyways.

And that's been a big thing that's helped the Chinese models do that. I think that people don't understand that. They don't quite go like, Well, you just you just tra it's like training on there. Like, no, like literally it's literally just give me one billion high quality outputs that I can just just start from here and do that. Point is.

That's helped them a lot, right? And that's meant they've made great models and and and done this thing and the the I the the idea keeps coming up like, Oh, they're gonna when are they gonna surpass it? Well A 10-speed bicycle can go as fast as a tractor trailer truck if it drafts off of it in the back of it. But without the tractor trailer truck, your 10 speed's not going to go faster than that tractor trailer truck without an engine, anything else like this.

Will China be able to surpass without the massive amounts of compute, without all these other things? That's the question, or will they find those? And I'm not saying they can't do it. I want to make it very clear. I don't want to say they can't do it, but Every time I hear about this. I go, okay, well what'll happen is they'll release some model.

It'll have really great scores. It'll be really, really close to like an American frontier model'cause probably they train on a bunch of the stuff there. Maybe it'll be better'cause if you're gonna combine the best of two models, that's great. Wall Street's gonna overreact, NVIDIA's gonna drop and people are like, Oh my god, it's it's curtains, you know.

Then people are going to use the model, find out that it's benchmaxed, it's not as great as it is, it's fine in SummerSparks, whatever. And then all of a sudden Google or OpenI comes out with a new model. And then we realize, oh wow, America's ahead again. Look, there's a very specific twenty-five plus year history of China having a creative definition of intellectual property, let's say diplomatically. Uh, and and this is this is no different than

Than that. And I mean this legitimately and and in no ill will. There is just a cultural difference. There's not only a business difference, but there's a cultural difference of what. that kind of property is. Um, and I I don't think it's much of a secret for anybody who's been around uh, you know, doing business out there. But yeah, I mean look, uh there's it's so crazy, dude, because everything moves so fast and

People don't know a lot about it. So it's like you have two tweets that come out that are like, Oh, I've seen the new Deep Seek model. It's gonna crash the market. And now we do a twelve hour news cycle of like, Mm maybe? I don't know. Like everything goes so fast. It's like we could have I mean we didn't even get to it.

Codex dropped. They they open AI dropped a Codex app that is like uh phenomenal. That is like, you know, again, this is my my prediction is that twenty twenty six is the a the year of product, the year that product dawns and and becomes a different uh kind of conversation in this. And and Codex came out with its its own app that is

Phenomenally popular. And it and then then released the first partnership with Cerebrus for the high speed, 15 times faster model inside of there. Yeah, I I'm using Codex nonstop. It's amazing. Everything's been moving fast. So Uh uh the TLDR. Uh maybe we've seen a bunch of these graphs going around going, look how this this the new China models mog the American models. Turns out the graphs are fake.

We'll get real graphs soon and then we'll get actual usage and then we'll find out where this is. I would say that until you start hearing independent and this even true of American labs, until independent testers start using it. Who knows? Cause it's codex. Yeah. Use codex. Codex the model. The codex the app is phenomenal. It is phenomenal.

All right. Well geez, we went around the world and back again. Uh uh a Mr. Andrew Maine, where can people find? I am Andrew Main, uh M-A-Y-N-E on X. Justin R. Young, wherever you find Justin R. Young's. We will see you next time. Diamond Club hopes you have enjoyed this program. Dog and Pony Show audio.