OpenAI Just Gave Away Their Secret Formula... For Free?

Josh: The unthinkable has just happened open ai Josh: has released an open source model open ai Josh: has been closed ai since the time that i knew them Josh: they have been named themselves open ai they were not Josh: open source they have finally released an open source model and surprise surprise Josh: it's actually really great and i think the downstream implications of an open Josh: source model from a company like this that is this good are really it's a really

Josh: big deal i think this really matters a lot just yesterday they announced the release of GPT-OSS.

Josh: There are two models. There's a 120 billion parameter model and there's a 20 Josh: billion parameter model. We're going to get into benchmarks. Josh: We're going to get into how good they are. Josh: But the idea is that OpenAI has actually released an open source model. Josh: And this can compare to the Chinese models because we recently had DeepSeek and we've had Kimi. Josh: And those would be very good. But this is the first really solid American-based open source model.

Josh: So Ijaz, I know you've been kind of digging in the weeds about how this works.

Josh: Can you explain us exactly why this is a big deal why this happened what's going on here Ejaaz: Yeah it's it's pretty huge so so here Ejaaz: are the hot highlights um as you mentioned there's two Ejaaz: models that came out the 20 billion parameter model which is actually small Ejaaz: enough to run on your mobile phone right now and they have a 120 billion parameter Ejaaz: model which is big but still small enough to run on a high performance laptop

Ejaaz: so if you guys have a macbook out there jump in go for it um it's fully customizable. Ejaaz: So remember, open source means Ejaaz: that you can literally have access to the design of the entire model.

Ejaaz: It's like OpenAI giving away their secret recipe to how their frontier models Ejaaz: work. And you can kind of like recreate it at home. Ejaaz: This means that you can customize it to any kind of use case that you want, Ejaaz: give it access to all your personal hard drives, tools, data, Ejaaz: and it can do wonderful stuff. Ejaaz: But Josh, here's the amazing part. Ejaaz: On paper, these models are as good as GPT-4 mini models, which is, Ejaaz: it's pretty impressive, right?

Ejaaz: In practice and i've been playing around with it for the last few hours they're Ejaaz: as good in my opinion and actually quicker than Ejaaz: gpt-03 which is their frontier model and i Ejaaz: mean this across like everything so Ejaaz: reasoning um it spits out answers super quickly and i can see its reasoning Ejaaz: it happens in like a couple of seconds and i'm so used to waiting like 30 seconds Ejaaz: to a couple minutes on gpt-03 josh so it's pretty impressive and an insane unlock

Ejaaz: on coding it's as good and on creativity as well. Ejaaz: So I'm my mind's pretty blown at all of this, right? Josh, what do you what do you think? Josh: Yeah, so here's why it's impressive to me is because a lot of the times I don't Josh: really care to use the outer bands of what a model is capable of. Josh: Like I am not doing deep PhD level research. I'm not solving these math Olympiad questions. Josh: I'm just trying to ask it a few normal questions and get some answers.

Josh: And what these models do is an excellent job at serving that need. Josh: They're not going to go out and solve the world's hardest problems, Josh: but neither do I. I don't want to solve those problems. Josh: I just kind of want the information that I want, whether it be just a normal Josh: Google type search or whether it be asking it some miscellaneous question about Josh: some work that I'm doing.

Josh: It's really good at answering that. So I think initial impressions, Josh: because they did allow you to test it publicly through their website, Josh: it's just really good at the things that I want.

Josh: So the fact that I can run one of these models on a local device on my iPhone, Josh: well, it feels like we're reaching this place that AI is starting to become Josh: really interesting because for so long we've had compute handled fully on the Josh: cloud and now this is the first time where Josh: Compute can really happen on your computer. It could happen on your laptop.

Josh: I could download the model and I could actually store the model, Josh: the 120 billion parameter model on a 56 gigabyte USB drive. Josh: So you can take the collective knowledge of the world and put it on a tiny little USB drive. Josh: And granted, it needs a bit of a bigger machine to actually run those parameters, Josh: but you can install all the weights. It's 56 gigabytes.

Josh: It's this incredibly powerful package. And it probably, I don't know if this Josh: is true, but it's probably the most condensed knowledge base in the history of humanity. Josh: They've really managed to take a tremendous amount of tokens, Josh: smush them into this little parameter set, and then publish it for people to Josh: use. So for me, I'm really excited. Josh: I like having my own mini portable models. I am excited to download, Josh: try it out, run it on my MacBook.

Josh: I'm not sure I could run the 120 billion parameter model, but at least the 20B Josh: and give it a shot and see how it works. Ejaaz: You need to get the latest MacBook, Josh. I know, I got to upgrade. We can test that out. Ejaaz: What I also love about it is it's fully private, right? So you can give it access Ejaaz: to your personal hard drive, your Apple Notes, whatever you store on your computer, basically. Ejaaz: And you can basically instruct the model to use those different tools.

Ejaaz: So one review that I keep seeing from a number of people who have been testing Ejaaz: it so far is that it's incredibly great and intuitive at tool use. Ejaaz: And the reason why this is such a big deal is a lot of the Frontier models right Ejaaz: now, when they allow you to give access to different tools, they're kind of clunky. Ejaaz: The model doesn't actually know when to use a specific tool and when not to.

Ejaaz: But these models are super intuitive, which is great. The privacy thing is also Ejaaz: a big thing because you kind of Ejaaz: don't want to be giving all your personal information away to Sam Altman. Ejaaz: But you want a highly personalized model. Ejaaz: And I think if I was to condense this entire model release in a single sentence, Ejaaz: Joss, I think I would say it is the epitome of privacy and personalization in an AI model so far.

Ejaaz: It is that good. it is swift it is cheap and I'm going to replace it completely Ejaaz: with all my GPT-4.0 queries as you said earlier like, Ejaaz: Who needs to use the basic models anymore when you have access to this?

Josh: Yeah. So it's funny you say that you're going to swap it because I don't think I'm going to swap it. Josh: I still am not sure I personally have a use case right now because I love the Josh: context. I want the memory. Josh: I like having it all server side where it kind of knows everything about me.

Josh: I guess in the case that I wanted to really make it a more intimate model experience Josh: where you want to sync it up with like journal entries or your camera roll or Josh: whatever, whatever interesting like personal things, this would be a really cool use case.

Josh: I think for the people who are curious why this matters to them, Josh: well, we could talk a little briefly about like the second order effects of Josh: having open source models as powerful, because what that allows you to do is Josh: to serve queries from a local machine. Josh: So if you are using an app or let's say you're an app developer and you're building Josh: an application and your app is serving millions of requests because it's a GPT wrapper.

Josh: Well, what you could do now is instead of paying API calls to the OpenAI server, Josh: you can actually just run your own local server, use this model, Josh: and then serve all that data for the cost of the electricity. Josh: And that's a really big unlock for the amount of compute that's going to be Josh: available for not only developers, but for the cost of the users in a lot of these applications.

Josh: So for the applications that aren't doing this crazy moon math and that are Josh: just kind of serving basic queries all day long, this like really significantly drops the cost. Josh: It increases the privacy, like you mentioned. And there's a ton of really important Josh: upsides to open source models that we just haven't seen up until now.

Josh: And I'm very excited to see come forward. Ejaaz: Well, Josh, the thing with most of these open source models, Ejaaz: we spoke about actually two major Chinese open source models that were released last week. Ejaaz: It's not accessible to everyone. Like you and me aren't necessarily going to Ejaaz: go to Hugging Face, a completely separate website, download these models, Ejaaz: run the command line interface. Ejaaz: Most of the listeners on the show doesn't even know what that means.

Ejaaz: I don't even know if I know what that means, right? Ejaaz: But here you have a lovely created website where you could just kind of log Ejaaz: on and play around with these open source models. And that's exactly what I've been doing. Ejaaz: I actually have a few kind of demo queries that I ran yesterday, Josh. Josh: Yeah, walk us through, let's see.

Ejaaz: Okay, so there's an incredibly complex test, which a lot of these AI models, Ejaaz: which cost hundreds of billions of dollars to train, can't quite answer. Ejaaz: And that is how many R's, the letter R's are there in the word strawberry? Most say two. Josh: The bar's on the floor, Ejaaz: Huh? Yeah, if we were to go with most models, they say two. They're convinced that they are only two. Ejaaz: And I ran that test today, rather yesterday, with these open source models,

Ejaaz: and it correctly guessed three, Josh. So we're one for one right now. Josh: We're on our way. Ejaaz: But then I was like, okay, we live in New York City. I love this place. Ejaaz: I'm feeling a little poetic today. Can you write me a sonnet? Ejaaz: And my goal with this wasn't to test whether it could just write a poem. Ejaaz: It was to test how quickly it could figure it out.

Ejaaz: And as you see it thought for a couple of seconds on this so it literally spat Ejaaz: this out in two seconds um and it was structured really well you know it kind Ejaaz: of flowed would i be you know reciting this out loud to the public no but you Ejaaz: know i was pretty impressed. Ejaaz: And then, Josh, I was thinking, you know, what's so unique about open source models? Ejaaz: You just went through a really good list of why open source models work.

Ejaaz: But I was curious as to why these specific open source models were better than Ejaaz: other open source models or maybe even other centralized models. Ejaaz: So I wrote a query. I decided to ask it. I was like, you know, Ejaaz: tell me some things that you could do that are the larger centralized models. Ejaaz: And I spat out a really good list. I'm not going to go through all of them, Ejaaz: but, you know, some of the things that we've highlighted so far, you can fine tune it.

Ejaaz: It's privacy. See, I really like this point that it made, Josh, Ejaaz: that it just shows that AI is probably getting smarter than us, Ejaaz: which is you can custom inject your own data into these models. Ejaaz: Now, without kind of digging deeper into this, when you use a centralized model, Ejaaz: it's already pre-trained on a bunch of data that companies like Anthropic and Ejaaz: Google have already fed it. Ejaaz: And so it's kind of formed its own personality, right?

Ejaaz: So you can't change the model's personality on a centralized model.

Ejaaz: But with an open model you have full reign to do whatever you want and so if Ejaaz: you were feeling kind of uh adventurous you could use your own data and make Ejaaz: it super personal and customizable so i thought that was really cool and fun Ejaaz: demo josh have you been playing around with this.

Josh: Yeah it's um it's it's smart it's fun it's smart i wouldn't say it's anything Josh: novel the like query results that i get are you know on par with everything Josh: else i don't notice the difference which is good because it means they're performing Josh: very well it's not like i feel like i'm getting degraded performance because Josh: I'm using a smaller model. Josh: But it's just like it's nothing too different, I would say.

Josh: The differences, I mean, again, all this boils down to the differences of it Josh: being open source versus being Ejaaz: Run on the server. Well, let me challenge you that, right? OK, Ejaaz: so you're saying it's good but nothing novel. Ejaaz: Would you say it's as good as GPT-4.0, Ejaaz: minus the memory let's just put memory aside for a second would you use it if Ejaaz: it had memory capability.

Josh: Actually no probably not um i still wouldn't Josh: because i love my desktop application too much i Josh: love my mobile app too much and i like that the conversations are Josh: shared in the cloud um so i can use them on my phone i could Josh: start on my laptop and go back and forth so even in Josh: that case i'm probably still not a user um because Josh: the convenience factor but there are there are a

Josh: lot of people and a lot of industries that would be and this is actually something probably Josh: worth surfacing is the new industries that are now able to Josh: benefit from this because a lot of industries have Josh: a tough time using these AI models because Josh: of the data privacy concerns particularly I mean if you think about a Josh: healthcare industry people who are dealing with patients data it's

Josh: very challenging for them to fork it over to open AI and just trust that they're Josh: going to keep it safe so what this does is it actually allows companies that Josh: are in like the healthcare industry the finance industry who's dealing with Josh: very like high touch personal finance the legal industry who's dealing with Josh: a lot of legality government and defense a lot of these industries that were Josh: not previously able to use these popular AI models,

Josh: well, now they have a pretty good model that they could run locally on their machines. Josh: And that doesn't have any possibility of actually leaking out their customer Josh: data, leaking out financials or healthcare data or, or like any sort of legal documents.

Josh: And, and that feels like a super powerful unlock. So for them, Josh: it feels like a no brainer, obviously get the 120 B model running on a local Josh: machine inside of your office, and you can load it up with all this context. Josh: And that seems to be who this would be most impacting, right? Ejaaz: But still to that point, I wonder how many of these companies can be bothered Ejaaz: to do that themselves and run their own internal kind of like infrastructure.

Ejaaz: I'm thinking about OpenAI, who cracked, I think, $10 billion in annual recurring Ejaaz: revenue this week, which is like a major milestone. Ejaaz: And a good chunk of that, I think 33% of that is for enterprise customers. Ejaaz: And to your point, like these enterprise customers don't wanna be giving open Ejaaz: AI their entire data. You know, they can be used to train other AI models.

Ejaaz: So their fix or solution right now is they use kind of like private cloud instances, Ejaaz: that I think are supplied by Microsoft by their Azure cloud service or something like that. Ejaaz: And I wonder if they chose that, Ejaaz: One, because there wasn't any open source models available or because they kind Ejaaz: of just want to offload that to Microsoft to deal with.

Ejaaz: My gut tells me they're going to want to go with the latter, Ejaaz: which is like, you know, just give it to some kind of cloud provider to deal with themselves. Ejaaz: And they just trust Microsoft because it's a big brand name. Ejaaz: But yeah, I don't really know how they'll materialize. I still think, Ejaaz: and maybe this is because of my experience in crypto, Josh, that the open source Ejaaz: models are still for like people that are at the fringe that are really experimenting

Ejaaz: with these things. but maybe don't have billions of dollars.

Josh: Yeah, that could be right. It'll be interesting to see how it plays out on all Josh: scale of businesses because I mean, as a, like I think of a lot of indie devs Josh: that I follow on Twitter and I see them all the time Josh: just running local servers and they just, if they had this local model that Josh: they could run on their machine and it takes the cost per query down from like Josh: a penny to zero, that's like a big zero to one change.

Josh: So he does this model special because there are also a number of breakthroughs Josh: that occurred in order to make this possible, Josh: in order to condense this knowledge to be so tight so here's this Josh: tweet from the professor talking about the cool tech tweaks in Josh: this new model and what open ai was able to achieve some of Josh: these i believe are novel some of these are seen before um if Josh: you look at point two mixture of experts we're familiar with mixture of experts

Josh: we've seen other companies use that like kimmy and deep Josh: seek basically instead of one brain doing everything the ai Josh: has this team of experts that are kind of like mini brains Josh: and specialize in different tasks it picks the right expert for Josh: the job and it makes it faster so like instead of Josh: having the entire 120 million parameter model search for one question maybe Josh: you just take a couple million of those parameters that are really good at solving

Josh: math problems and they use it and that that's what brings compute down the first Josh: point is this thing called the sliding window attention so if you imagine an Josh: ai is like reading a really long book Josh: It can only focus on a few pages at a time this trick Josh: kind of lets it slide its focus window along the text so Josh: when you think of a context window generally it's fixed right where you can see Josh: a fixed set of data this sliding window

Josh: attention allows you to kind of move that context back and forth a Josh: little bit so it takes what would have normally been Josh: a narrow context window and extends it out a little bit to Josh: the side so you get a little bit more context which is great for a Josh: smaller model again you really want to consider that all of these are Josh: are optimized for this microscopic scale that Josh: can literally run on your phone and then the third point is this

Josh: thing called rope with yarn which sounds like a cat toy but this Josh: is how the ai keeps track of the order of words so like the position Josh: of the words in a sentence um so rope Josh: you could imagine it like like the twisty math way to do Josh: it and yarn makes it stretch further for really long stuff Josh: so we have the context window that is Josh: sliding we have this rope with yarn that allows you Josh: to just kind of like stretch the words a little bit further and

Josh: then we have attention sinks which is the last one which is Josh: there's a problem when ai is dealing with these endless chats that Josh: lets it it kind of sinks in or ignores the boring old Josh: info so it can pay attention to the new stuff so basically what it Josh: is is if you're having a long chat with it and it determines hey this stuff Josh: is kind of boring i don't need to remember it it'll actually just throw it away

Josh: and it'll increase that context window a little bit so again hyper optimizing Josh: for for the small context window that it has and those are kind of the key four Josh: breakthroughs that made this special again i'm not sure any of them are particularly novel, Josh: But when combined together, that's what allows you to get these 04 mini results Josh: or even 03 results on the larger model on something that can run locally on your laptop.

Josh: So it's a pretty interesting set of breakthroughs. I think a lot of times OpenAI, Josh: we talk about them because of their feature breakthroughs, not really their Josh: technical breakthroughs. Josh: I think a lot of times the technical breakthroughs are reserved for like the Josh: Kimi models or the DeepSeq models Josh: where they really kind of break open the barrier of what's possible.

Josh: But I don't want to discredit OpenAI because these are pretty interesting things Josh: that they've managed to combine together into this like one cohesive, Josh: tiny little model, and then just gave it away. Ejaaz: Yeah. I mean, they actually have a history of front-running open source frontier breakthroughs.

Ejaaz: If you remember when DeepSeek got deployed, Josh, one of their primary training Ejaaz: methods was reinforcement learning, which was pioneered by an open AI researcher, Ejaaz: which who probably like now works at Meta. Ejaaz: Yeah, and I was I was I was looking at the feature that you mentioned just not Ejaaz: the feature, but the breakthrough sliding window attention, and you mentioned Ejaaz: that it can basically toggle reasoning.

Ejaaz: And I was pleasantly surprised to just notice that on the actual interface of Ejaaz: the models here, Josh, can you see over here? Ejaaz: You can toggle between reasoning levels of high, medium and low.

Ejaaz: So depending on what your prompt or query is, if it is kind of like a low level Ejaaz: query where you're like hey just record this shopping or grocery list you know Ejaaz: that's probably like a medium or a low query so oh it's pretty cool to to see Ejaaz: that surface to the user like see it actively being used. Josh: Yeah, no, super cool. I think I like the fine tuning of it.

Josh: And again, allowing you to kind of choose your intelligence levels, Josh: because I imagine a lot of average people just don't, a lot of average queries Josh: just don't need that much compute. Josh: So if you can toggle it for the low reasoning level and get your answers, Josh: that that's amazing. Super fast, super cheap.

Ejaaz: Did you see that trending tweet earlier this week, Josh, which basically said Ejaaz: that the majority of ChatGPT users have never used a different model than ChatGPT 4.0? Josh: I haven't seen it, but that makes sense.

Ejaaz: Yeah i i feel like the bulk of people i was chatting to Ejaaz: my sister yesterday and she was kind of Ejaaz: like using it for some research project at work and the Ejaaz: screenshot she sent me over was foro and i was like hey you know like Ejaaz: you could just run this on like a model that's like Ejaaz: five times better than this right uh we'll come Ejaaz: up with a much more creative set of ideas so just made me think that

Ejaaz: like i don't know how many people like care that they are like Ejaaz: these brand new novel models and maybe um you know Ejaaz: this kind of like basic model is good enough for everyone i don't know Ejaaz: but um but moving on josh um there Ejaaz: was a big question that popped into my head as Ejaaz: soon as these models released which was are they as good Ejaaz: as the chinese open source models right i wanted Ejaaz: to get some opinions from people and and the reason

Ejaaz: why this matters i'm just give the listeners some context Ejaaz: is china has been the number one Ejaaz: nation to put out the best open source Ejaaz: models over the last 12 months it started with deep seek Ejaaz: and then alibaba's quen models got involved Ejaaz: and then recently we had kimmy k2 and i think Ejaaz: there was another ai lab out of china which came out so they Ejaaz: have outside of america the highest density.

Ejaaz: Of the top ai researchers they all come out of this one university Ejaaz: zinghua i believe they kind of like partially work Ejaaz: or train in the u.s as well so they've got this like kind of hybrid ai Ejaaz: mentality of how to build these models and they come up with a lot of these Ejaaz: frontier breakthroughs um kimmy k2 for context had uh one trillion parameters Ejaaz: in their model right comparing this to like 120 billion and 20 billion parameters

Ejaaz: models from open air i was curious like does this beat them to the punch some people josh.

Ejaaz: Don't think so okay this guy jason lee Ejaaz: he asks uh is the gpt oss stronger Ejaaz: than quen or kimmy or chinese open models and then Ejaaz: he later kind of quote tweets that tweet and says answer the model is complete Ejaaz: junk it's a hallucination machine overfit to reasoning benchmarks and has absolutely Ejaaz: zero recall ability so a few things he's mentioning here is one it hallucinates Ejaaz: a lot so it kind of makes up jargon terms,

Ejaaz: ideas, or parameters that didn't really exist before. Ejaaz: Number two, he's saying that OpenAI designed this model purely so that it will Ejaaz: do well on the exams, which are the benchmarks that rate how these models compare to each other. Ejaaz: So they're saying that OpenAI optimized the model to kind of like do really Ejaaz: well at those tests, but actually fail at everything else, which is what people want to use it for.

Ejaaz: And the final point that he makes is that it has zero recall ability, Ejaaz: which is something you mentioned earlier, Josh, which says it doesn't have memory Ejaaz: or context so you can have a conversation and then open up another conversation Ejaaz: and it's completely forgotten about the context that it has for you from that Ejaaz: initial conversation okay.

Josh: So not not the best not to be unfair to open ai but it feels like they delayed Josh: this model a good bit of times oh yeah and they wanted it to look good and it Josh: intuitively makes sense to me that they would be kind of optimizing for benchmarks Josh: with this one um but nonetheless it's still impressive i'm seeing this big wall Josh: of text now what is what is this what is this post here Ejaaz: Well it's this post from uh one of these accounts i follow and they have an

Ejaaz: interesting section here which says comparison to other open weights oh sick.

Josh: Yeah what is this Ejaaz: So he goes while the larger gpt oss Ejaaz: 120 billion parameter model does not come Ejaaz: in above deep seek r1 so he's saying that deep seek r1 Ejaaz: just beats it out the park it is notable that Ejaaz: it is significantly smaller in both total and active Ejaaz: parameters than both of those models deep seek Ejaaz: r1 has 671 billion total parameters and Ejaaz: 37 billion active parameters and is released natively right which makes it 10x

Ejaaz: larger than gpt's 120 billion parameter models but what he's saying is even Ejaaz: though gpt's model is smaller and doesn't perform as well as deep seek it's Ejaaz: still mightily impressive for its size.

Josh: Okay that's cool because that gets back to the point we made earlier in the Josh: show that this is probably the most densely condensed Josh: however you want to say it like base of Josh: knowledge in the world they've used a lot of efficiency gains Josh: to squeeze the most out of it so in this small model Josh: it is i guess if we're optimizing maybe we Josh: can make up a metric here on the show which is like um output per

Josh: per parameter or something like that like based on the total parameter Josh: count of this model it gives you the best value per Josh: token and that seems to be where this falls Josh: in line where it's not going to blow any other open source model out of the Josh: water but in terms of its size the fact that we can Josh: take a phone and literally run one of these models on a phone and Josh: you could go anywhere in the world with no service and have access to these models running

Josh: on a laptop or whatever mobile device that that's super Josh: powerful and that's not something that is easy to do with the other open source Josh: models so perhaps that's the advantage that open ai has it's just the density Josh: of intelligence and the efficiency of these parameters that they've given to Josh: us versus just being this like home run open source model that is going for the frontier, Josh: it's just a little bit of a different approach.

Ejaaz: Yeah, we need like a small but mighty ranking on this show, Josh, Ejaaz: that we can kind of like run every week when these companies release a new model. Ejaaz: No, but it got me thinking, if we zoomed out of that question, Ejaaz: right, because we're talking about small models versus large models, Ejaaz: parameters and how effectively they use versus other models that are bigger.

Ejaaz: What really matters in this, Josh? In my opinion, it's user experience and how Ejaaz: useful these models are to my daily life, right? Ejaaz: At the end of the day, I kind of don't really care what size that model is unless Ejaaz: it's useful for me, right? It could be small, it could be personal, it could be private.

Ejaaz: It depends on, I guess, the use case at the time. And I have a feeling that Ejaaz: the trend of how technology typically goes, you kind of want a really high-performant Ejaaz: small model, eventually. Ejaaz: Right? I try and think about like us using computers for the first time, Ejaaz: you know, back in our dinosaur age. Ejaaz: And then, you know, it all being condensed on a tiny metal slab that we now Ejaaz: use every day. And we can pretty much work from remotely from wherever.

Ejaaz: And I feel like this is where models are going to go. They're going to become Ejaaz: more private. They're going to become more personal. Ejaaz: Maybe it'll be a combination of, you know, it running locally on your device Ejaaz: versus cloud inference and trusting certain providers. Ejaaz: I don't know how it's going to fall out, but I think Like it's not a zero to Ejaaz: one. It's not a black or white situation.

Ejaaz: I don't think everyone's just going to go with large centralized models that Ejaaz: they can inference from the cloud.

Ejaaz: I think it'll be a mixture of both. And how that materializes, Ejaaz: I don't know, but it's an interesting one to ponder. Josh: Yeah, I think this is funny. This is going to sound very ironic, Josh: but Apple was the person that got this most right. Ejaaz: Sorry, who's Apple again? Josh: Yeah, right. I mean, it sounds ridiculous to say this. And granted, Josh: they did not execute on this at all.

Josh: But in theory, I think they nailed the approach initially, Josh: which was you run local compute where all of Josh: your stuff is so my iphone is the device i never Josh: leave without it is everything about me it is all of my messages my Josh: contacts all the contacts you could ever want from me and then the idea was Josh: they would give you a local model that is integrated and embedded into that Josh: operating system and then if there's anything that requires more compute well

Josh: then they'll send the query off into the cloud but most of it will get done Josh: on your local device because most of it isn't that complicated and i think as Josh: a user when i ask myself what i want from AI. Josh: Well, I just want it to be my ultimate assistant. I just want it to be there Josh: to make my life better. And so much of that is the context. Josh: And Apple going with that model would have been incredible.

Josh: It would have been so great. It would have had the lightweight model that runs Josh: locally, it has all the context of your life, and then it offloads to the cloud. Josh: I still think this model is probably the correct one for optimizing the user Josh: experience. But unfortunately, Apple just has not done that. Josh: So it's up for grabs. I mean, again, Sam Altman's been posting a lot this week, Josh: we do have to tease what's coming because this is probably going to be a huge

Josh: week. There's a high probability we get GPT-5. Josh: And then they've also been talking about their hardware device a little bit. And they're saying how Josh: It's like it's genuinely going to change the world. And I believe the reason Josh: why is because they're taking this Apple approach where they're building the Josh: operating system, they're gathering the context, and then they're just they're Josh: able to serve it now locally on device.

Josh: They're able to go to the cloud when they need more compute. Josh: And it's going to create this really cool, I think, duality of AI where you Josh: have your your super private local one, and then you have the big brain one, Josh: the big brother that's off in the cloud that does all the hard computing for you. Ejaaz: Well, one thing is clear. There are going to be hundreds of models and it's Ejaaz: going to benefit the user, you and I, for so many multiple...

Ejaaz: It's the big company's problems to figure out how these models work together Ejaaz: and which ones get queried. I don't care. Ejaaz: Just give me the good stuff and I'm going to be happy. Ejaaz: Folks, OpenAI has been cooking. This was the first open source models they've Ejaaz: released in six years, Josh. Ejaaz: The last one was 2019 GPT-2, which seems like the stone age and it was only like four years ago.

Ejaaz: Thank you so much for listening. We are pumped to be talking about GPT-5, Ejaaz: which we hope to be released in maybe 24 hours. Ejaaz: Hopefully this week, fingers crossed. I don't know, we might be back on this Ejaaz: camera pretty soon. Stay tuned. Ejaaz: Please like, subscribe, and watch out for all the updates. We're going to release Ejaaz: a bunch of clips as well if you want to kind of like get to the juicy bits as well.

Ejaaz: Share this with your friends and give us feedback. If you want to hear about Ejaaz: different things, things that we haven't covered yet or things that we've spoken Ejaaz: about, but you want to get more clarity on or guests that you want to join the show, let us know. Ejaaz: We're going full force on this and we'll see you on the next one. Josh: Sounds good. See you guys soon. Peace. Music: Music