How can the most powerful AI today, you know, GPT -4, these huge models, how can they also be called a philosophical dead end? Yeah, it's a real head scratcher, isn't it? You've got Richard Sutton, who basically invented modern reinforcement learning, saying the tech winning right now. Yeah. It's not the path to true AGI. Right. And that's what this deep dive is all about. We're going to look past the benchmarks, look at the
actual architecture underneath. We've pulled sources on this critique, but also on the... well, the crazy cost of scaling this stuff. And our goal really is to figure out where the smart money, the smart thinking is going for AGI. Is it just making current models bigger or is it something fundamentally different, like a new way to learn? Okay, so we've got three main areas for you today. First, why Sutton thinks LLMs have this fatal flaw. Second, the absolutely
staggering scale. And yes, the cost of this AI arms race. And third, these new video models that seem to actually kind of... Think over time. Sounds good. Let's dive in. Let's start with Sutton, the Turing Award winner. He's not saying LLMs are useless, right? No, not at all. They're amazing prediction machines, obviously. But Sutton comes from reinforcement learning, which is all about agents acting in the world and learning from feedback. He says LLMs lack that core loop.
Well, passive. Passive meaning they don't have goals. They can't be surprised. Is that the idea? Exactly. And crucially, they don't learn from consequences. They're just incredibly good mimics of human text. They don't truly understand the real world impact of the words they string together. OK, I get the difference. But isn't being passive, goalless, actually safer? I mean, the whole market seems built on making these things predictable, controllable. Why build something goal driven
if it's riskier? That's the big tradeoff. Right. Their safety comes from that passivity. But Sutton argues that very passivity limits their intelligence potential. Just scaling up imitation even to GPT -7 or 8 won't get us to AGI. So he has an alternative. Yeah. He proposes this new architecture called OK. The key thing is it learns on the fly. It doesn't need that massive, hugely expensive pre -training phase that LLMs rely on. Okay,
can we give folks an analogy? So if an LLM is like a giant static textbook that predicts the next word, what's okay? Is it like an agent that can actually, you know, burn its hand on a stove and learn don't touch? That's a pretty good way to put it, yeah. It's about learning through doing, through direct consequence. It's not just copying patterns from a giant pile of text. AGI, in this view, needs these smarter loops. Action,
feedback, memory, even motivation. So if LLMs are flawed because they just imitate, what exactly is the mechanism Sutton proposes for real consequence driven learning? It's about learning via action and direct consequence, not massive pre -trained imitation. Gotcha. OK, so if real intelligence needs this whole new architecture, then this race to just build bigger and bigger LLMs. It's a massive bet, isn't it? Strategically speaking. It really is. And yet the scaling is happening
at a rate that's hard to comprehend. Yeah. Let's talk about those numbers. The sources we saw on OpenAI's compute plans, just wild. Totally wild. They apparently 9x their compute power in 2025 alone. Okay. Huge jump. But then by 2033, the projection is 125 times bigger than that.
Two sec silence. Whoa. okay wait that number 125x it kind of breaks my brain to put that in perspective for you listening that could mean needing more electricity than like the entire country of india uses today for 1 .4 billion people it just sounds Physically almost impossible. It does. It's like stacking Lego blocks of data centers reaching into the clouds. Right. And it explains why we're seeing these massive investments like Nescale raising that record $1 .1 billion
Series B. Yeah, biggest in Europe. Yeah. And that money is specifically earmarked for building these AI factories. We're talking facilities with like 100 ,000 NVIDIA GPUs each. Backed by huge names, Nokia, Dell, NVIDIA itself. Yeah. They're building the infrastructure for that 125X future. But then there's the flip side, the operational cost. We saw that post from the developer, right? Built over 30 AI agents. And tracked the cost. And called it the brutal cost
truth. Basically, running complex agents in the real world gets really expensive, really fast. It's not just the upfront training cost. Absolutely. That massive scale translates directly to higher running costs for everyone using these models. And meanwhile. You see OpenAI launching its biggest ad push ever for ChatGPT. Streaming, billboards, influencers. They're trying to lock in that market
share now. Given this exponential compute growth, is the cost of running real -world agents actually sustainable for, say, the average developer or a small company? Initial data suggests complexity dramatically increases operational expenditures. Yeah, that's a tough reality check. Okay, let's pivot a bit. Away from the cost side. towards a big technical leap. The official paper just dropped on VO3. That's Google DeepMind's new video model. And people are calling this Google's
GPT -3 moment for vision. Which is a big claim. Why? What's the big deal? Well, it connects back to Sutton's point, actually. It looks like we're seeing a shift from just imitation to something more like reasoning, but in the visual domain. VO3 seems to be able to reason across a video scene over time, not just generate pretty frames one after another. Okay. And the key concept here is chain of frames. Sounds like chain of thought for LLMs. Exactly. It's the visual peasant.
Chain of thought helps LLMs break down text problems step by step. Chain of frames lets the video model think across time. It can anticipate what happens next, understand physics in a basic way. It's not just processing static images. That's fascinating. But is it really reasoning? Or is
it just a super sophisticated mimic? If it's seen millions of videos of Jenga blocks falling, is it... understanding physics or just generating the most likely visual sequence based on that data, how do we know it's not just a fancy deep fake? That's the million -dollar question, always. But the evidence suggests it's more than mimicry because of its zero -shot performance on diverse tasks it wasn't explicitly trained for. Like what? Well, it can solve complex mazes visually.
It can simulate physics pretty accurately, predicting how those Jenga blocks will fall. It can even restore blurry images or animate a scene from just a rough hand -drawn sketch. Things require some kind of internal model of the world. Right. They use that four -level framework to measure it. Perception, modeling, manipulation, and then reasoning at the top. Simulating physics definitely feels like it's up in the modeling or even reasoning category. Exactly. And the implication is huge.
One good prompt into a model like this might eventually replace dozens of specialized computer vision tools engineers use today. It could simplify workflows dramatically. So how drastically does this new chain of frames approach change the job of computer vision engineers? One prompt can replace dozens of specialized vision tools, simplifying workflows dramatically. OK, let's shift to AI out in the wild because this power.
It has immediate, messy consequences. We saw the political example need to be neutral here, the instance with Trump posting an AI -generated clip. Yeah, the one claiming med beds could cure anything, reportedly from Fox News, but AI -generated. He later deleted it. But it shows how fast this stuff can spread and how convincing it can look, even if it's totally fabricated. It's a huge challenge for, you know. figuring out what's
real online. The speed is incredible. Yeah. I mean, I still wrestle with prompt drift myself sometimes, just trying to get an AI to do what I want consistently. Vulnerable admission. So seeing these convincing deep fakes pop up, it is worrying. It really highlights the complexity we're dealing with. Beat. And the battles aren't just political. They're corporate, too. Oh, yeah. Elon Musk's XAI is now suing OpenAI. The claim. Yeah. Stealing trade secrets. It's getting litigious.
And Meta is reportedly talking to Google about potentially using their Gemini model. Seems like it. The big players are definitely maneuvering, making alliances, getting ready for the next phase. At the same time, platforms are just drowning in AI content. Spotify deleting 75 million AI generated tracks. 75 million. Just using their spam filters. It shows the sheer scale of automated content generation they're fighting. It's like a tidal wave. Wow. But it's not all problematic.
There are useful tools emerging too, right? Like that Kimi assistant from Moonshot AI. Right, backed by Alibaba. It apparently has an agent mode now. You give it a simple prompt and it can create complex things like a multi -page website draft or editable presentation slides. Stuff that takes real work. With the volume of AI -generated content exploding, both useful and not, can platforms realistically keep pace with the necessary filtering and moderation?
The 75 million deleted tracks suggest filtering is already a massive ongoing battle. Okay. We've definitely covered a lot today. Philosophy, physics simulation, billion dollar funding rounds, fake news. Let's just take a quick pause. When we come back, let's boil it down. What's the single biggest idea, the main takeaway about this architectural shift for you, the listener? Midroll sponsor
read. All right. So if we boil down everything we discussed, the core tension really is between two fundamentally different approaches to AI. You've got the passive imitation that powers today's big LLMs. And then you have this idea of active consequence driven learning. That's the goal behind stuff like Oak. And it seems to be what's making models like VO3 capable of visual reasoning. Passive imitation versus active learning. LLMs are incredible statistical parrots,
basically. But if something's right. Getting to true AGI means moving beyond just predicting the next word or pixel. It means building systems that can actually model consequences, understand cause and effect, maybe even have intrinsic motivation. So the key thing for you to take away today is probably this. We're not just seeing AI get bigger. We might be seeing the very architecture of AI begin to shift, that chain of frames concept
in VO3. It's a sign we're moving beyond just mimicking language towards models that can actually reason visually, simulate outcomes, understand things across time. Right. That shift from mimicry to modeling consequences, that feels like the really big story here. So maybe a final thought for you to chew on. If VO3 can simulate simple
physics today, like Jenga blocks falling. What does it mean when we start handing over critical real -world decisions to models that can simulate the future consequences of their own potential actions? Beat. That feels significant. Definitely something to think about. That distinction between passive safety, which we have now, and truly goal -driven intelligence. Yeah. That's the next frontier, potentially the next big challenge.
Well, thanks for joining us on this deep dive into AI, architecture, scale, and everything in between. We hope it gave you some things to think about. We'll catch you next time, OTRO Music.