You know, we usually assume more power is always better. It just like makes intuitive sense. Yeah. You buy the fastest processor, you get the smartest AI, you just throw that massive brain at every single problem you encounter. Right. It's our default setting. But what if that's entirely wrong? What if using our absolute smartest tools, you know, less? is actually the secret to better results. It sounds completely counterintuitive at first. I mean, we're totally conditioned to
want the absolute best all the time. Exactly. Welcome to the Deep Dive. Today, we're exploring Anthropics Advisor Strategy. Glad to be here. We're going to deconstruct why defaulting to the smartest AI model for every single step is a mistake. And we're going to look at the exact cost math, too. The token economics here are honestly fascinating. Yeah, they really are. We'll also untangle the confusion between the Cloud Messages API and Cloud Code. Right, because
there's a huge shift happening right now. Oh, massive. Right. We're moving from picking the smartest model to designing the smartest tasks. It completely changes how you build systems. So let's unpack the core philosophy here. Anthropic treats intelligence as a managed resource. And I have to admit something right here. I still wrestle with burning budget on simple tasks myself. Oh, you're definitely not alone in that trap.
It's just so incredibly tempting to pick the strongest model in the dropdown and just hit go. Yeah. Most people build their AI workflows exactly that way. They grab the most capable. model available, then they just wire it into every single node of their system. They essentially treat maximum intelligence as like an insurance policy against failure. But using an insurance policy for everyday driving is incredibly inefficient. Exactly. It wastes an enormous amount of money.
It burns through your API rate limits instantly. And fundamentally, well, it solves the wrong problem. It ignores how workflows actually function. Break down what you mean by solving the wrong problem. Think about a compound agentic workflow. Most tasks aren't just one monolithic thought. Right, they have pieces. Yeah, they're a sequence of mixed steps. Maybe one step requires intense,
multi -step logical reasoning. But the next four steps, they might just involve basic tool use or, you know, fetching a specific document from a database. Or just writing a three -sentence summary of a text file. Exactly. So the advisor strategy is basically a structural fix for that inefficiency. Let me define that jargon for you real quick. A cheaper AI does basic work, calling smarter AI for art parts. That is the perfect distillation of it. Yeah. Inside the system,
you pair two models together. Okay. A fast, highly efficient model acts as the frontline worker. It handles all the routing, the searching, the basic parsing. Right. A much stronger, more expensive model sits in the background as an advisor. The frontline model only calls the advisor when it hits a wall. It makes me think of a high -end software engineering team. Imagine hiring a top -tier, world -renowned senior systems architect.
You pay them an absolute fortune, but then you force them to sit there and fix basic markdown typos in your documentation all day? Right, and they'd probably quit out of sheer boredom. Exactly. You'd let a junior developer handle the typos. You only tap the senior architect when the server suddenly crashes. Or when the database architecture needs a fundamental redesign. Yeah, exactly. The analogy tracks perfectly. AI models don't have egos, obviously, but the economic principle
is identical. Right. You simply don't need top -level intelligence from start to finish. If 90 % of a task stays on the cheaper default path, your whole system runs lighter. It runs drastically faster. And obviously it costs a fraction of the price. Which brings up a crucial operational question for you. How often do these mixed workflows actually need to escalate to the smarter model? Honestly, it's far less often than you'd intuitively
expect. A massive chunk of daily computational work is just moving data around, which rarely requires deep reasoning. So basic tasks rarely escalate, keeping the whole system incredibly efficient. Yeah. Once you genuinely internalize intelligence as a managed resource, you stop asking which single AI should do everything. You start asking how to divide the labor. Yeah. Philosophically, putting a junior developer on typo duty makes perfect sense. But AI isn't human.
Right. Does this theory actually survive contact with real world math? What happens when Anthropic forces these models to team up in a lab? That's where the benchmark data becomes incredibly relevant. Yeah. Anthropic ran this through S2BE Bench. And that's a notoriously difficult software engineering benchmark, right? Yeah, it is. Let's clarify what SWE Bench actually tests. It's not just answering trivia. No, it tests complex, agentic tasks. The AI is given a real -world GitHub issue
and a massive code base. Okay. It has to navigate the files, find the bug. write the patch, and ensure it passes tests. So it's heavily reliant on multi -step execution. Highly reliant. For this test, they used Clod 3 .5 Sonnet as the frontline model, but they gave it Clod 3 Opus as an advisor. Opus is their most capable, deepest reasoning model. What were the actual results
of pairing them up? The Sonnet and Opus combination improved the overall score by 2 .7 percentage points compared to Sonnet working entirely alone. Okay, so the output quality went up. But what happened to the actual cost of running that test? That's the fascinating part. Adding the smarter, more expensive model actually cut the cost per task by almost 12%. Wait, really? Adding an expensive model lowered the total bill. Yes, because Sonnet was doing all the tedious, time -consuming file
searching. It only invoked Opus for the final critical code generation. Wow. And they ran another test that's even more dramatic. They used BrowseComp, a web browsing benchmark. And they used Haiku for that one, right? Their absolute fastest, cheapest model. Exactly. Haiku alone scored a 19 .7 % success rate. Okay. But when they gave Haiku access to Opus as an advisor, the score shot up to 41 .2%. It more than doubled the performance.
It doubled the performance. And crucially, it still costs significantly less than if they had just forced Opus to do the entire browsing task from scratch. B, I want to dive into the specific token economics here because this explains the why behind those savings. Yeah, let's do it. Let's look at the actual pricing. Opus is $5 per million input tokens. But it's $25 per million output tokens. Right. And Sonnet is $3 for input, $15 for output. Haiku is just $1 for input and
$5 for output. The pattern is glaring. Output tokens cost five times as much as input tokens across the board. Yeah, they do. Why is generating text so much more expensive than reading it? It comes down to how the underlying transformer architecture actually processes data. Okay. When you feed a model a massive document, Kodo, The EPUs can process those tokens in parallel. Right. They crunch the whole block of text simultaneously. It's highly efficient. But generation is different.
Generation is autoressive. It happens sequentially. The model has to calculate the probability of the first word. Then it has to read that first word to calculate the second word. It's a continuous computationally heavy loop. So if you force an expensive model like Opus to output long strings of basic text, you're just setting money on fire. Precisely. You're paying a premium compute penalty for incredibly trivial generation. I need to push back on this a bit, though. Sure. Benchmarks
are tightly controlled laboratory tests. The prompts are clean. The environments are static. True. How does this actually hold up in messy production environments with real unpredictable users? That's a very fair critique. Production environments are inherently chaotic. Yeah. Users will submit bizarre edge cases that benchmarks simply don't account for. The savings aren't an automatic guarantee. So does saving money this way mean we're sacrificing overall output
quality? Not if your escalation logic is tight. You are forcing a weak model to guess. You're just stopping an expensive model from overworking. No, you just stop paying a premium for work that requires zero deep reasoning. Exactly. You protect your budget for the moments that actually require maximum intelligence. The economic value is clearly proven, but we really need to clarify where the strategy actually lives. Yeah, we do. Anthropix toolset is expanding rapidly, and it's causing
a lot of friction and confusion for users. The naming conventions definitely don't help. People conflate these distinct tools constantly. Let's define the core infrastructure first. What is the Cloud Messages API? The raw building blocks developers use to create custom AI apps. That's the perfect way to frame it. The advisor strategy is a design pattern that lives inside the Cloud Messages API. Okay. It is strictly for developers.
It's for engineering teams writing their own code to dictate exactly how an AI system behaves. But then you have Cloud Code. People hear about this advisor strategy and assume it's just baked into everything Anthropic releases. That's the biggest misconception right now. Cloud Code is a completely different beast. Right. It's a finished product. It's a standalone. terminal coding assistant that you install and use. It does not automatically use this intricate advisor routing setup under
the hood. Wait, so this intelligent routing isn't just a toggle switch in the settings? Not at all. That routing is a highly specific capability. It has to be manually built and tuned by developers using the Messages API. You have to write the code that tells the models how to talk to each other. Let's try an analogy. Think of the Messages API. like buying raw engine components. You're buying the pistons, the crankshaft, and the spark plugs to build your own custom race car from
scratch. That's the API. But Cloud Code is like walking onto a dealership lot and buying a finished sedan. You just turn the key and drive. You can't suddenly swap the engine out while you're on the highway. That perfectly captures the dynamic. The Messages API gives you immense granular control over model interaction. Cloud Code prioritizes... instant frictionless usability for a single user. There are other layers to this too, right? Like the agent SDK. Yeah, the agent SDK and managed
agents are another layer of abstraction. They're designed to help teams package agentic behavior into their software more easily. Gotcha. But again, if you want to orchestrate this highly specific multi -model advisor routing, you are working directly in the API. Let me make sure this is crystal clear for you. If I am just using cloud code, Do I have this routing built in automatically? You don't. You're simply relying on whichever single model you selected for that specific coding
session. No, this exact automated routing strategy requires building it yourself in the API. That's right. You have to intentionally design the handoff logic yourself. And building that custom logic takes incredible focus and reliable infrastructure. We're going to take a quick pause here for our sponsor. We'll be right back to talk about how you actually orchestrate this. We are back. So assuming we have our infrastructure sorted, let's explore the actual skill required to make this
API routing work. It all comes down to orchestration and rigorous testing. This is where the engineering gets genuinely fascinating. Routing the simple, obvious tasks is an easy win. Sure. The real challenge is managing the transition point on the harder tasks. Why is the transition so difficult to manage? Because the core challenge isn't just solving a complex task. The system's primary hurdle is the frontline model actually noticing
that the task is too hard. Yeah. It has to possess the self -awareness to recognize it needs the advisor. Whoa, imagine the system organically realizing it is out of its depth and calling for help. It's an incredible mechanism. You're essentially prompting a model. to evaluate its own limitations in real time. That's wild. It has to pause mid -workflow and say, my confidence score on this is dropping. I cannot execute this
reliably alone. That means the escalation logic is literally a component of the system's overall intelligence. Precisely. You have to give the cheaper model a specific tool, like an escalate to expert function. Right. But a weaker model might get lucky and guess the right answer without using the tool. Or worse, it might fail to notice the complexity entirely. I have to ask about a very specific risk here. Is there a danger of an AI version of the Dunning -Kruger effect?
Expand on that. How do you mean? Well, the Dunning -Kruger effect in humans is when someone incompetent vastly overestimates their own ability. Right. Isn't a cheaper, less capable model. inherently worse at judging its own competence, that feels like a paradox. Yeah. What if it confidently hallucinates a totally wrong answer instead of escalating? That paradox is the exact danger you have to engineer against. A cheaper model doesn't inherently know its own blind spots.
So you can't just deploy this. No, you can't just deploy the strategy blindly and assume it works. You have to explicitly teach it how to doubt itself. Yes. You might write a meta prompt saying, if you cannot extract the exact variable in three attempts, or if the request involves multivariable calculus, you must use the escalate tool. You have to test those boundaries rigorously. Because if it fails quietly and confidently,
the user experience is destroyed. If the haiku model misses critical context but confidently proceeds, the cost savings are entirely useless. If it escalates too late, You've already burned tokens on a doomed path. You're no longer just evaluating the AI's final text output. You are evaluating the system's internal judgment. So what is the main point of failure when designing this system? It's almost exclusively the routing logic. The cheaper model simply fails to recognize
the edge of its own capabilities. Weak routing logic failing to escalate when the task suddenly becomes truly complex. It's the biggest trap developers fall into. They see the 12 % cost savings on a spreadsheet and rush the implementation without tuning the thresholds. Two sec silence. Let's scale this back from the developer level for a moment. Let's talk about how you, the listener, can manually apply this mindset today. How can we integrate this philosophy into our everyday
personal workflows? The underlying logic absolutely applies to the everyday user, even if you're just typing prompts into the standard web interface. Right. Your deliberate choice of model dictates your daily budget and your time efficiency. So how should we structure our own daily projects? Let's say I'm writing a massive research report. You should use Opus for the initial planning step. Then let Sonnet handle the heavy execution. Plan with Opus, execute with Sonnet. That's a
remarkably clean framework. It's simple. But it fundamentally shifts how you work. Planning and execution require entirely different cognitive muscles from the AI. Explain the difference in those cognitive muscles. Planning requires high leverage holistic judgment. When you're outlining a research report. You need the AI to spot hidden logical gaps. You need it to synthesize disparate ideas into a cohesive structural vision. You want it to deeply understand the root problem.
Yeah. That deep architectural synthesis is exactly where Opus shines. It's the architect. Opus is the architect drawing the intricate blueprints for the skyscraper. Yes. But once those blueprints are locked in, the nature of the work changes completely. How so? The execution phase, actually drafting the sections, is much longer. It's highly repetitive. It's essentially just following instructions. So Sonnet is the highly capable builder hammering the nails according to the blueprint. Exactly.
Sonnet is brilliant at following a well -defined, strict plan. It writes well, it's fast, and it doesn't need to invent the core structure. And remember, you also have Haiku in your toolkit. Right. Where does Haiku fit into that research report workflow? Haiku is brilliant for the administrative microtasks. You need to quickly reformat a list of citations. Use haiku. Okay. You need to summarize a 50 -page PDF to see if it's worth reading. Use haiku. Exploring a bunch of random tangential
ideas very quickly. Haiku is perfect for that. You really are acting as the orchestrator of your own manual workflow. You are. You're internalizing the advisor strategy and applying it to your own digital life. But does the execution phase really require less intelligence than the planning phase? Almost universally, yes. Following a beautifully detailed map requires significantly less genius than charting the unknown territory to draw the
map in the first place. Yes. Once the exact path is clear, steady execution is usually perfectly fine. That's why you place your strongest, most expensive model... at the point of maximum leverage. Let the cheaper, faster models handle the steady, repetitive execution. Beat, we've covered a massive amount of technical and philosophical ground today. Let's bring it all together for you. We need to recap the central big idea here. We are sitting in the middle of a massive paradigm shift
in how computing is structured. AI system design is moving completely away from brute force model selection. Two years ago, developers just asked, What is the smartest model I can possibly afford? That question is officially obsolete. Now the entire industry is pivoting to task design. The new foundational question is, what is the most elegant way to structure this specific task? System orchestration is quickly becoming the single most valuable skill in software engineering.
Yeah. You have to know how to aggressively deconstruct work into micro components. You have to know exactly which model fits which specific fragmented piece. It's not about throwing maximum compute at every minor inconvenience anymore. It's about precision. It's about systemic elegance. The companies that win the next decade won't just be the ones paying for the most expensive API call. They'll be the ones who engineer systems
that know exactly when not to use them. This leaves us with a really fascinating thought to end on. I want you to actively think about this as you go about your work today. Look closely at your own daily workflow. Look at how you deploy your own human brain. How do you mean? Exactly. How much of your actual workday requires your personal opus level deep reasoning? Think about the specific tasks that demand your absolute best unfiltered cognitive effort. The deep strategy.
The complex problem solving. And then look at everything else. How much of your day is just haiku level mindless administrative execution? Answering basic emails. Formatting spreadsheets. Exactly. What if you started fiercely preserving your own mental bandwidth? That is a phenomenal way to frame personal productivity. What if you actively managed your human energy the exact same way we are teaching these APIs to manage their compute cycles? Save your deep reasoning
for the blueprints. Keep questioning. Keep exploring. Catch you next time on the Deep Dive.