If you don't change how you structure your work, AI just amplifies your sloppiness. Wow. I read that this morning and I had to put my coffee down. It felt less like tech advice and more like a personal attack. It's pretty aggressive, isn't it? But honestly, for 2026, it's the reality check I think a lot of us need. It is. Welcome back to the deep dive. It's Friday, January 23rd. Today we are. We're slowing things down. We're not talking about the newest apps. No new gadgets.
No. We're looking at the invisible layer, the psychology of how we actually talk to these new kinds of intelligence. We're deconstructing a piece called AI problem solving, systems for reliable intelligence. Which is, you know, a very dry title for what is basically a manual on how to stop messing around with these tools. Right. And it feels like the timing is perfect. For the past few years, it's been all about experimenting, just, you know, throwing stuff at the wall. The
let's see what happens phase. Exactly. But the whole idea here is that phase is over. It's time to build systems or you're just going to get buried in your own mess. That's the great divergence we're seeing. People who treat AI like a magic eight ball and people who treat it like an engineering problem. So here's the plan for today. First, we're going to dismantle that search box idea why that blinking cursor is actually a trap. Then we'll get into reliability, things like
grounding. And the I don't know rule, which is way harder than it sounds. Then there's the LLM council. Sounds very grand. Very sci -fi. It does. We'll talk orchestration versus agents, and then spend some real time on a term I love, vibe coding. My favorite. It's a real shift in how we think. And finally, we'll talk about the last human bottleneck, things like judgment and curation. So let's start at the beginning, the search box trap. OK. The source argues our biggest
problem is just. muscle memory. It's cognitive inertia. I mean, think about it. For what, 30 years, the entire internet era, we've been trained to do one thing. You have a question? You type it in a box, you hit enter, and Google or whatever finds a list of answers that already exist. It's a retrieval task. You're a librarian, and you're just fetching a book from a shelf. The answer is already out there. Precisely. But here's the fundamental breakdown. An LLM is not a librarian.
It's a poet. It's a poet. It's a generator. When you type in a prompt, it is not looking up an answer. It is calculating the most probable next word. So if I treat a generator like a retriever? You get hallucinations. You're asking a probability engine to act like a database. And because it's designed to be helpful, it will just confidently lie to you. It fills the gaps with noise that sounds plausible. And in a professional setting, legal medical, that's a huge liability. It's
just dangerous. If your input is vague, if you just kind of toss a question out there, the AI's output is going to be just as vague. It defaults to the average of the internet. And the average of the internet is? Mediocrity. At best. So the goal is to shift from just getting an output to actually solving a problem. It's about building a workflow where errors are caught by the structure itself. You have to assume the model is a little
bit drunk. And you have to build guardrails so that even if it wants to go off the rails, it can't. So let me pause on that. If that search box method is our default, what's the immediate consequence of using it for high stakes work? It creates fast but risky outputs. It gives you speed without reliability. Speed without reliability. That feels like the slogan for the last three years. So how do we engineer that reliability in? The source talks about grounding. We hear
that term a lot. Explain it to me like I'm a skeptic. OK. So an ungrounded AI is improvising. It's just making things up based on its vast, messy training data. Grounding is like handing the AI a script. You're telling it, ignore everything you think you know. Look only at these three PDFs I just uploaded. Answer my question using only this information. So it's the difference between an essay from memory versus an open book
test, where you have to cite your sources. It's even stricter, because with an open book test, you can still kind of fudge it. The magic trick with grounding is you have to add one more instruction. You have to explicitly tell the model. If the answer's not in this text, you must say, I don't know. That feels almost too simple. Does that really work? It changes everything. Because these models are people pleasers. They're optimized to give you an answer. They hate saying they
don't know. It feels like a failure to them. Exactly. So by giving them permission to fail, you eliminate all those forced hallucinations where they're trying to connect dots that just aren't there. So you're giving it permission to be useless. And that... paradoxically makes it incredibly useful. Because then when it does give you an answer, you know where it came from. And this connects to ARAG, right? Retrieval Augmented Generation. Right. That's just the plumbing for
grounding at scale. If you have, say, 10 ,000 company documents, you can't paste them all into the prompt. Of course not. ARAG is just the system that first searches your documents, finds the right page, and then hands just that one relevant page to the AI to ground its answer. So you move from an answer based on vibes to an answer based on evidence. Evidence is the word. You stop judging if the answer sounds right and you start checking
its citations. So let's nail this down. What's the one critical instruction that turns a guessing AI into a grounded one? Explicitly permitting the model to say, I don't know, eliminates forced hallucinations. Yeah, I don't know rule. I feel like I need to apply that to my own life. No, we all. OK, let's scale this up. One model, even a grounded one, can still make a mistake. The source brings up this idea from André G. Carpathy,
the LLM Council. That's the council, yes. It sounds like something out of Lord of the Rings. It really does. But it's actually just expensive and redundant computing. Walk me through it, because my first thought is, why would I need four different AIs to answer one question? Well, you wouldn't for a simple email. This is for high stakes work. Imagine you're analyzing a legal contract for a loophole. OK. If you just use one model, say GPT -5, you're a victim of
its specific blind spots. Every model has a different personality, a different way of reasoning. A different flavor. Right. Some are more creative, some are more literal. The council approach is you write one clear prompt and you send it to three or four different models all at the same time. And then you compare the answers. Exactly. If three say this is safe and one says this is a risk, you stop. That disagreement is your signal. It's a smoke alarm telling you there's nuance
you missed. And the source mentioned a meta step that I thought was brilliant, using another AI to judge the council's answers. The judge model. It turns out models are often better at critiquing than creating. It's easier to spot a flaw than have an original idea. So you just feed the four answers into a fifth model. And you say, rank these. Who's right? Who missed the point? It's like peer review at the speed of light. It's an insurance policy against a single point of
failure. OK, so the probing question is, why go through all that trouble and expense of consulting multiple brains for one problem? It exposes blind spots. If the models disagree, you know you need to verify. Find spots. Okay. So that's how we get reliable answers. Now let's talk about getting work done. The source draws a really clear line between two things, orchestration and agents. The train versus the taxi. This is the mental model everyone needs. Break it down. What's the
train? The train is orchestration. It's on rails. It goes from station A to station B to station C. It's rigid. It can't improvise. Not at all. It's brittle, but it's 100 % predictable. Orchestration is for when you map out a boring, repetitive task. Step one, summarize this text. Step two, extract the key dates. Step three, put them in a calendar invite. And if something changes in step one? The whole train derails. But as long as the track is clear, it works perfectly every
single time. OK, so that's the train. What's the taxi? The taxi is an agent. When you get into a taxi, you don't give it step by step directions. No, you give it a destination. Right. Get me to the airport. The driver, the agent, figures out the route. It adapts. If there's traffic on one street, it takes another. Agents are goal -oriented, not step -oriented. And you give them tools like access to a browser or your calendar. Exactly. You tell it schedule a meeting with
Sarah for next week. You don't tell it how to do that. It figures it out. See, that's the part that makes me and I think a lot of people a little nervous giving an AI that much agency. What if it decides the fastest way to the airport is through a park? That is the core alignment problem. and it's why real agents are only just now becoming practical. The risk was too high. The train is safe because you built the track yourself. The
taxi requires trust. So you use trains for boring, predictable tasks and agents for messy problems. For dynamic, messy problems. Like, do some research on this new company and find a good person to contact. You can't script that. Every website is different. You need an agent that can try something, fail, back up, and try a different approach. It's creating its own map as it goes. Yes. but you have to put it in a sandbox. You
have to give it clear boundaries. So what's the fundamental distinction here between an agent and just a standard automation workflow? Automations follow rigid steps. Agents follow a goal and adapt their path. Adaptability. Okay. I want to shift to the term that I admit made me roll my eyes at first. Vibe coding. Ah, don't do that. It's important. I'm trying not to. It just sounds like something you'd overhear at a skate park. But the source treats it as this huge Revolution.
It is a revolution. Just forget the slang for a second. Think of it as natural language programming. Go on. In the old days, meaning like 2023, if you wanted to build software, you were a translator. You had a human idea in your head and you had to painstakingly translate it into the machine's language. All the semicolons and brackets. Right. Syntax was the barrier. Yeah. Miss one comma, the whole thing breaks. Vibe coding is the shift where you stop being a translator and start being
an architect. You just describe the intent. So instead of set background color to hex code FFF. You say, make the landing page feel clean and minimalist. The AI handles the implementation. It translates clean into the right hex codes and CSS. And this matters because it completely changes who can build things. Totally. You don't need to know Python anymore. You need to know how to clearly describe a problem and what a
solution should feel like. So instead of being stuck with a messy spreadsheet, you can just... You can just say to the AI, build me a simple web page that lets me search and filter the rows in this spreadsheet. You describe the function, the vibe of the tool, and it generates the code. But that requires a totally different skill set, right? If it's not syntax, what is it? It's systems thinking. It's clarity of communication. If your description of the problem is sloppy, the tool
the AI builds will be sloppy. We're seeing this weird thing where people with, say, a background in literature are becoming great vibe coders because they're masters of describing things with precision. That's a wild thought. The entire skill stack is flipping upside down. It's the revenge of the liberal arts. I love that. So the probing question. What's the main skill for vibe coding if it isn't programming? The ability to clearly describe a problem and a desired functional
outcome. Describing the outcome. Sounds so simple, but it's incredibly hard. We're going to take a very quick break when we get back. If AI is handling all this, what's actually left for us to do? The good stuff. Stay with us, sponsor. And we are back on the deep dive. Before the break, we covered the search box, the LLM council, trains, taxis, and the rise of vibe coding. Now let's talk about the human in the loop. Where do we fit in all this? The source points to two
main roles, debugging and curation. Let's start with debugging. Well, it makes sense, right? If you're vibe coding an app or using an agent, things are going to go wrong, the AI is going to misunderstand you, the agent will get stuck. So we stop being writers and we become mechanics. We become diagnosticians. Yeah. And the source warns against what it calls the retry loop. You know, when you get a bad answer from the AI and you just hit regenerate over and over, hoping
for a better one. I do that all the time. We all do. It's the definition of insanity. The new skill is to look at the failure and ask why. Was my context bad? Did I assume the AI knew something it didn't? The source mentioned a technique called stress testing. Oh, I use this constantly. After you get an answer you like, you ask the AI a follow -up. Where is this output likely to break? Or what assumptions did you make to get here? You're asking it to audit its own work.
You're forcing it to reveal its own weak points. And that requires a human. to understand the context of the real world, which the AI just doesn't have. And then there's the second role, curation. This line really hit me. By 2026, creation is trivial. It's basically free. The cost to generate a thousand words or a piece of code or an image is trending to zero. And in economics, when supply is infinite, value collapses. So being able to write generic text isn't a valuable
skill anymore. No. The value moves up the stack. It moves to judgment. It moves to the role of the editor. I like the magazine editor analogy. It's perfect. It used to be that the hard part was getting the articles written. Now you can generate 1 ,000 articles in a second. The hard part, the valuable part, is knowing which one is true, which one is important, and which 999 to throw away. So you stop asking the AI to give me 10 ideas. And you start asking it to filter
these 50 ideas. You give it 100 pages and say, reduce this to the single most important paragraph. You use it to prune, not just to generate. The source calls this cognitive offloading. We offload the busy work, but never the final judgment. That is the bright red line. You can outsource labor. You can outsource summarization. You can not outsource the decision of what actually matters. The moment you do that, You're not using the tool anymore. The tool is using you. Wow. That's
a powerful distinction. So the final probing question. In a world of infinite AI generation, what becomes the most valuable human contribution? Curation. The ability to decide what is worth keeping and what to ignore. Deciding what to ignore. That might be the most important skill of the century. I really think it is. Let's bring this all together. We have covered a ton of ground today. We really have. We started by tearing down the search box idea. Then we built reliability
with grounding and the LLM council. We look at the actual systems. orchestration for the predictable stuff, agents for the messy stuff. And we talked about the big shift in our roles from being just users to being architects with vibe coding and editors through curation. The single theme connecting all of this seems to be. That's it. The people who are succeeding with AI are the ones who treat it like an engineering discipline. They aren't just chatting with it. They're designing systems
for it to operate within. It's the difference between just experimenting and actually operating. That's the whole game right there. So our challenge for you this week is to pick one boring, repetitive task you do. And don't just ask AI to do it. Try to orchestrate it. Write down the steps A, then B, then C. See if you can build a reliable track for it. And try the, I don't know, rule. Just add that one sentence to your instructions. See how much better and quieter your outputs
become. I want to leave you with one final thought from the source, a bit of a tough one to sit with. If AI exposes how you think and your AI outputs are messy, what does that say about your current thinking process? Oof. Yeah. That's the one that'll keep you up at night. Something to reflect on. Thanks for diving in with us. Always a pleasure. We'll see you next time.