Okay, we really have to kick off this deep dive talking about that sensational claim, the one that just blew up the AI headlines for a bit. Yeah, the deleted tweet. Exactly. From an open AI VP suggesting, well, something almost unbelievable. That GPT -5 had made real progress on multiple really tough, long unsolved math problems. The Urdu's list stuff. Which, you know. Solving even one of those. That's huge for a mathematician. Career defining. Absolutely. So it went viral
like crazy. But the reality check came fast. And it was public and honestly kind of brutal. Yeah. And the core lesson, the thing that rival CEOs and academics jumped on. was this key difference. Confusing just finding information retrieval with actual like new thinking, real reasoning. Precisely. Welcome to the deep dive. We've sifted through your latest pile of sources. It's a real mix this time. High stakes drama, some really practical advice, and one genuinely surprising
new technique. Yeah, we're going to distill all that for you. Our mission today, cut through the noise, the hype. We need to really unpack what these big AI models actually do versus what the labs, well. sometimes claim they can do. So first up, we'll dig into that GPT -5 math thing properly. Then we're shifting gears. We'll talk about some essential tools and workflows, stuff for the builders and professionals listening. And finally, there's this technique out of Stanford.
Simple, costs nothing, but it really changes how we prompt these models. Might even be the end of complex prompt engineering as we know it. So yeah, let's get into this source material. Okay, segment one, the GPT -5 controversy. The initial claim itself was, it was pretty well. Totally. The VP tweeted, GBT -5 solved 10 unsolved Erdos problems. 10. And showed progress on 11 others. And these Erdos problems, just to be clear, they're notoriously hard. Right, yeah.
They need genuine mathematical creativity. Formal proofs. They're not like, you know, multiple choice questions. Right. So then mathematician Thomas Bloom, who actually tracks this stuff officially, he looked into it. And he quickly confirmed, nope. The model found old solutions. Stuff already in the training data. Exactly. Not new, original proofs. Yeah. Just found stuff it had already seen. And the reaction from the big names. Pretty intense. Yeah. Jan LeCun from
Meta apparently used some sharp words. Said they got hoisted by their own petard. Yeah. Basically meaning their own hype backfired. Ouch. And Demis Hassabis from DeepMind. Called it embarrassing. Straight up. Wow. But what's really interesting is that even OpenAI's own Sebastian Bubik kind of acknowledged the core issue here. Which is? The huge difference intellectually between just retrieving an old paper from the training data, which DBT5 did well, and actually inventing a
new proof. By implied invention. But delivered retrieval. That's the crux of it. So, okay, it was one deleted tweet. Why does this really matter in the bigger picture? Because AI labs are always using these leaderboards, right? Yeah. GSMAK, the math benchmark, they boast about these scores constantly. We see those headlines all the time. New model tops the charts. Right. But those scores often just test, well, stored reasoning, pattern matching, stuff the model learned during training.
So it sees a new question, finds a similar solved one in its memory, and kind of copies the method. Pretty much. It's retrieval disguised as reasoning. To claim real discovery, you need that formal proof verifiable novelty. Which wasn't there in this case. Nope. Proving those quick test scores can be, frankly, pretty misleading sometimes. Okay, so if these standard reasoning tests have this flaw, this susceptibility to just retrieving data, how should we actually measure real AI
discovery going forward? What's a better way? We must prioritize provable novelty over mere retrieval. Simple as that. Right. Verifiable newness. Exactly. Now, moving from those big claims to something more practical. Segment two, essential tools for builders. Yeah, this is important because the sheer volume of new AI tools, new features, it's overwhelming. Totally. Causes real analysis paralysis for people trying to actually use this stuff professionally. You need
a system to cut through it. Our notes mentioned a specific builder's framework. What's the core idea there? The main thing is focusing on time to value. How quickly can this tool actually help you solve a real problem you have? Rather than just chasing the highest score on some benchmark. Precisely. Evaluating tech based on your needs, not just the market hype. It's a solid system. Okay. And for people using, say, ChatGPT every day, there's a feature that gets missed. Yeah,
the projects feature. Often overlooked, but it lets you create separate, dedicated contexts for different tasks or workflows. Ah, so it stops the chatbot forgetting what you were talking about 10 minutes ago. Exactly. It gives it much better memory within that specific project. Yeah. For anything complex, multi -step, that continuous memory is a huge productivity win. Yeah, that continuity is massive. Honestly, I still wrestle with prompt drift and context windows myself
sometimes. Oh, me too. It happens. Trying to keep a consistent style or persona across a bunch of outputs, it can be tricky. For sure. But then when you need those really high quality, reliable results, like agency level stuff, but without the agency price tag. The source is pointed towards Google AI Studio. Absolutely. AI Studio gives you much deeper controls than your basic chatbot interface. Like what kind of controls? Well, the material details five specific professional
methods. One example is demanding output in really structured formats like JSON. Okay. Or forcing it to generate, say, a detailed negotiation brief with specific risk parameters clearly defined. Gotcha. So it's about making the output reliable and usable for serious work. Exactly. for when critical decisions are involved, or you need that consistency for creative or analytical tasks.
Okay, so beyond just better memory, what would you say is the single biggest productivity game for a professional using these more specialized AI features, like in AI Studio? Gig controls allow for agency -quality work without high cost. Right, that control is key. Definitely. Okay, let's shift gears again. Segment three, the economics, the talent side. Because things are getting intense. Intense is one word for it. The talent wars are real. We're seeing reports of AI startups and
SF leasing actual luxury apartments. And offering $1 ,000 rent stipends just to lure top engineers away from the Googles and open AIs. Whoa. I mean, imagine scaling that kind of competition across the whole industry. Billions being thrown at talent. It just shows the insane value placed on people who can actually push the research forward, you know, make fundamental breakthroughs. And if you're someone listening who wants to
get into that hot market. Yeah. The sources actually shared a super practical guide, how to get a job at an AI company. And this wasn't just random advice, right? No. It came straight from Jure Leskovec, Stanford, professor, founder, and he's actively hiring right now. So real insider stuff. Very useful. What else is happening? Quick headlines. Okay, rapid fire. OpenAI hired a black hole physicist. Oh, interesting. Google AI Studio. They just combined all their features into one single UI.
Big win for usability. Nice. Europe is deploying AI for massive water projects in its driest areas. Resource management focus. Important stuff. And OpenAI is pitching ChatGPT login features to other companies now, letting them use it for their own user authentication. Okay, the job market's clearly on fire. But let's go back to that black hole physicist joining OpenAI. What does something like that signal about where AI
research might be heading? AI is moving beyond language models into core scientific discovery. Got it. Fundamental science. Yeah. Thinking bigger. All right. This next segment, this could genuinely be a game changer for pretty much everyone listening. Segment four. Yeah. This is about the innovation killer. And the surprisingly simple fix. Okay, what's the killer? It's called typicality bias. It's the reason AI often gives you the same kind
of bland, boring answers. Or that same poem about misty mornings every time you ask for something creative. Exactly. It avoids risk. It plays it safe. Why does it do that? What's the root cause? It comes down to the training, specifically RLHF reinforcement learning with human feedback. Okay. Basically, typicality bias means when human reviewers rate the AI's answers, they tend to prefer the safe, familiar, typical ones. Ah, so the humans themselves are kind of biased towards average?
In a way, yeah. And this trains the model. To suppress the randomness, the statistical outliers that you actually need for real creativity. Which leads to this mode collapse thing you mentioned. Right. Mode collapse is when the model just keeps defaulting to the statistical average. The most common, safest, blandest response. Kills innovation. So we've basically been prompting wrong, or at least inefficiently. Pretty much. Yeah. We've been fighting the model's tendency towards the
average. But Stanford found this fix. Zero cost. called verbalized sampling. And the fix is almost ridiculously simple. It really is. You just add one line to your prompt. Go on. Instead of just asking for, say, five jokes, you add a statistical frame. You ask it. Generate five jokes with their probabilities. That's it. Just adding with their probability. That's it. You're basically telling the model, hey, think about the diversity of your possible answers. Show me the spread. Wow.
Okay. And the results. The source material seemed pretty emphatic about this. Oh, the results are nuts, especially considering it costs nothing extra. Creative writing. They saw a 92 % jump in diversity for poems. 92%. And for jokes. Right. 109 % increase in diversity. Double the variety, basically. Yeah. And it wasn't just fluff. High -stakes stuff, too. Dialogue simulation. The AI's responses became twice as close to actual human donation behavior in a test scenario. So
more realistic, human -like variation. Exactly. An open -ended Q &A. Seven times better answer spread. And four times lower KL divergence. Right. Which is just a fancy way of saying the answers were much less statistically similar, much more varied. That's impressive. Does it work everywhere? Seems like it. Yeah. Works across different models, GPT, Claude, Gemini. And it scales with model size. GPT -4 got twice the diversity boost compared
to the smaller GPT -4 Mini. Just reframing the request statistically unlocks all this built -in variety. It's kind of mind -blowing. It really is. Okay, so does this super simple technique basically mean that the whole complex art of prompt engineering, you know, crafting these elaborate instructions, is that kind of over now? The shift is towards statistical framing for built -in creativity. Less hand -holding,
more guiding the statistics. Interesting. Okay, let's try and pull the main threads together from this deep dive. Two big takeaways for you as you navigate this constantly changing AI landscape. All right, takeaway number one. First, be skeptical of the hype. Just trust it. That GPT -5 retrieval mess, it shows the line between just accessing knowledge and real innovation is still super blurry. Demand proof, demand novelty. Exactly. Verifiable novelty is key. And takeaway number
two. Trust the simple, smart fixes. That verbalized sampling technique. It gives you a huge, zero -cost creativity and diversity boost just by changing how you ask. Yeah, it's about applying these simple, elegant, structural changes. Right. To get way better, less predictable results from the models you're probably already using every day. That's where the real efficiency gain is, isn't it? Getting more out of the tools you have. Totally. It's about smarter interaction, not
just bigger models. So the challenge for you listening right now is maybe to go try this today. Yeah, test it out. Think about your own work, your field. How could adding that simple phrase, generate X results with their probabilities, how could that unlock something new? Move beyond the same old answers the AI usually spits out. Right. How can it help you break out of that typicality trap, that mode collapse in your specific context? That's the path forward. Definitely
something to experiment with. And thanks again for sharing all these sources with us for the deep dive. Absolutely. We couldn't do it without you. We'll catch you next time.