Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more
New blackboard lecture with Reiner Pope: how do chips actually work - starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do. Reiner is CEO of MatX , a new chip startup (full disclosure - I’m an angel investor). He was previously at Google, where he worked on software efficiency , compilers, and TPU architecture. Watch this one on YouTube so you can see the chalkboard. Read the transcript . Sponsors * Crusoe was one of only five GP...
Eric Jang walks through how to build AlphaGo from scratch, but with modern AI tools. Sometimes you understand the future better by stepping backward. AlphaGo is still the cleanest worked example of the primitives of intelligence: search, learning from experience, and self-play. You have to go back to 2017 to get insight into how the more general AIs of the future might learn. Once he explained how AlphaGo works, it gave us the context to have a discussion about how RL works in LLMs and how it co...
David Reich is back. He and collaborator Ali Akbari just published a paper that overturns a long-standing consensus about human evolution — that natural selection has been dormant in our species since the agricultural revolution. By scaling ancient DNA sequencing and developing a new statistical method, they found that selection has actually sped up. Selection went especially bonkers during the Bronze Age (around 3,000 years ago). That’s when gene frequencies for everything from immune function ...
Did a very different format with Reiner Pope - a blackboard lecture where he walks through how frontier LLMs are trained and served. It’s shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk. It’s a bit technical, but I encourage you to hang in there – it’s really worth it. There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real de...
Jensen Huang discusses NVIDIA's significant influence on the AI industry's supply chain, emphasizing the company's commitment to ecosystem development rather than becoming a hyperscaler. He explains why NVIDIA's programmable architecture and CUDA ecosystem provide a lasting advantage over competitors like TPUs, fostering rapid algorithmic innovation. The conversation also delves into the complex debate around selling AI chips to China, where Huang argues for maintaining global market presence to advance the American tech stack and avoid unintended consequences.
Really enjoyed chatting with Michael Nielsen about how we recognize scientific progress. It's especially relevant for closing the RL verification loop for scientific discovery. But it's also a surprisingly mysterious and elusive question when you look at the history of human science. We approach this question stories like Einstein (who claimed that he hadn't even heard of the famous Michelson-Morley experiment, which is supposed to have motivated special relativity, until after he had come up wi...
We begin the episode with the absolutely ingenious and surprising way in which Kepler discovered the laws of planetary motion. People sometimes say that AI will make especially fast progress at scientific discovery because of tight verification loops. But the story of how we discovered the shape of our solar system shows how the verification loop for correct ideas can be decades (or even millennia) long. During this time, what we know today as the better theory can actually make worse prediction...
Dylan Patel, founder of SemiAnalysis, provides an in-depth look at the current and future constraints impacting AI compute scaling, focusing on logic, memory, and power. He discusses the contrasting strategies of AI labs like OpenAI and Anthropic in securing compute, the surprising appreciation of H100 GPU value, and the critical role of ASML in the long-term semiconductor supply. The episode also covers the impending memory crunch, the potential for China to catch up in chip production, and debunks the near-term feasibility of space GPUs.
Read the full essay here: https://www.dwarkesh.com/p/dow-anthropic Timestamps (00:00:00) - Anthropic vs The Pentagon (00:04:16) - The overhangs of tyranny (00:05:54) - AI structurally favors mass surveillance (00:08:25) - Alignment...to whom? (00:13:55) - Coordination not worth the costs Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe...
Renaissance history is so much wilder and weirder than you would have expected. Very fun chatting with Ada Palmer (historian, novelist, and composer based at the University of Chicago). Some especially fascinating things I learned from the conversation and her excellent book, Inventing the Renaissance : Not only did Gutenberg go bankrupt in the 1450s (after inventing the printing press), but so did the bank that foreclosed on him, and so did his apprentices. This is because paper was still very ...
Dario Amodei thinks we are just a few years away from AGI — or as he puts it, from having “a country of geniuses in a data center”. In this episode, we discuss what to make of the scaling hypothesis in the current RL regime, why task-specific RL might lead to generalization, and how AI will diffuse throughout the economy. We also dive into Anthropic’s revenue projections, compute commitments, path to profitability, and more. Watch on YouTube ; read the transcript . Sponsors * Labelbox can get yo...
In this episode, John and I got to do a real deep-dive with Elon. We discuss the economics of orbital data centers, the difficulties of scaling power on Earth, what it would take to manufacture humanoids at high-volume in America, xAI’s business and alignment plans, DOGE, and much more. Watch on YouTube ; read the transcript . Sponsors * Mercury just started offering personal banking! I’m already banking with Mercury for business purposes, so getting to bank with them for my personal life makes ...
Adam Marblestone is CEO of Convergent Research . He’s had a very interesting past life: he was a research scientist at Google Deepmind on their neuroscience team and has worked on everything from brain-computer interfaces to quantum computing to nanotech and even formal mathematics. In this episode, we discuss how the brain learns so much from so little, what the AI field can learn from neuroscience, and the answer to Ilya’s question: how does the genome encode abstract reward functions? Turns o...
Read the essay here . Timestamps 00:00:00 What are we scaling? 00:03:11 The value of human labor 00:05:04 Economic diffusion lag is cope00:06:34 Goal-post shifting is justified 00:08:23 RL scaling 00:09:18 Broadly deployed intelligence explosion Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe...
This is the final episode of the Sarah Paine lecture series, and it’s probably my favorite one. Sarah gives a “tour of the arguments” on what ultimately led to the Soviet Union’s collapse, diving into the role of the US, the Sino-Soviet border conflict, the oil bust, ethnic rebellions and even the Roman Catholic Church. As she points out, this is all particularly interesting as we find ourselves potentially at the beginning of another Cold War. As we wrap up this lecture series, I want to take a...
Ilya & I discuss SSI’s strategy, the problems with pre-training, how to improve the generalization of AI models, and how to ensure AGI goes well. Watch on YouTube ; read the transcript . Sponsors * Gemini 3 is the first model I’ve used that can find connections I haven’t anticipated. I recently wrote a blog post on RL’s information efficiency, and Gemini 3 helped me think it all through. It also generated the relevant charts and ran toy ML experiments for me with zero bugs. Try Gemini 3 toda...
As part of this interview, Satya Nadella gave Dylan Patel (founder of SemiAnalysis ) and me an exclusive first-look at their brand-new Fairwater 2 datacenter. Microsoft is building multiple Fairwaters, each of which has hundreds of thousands of GB200s & GB300s. Between all these interconnected buildings, they’ll have over 2 GW of total capacity. Just to give a frame of reference, even a single one of these Fairwater buildings is more powerful than any other AI datacenter that currently exist...
In this lecture, military historian Sarah Paine explains how Russia—and specifically Stalin—completely derailed China’s rise, slowing them down for over a century. This lecture was particularly interesting to me because, in my opinion, the Chinese Civil War is 1 of the top 3 most important events of the 20th century. And to understand why it transpired as it did, you need to understand Stalin’s role in the whole thing. Watch on YouTube ; read the transcript . Sponsors Mercury helps you run your ...
The Andrej Karpathy episode. During this interview, Andrej explains why reinforcement learning is terrible (but everything else is much worse), why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. It was a pleasure chatting with him. Watch on YouTube ; read the transcript . Sponsors * Labelbox helps you get data that is more detailed, more accurate, and higher signal than you could get by d...
Nick Lane has some pretty wild ideas about the evolution of life. He thinks early life was continuous with the spontaneous chemistry of undersea hydrothermal vents. Nick’s story may be wrong, but I find it remarkable that with just that starting point, you can explain so much about why life is the way that it is — the things you’re supposed to just take as givens in biology class: * Why are there two sexes? Why sex at all? * Why are bacteria so simple despite being around for 4 billion years? Wh...
I have a much better understanding of Sutton’s perspective now. I wanted to reflect on it a bit. (00:00:00) - The steelman (00:02:42) - TLDR of my current thoughts (00:03:22) - Imitation learning is continuous with and complementary to RL (00:08:26) - Continual learning (00:10:31) - Concluding thoughts Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe...
Richard Sutton is the father of reinforcement learning, winner of the 2024 Turing Award, and author of The Bitter Lesson. And he thinks LLMs are a dead end. After interviewing him, my steel man of Richard’s position is this: LLMs aren’t capable of learning on-the-job, so no matter how much we scale, we’ll need some new architecture to enable continual learning. And once we have it, we won’t need a special training phase — the agent will just learn on-the-fly, like all humans, and indeed, like al...
Sergey Levine , one of the world’s top robotics researchers and co-founder of Physical Intelligence , thinks we’re on the cusp of a “self-improvement flywheel” for general-purpose robots. His median estimate for when robots will be able to run households entirely autonomously? 2030. If Sergey’s right, the world 5 years from now will be an insanely different place than it is today. This conversation focuses on understanding how we get there: we dive into foundation models for robotics, and how we...
In this lecture, military historian Sarah Paine explains how Britain used sea control, peripheral campaigns, and alliances to defeat Nazi Germany during WWII. She then applies this framework to today, arguing that Russia and China are similarly constrained by their geography, making them vulnerable in any conflict with maritime powers (like the U.S. and its allies). Watch on YouTube ; listen on Apple Podcasts or Spotify . Sponsors * Labelbox partners with researchers to scope, generate, and deli...
Jacob Kimmel thinks he can find the transcription factors to reverse aging. We do a deep dive on why this might be plausible and why evolution hasn’t optimized for longevity. We also talk about why drug discovery has been getting exponentially harder, and what a new platform for biological understanding to speed up progress would look like. As a bonus, we get into the nitty gritty of gene delivery and Jacob’s controversial takes on CAR-T cells. For full disclosure, I am an angel investor in NewL...
How will we feed the 100s of GWs of extra energy demand that AI will create over the coming decade? On this episode, Casey Handmer (Caltech PhD, former NASA JPL, founder & CEO of Terraform Industries) walks me through how we can pull it off, and why he thinks a major part of this energy singularity will be powered by solar. His views are contrarian, but he came armed to defend them. Watch on YouTube ; listen on Apple Podcasts or Spotify . SPONSORS - Lighthouse helps frontier technology compa...
A deep dive with Lewis Bollard, who leads Open Philanthropy’s strategy for Farmed Animal Welfare , on the surprising economics of the meat industry. Why is factory farming so efficient? How can we make the lives of the 23+ billion animals living on factory farms more bearable? How far off are the moonshots (e.g., brainless chickens, cultivated meats, etc.) to end this mass suffering? And why does the meat industry have such a surprising amount of political influence? For decades, innovation in t...
After my last lecture series with Sarah Paine ended, I still had so many questions . I knew we’d only scratched the surface of Sarah’s scholarship, so I immediately invited her back for another series: she graciously agreed, and we’ll be releasing the results online over the coming weeks and months! This first lecture is focused on the balance of power in East Asia at the turn of the 20th century. Specifically, how did Japan (population 47M) defeat China (400M) and Russia (130M) to become Asia's...
The Stephen Kotkin episode. Kotkin is arguably the world’s foremost expert on Joseph Stalin and has written a massive 2-volume biography on him (with a 3rd volume in the works). No other individual had more of a profound impact on the 20th century than Stalin. He held the power of life and death over every single person across 11 time zones, and he killed tens of millions of people, utterly consumed by an ideology aimed at building paradise on Earth. And, he was one half of the biggest and most ...
I’ve had a lot of discussions on my podcast where we haggle out timelines to AGI. Some guests think it’s 20 years away - others 2 years . Here’s an audio version of where my thoughts stand as of June 2025. If you want to read the original post, you can check it out here . Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe...
