Artificial Intelligence has too much hype. In this podcast, linguist Emily M. Bender and sociologist Alex Hanna break down the AI hype, separate fact from fiction, and science from bloviation. They're joined by special guests and talk about everything, from machine consciousness to science fiction, to political economy to art made by machines.
Don't just listen to podcasts. Learn from them with transcripts, summaries, and chapters for every episode. Skim, search, and bookmark insights. Learn more
Drs. Emma Strubell and Sasha Luccioni join Emily and Alex for an environment-focused hour of AI hype. How much carbon does a single use of ChatGPT emit? What about the water or energy consumption of manufacturing the graphics processing units that train various large language models? Why even catastrophic estimates from well-meaning researchers may not tell the full story. This episode was recorded on November 6, 2023. References: "The Carbon Footprint of Machine Learning Training Will Plateau, ...
Emily and Alex read through Google vice president Blaise Aguera y Arcas' recent proclamation that "artificial general intelligence is already here." Why this claim is a maze of hype and moving goalposts. References: Noema Magazine: "Artificial General Intelligence Is Already Here." "AI and the Everything in the Whole Wide World Benchmark" "Targeting the Benchmark: On Methodology and Current Natural Language Processing Research" "Recoding Gender: Women's Changing Participation in Computing" "The ...
Emily and Alex are joined by Stanford PhD student Haley Lepp to examine the increasing hype around LLMs in education spaces - whether they're pitched as ways to reduce teacher workloads, increase accessibility, or simply "democratize learning and knowing" in the Global South. Plus a double dose of devaluating educator expertise and fatalism about the 'inevitability' of LLMs in the classroom. Haley Lepp is a Ph.D. student in the Stanford University Graduate School of Education. She draws on criti...
Alex and Emily are taking another stab at Google and other companies' aspirations to be part of the healthcare system - this time with the expertise of Stanford incoming assistant professor of dermatology and biomedical data science Roxana Daneshjou . A look at the gap between medical licensing examination questions and real life, and the inherently two-tiered system that might emerge if LLMs are brought into the diagnostic process. References: Google blog post describing Med-PaLM Nature: Large ...
Emily and Alex tackle the White House hype about the 'voluntary commitments' of companies to limit the harms of their large language models: but only some large language models, and only some, over-hyped kinds of harms. Plus a full portion of Fresh Hell...and a little bit of good news. References: White House press release on voluntary commitments Emily’s blog post critiquing the “voluntary commitments” An “AI safety” infused take on regulation AI Causes Real Harm. Let’s Focus on That over the E...
Emily and Alex are joined by technology scholar Dr. Lucy Suchman to scrutinize a new book from Henry Kissinger and coauthors Eric Schmidt and Daniel Huttenlocher that declares a new 'Age of AI,' with abundant hype about the capacity of large language models for warmaking. Plus close scrutiny of Palantir's debut of an artificial intelligence platform for combat, and why the company is promising more than the mathy-maths can provide. Dr. Lucy Suchman is a professor emerita of sociology at Lancaste...
Emily and Alex talk to UC Berkeley scholar Hannah Zeavin about the case of the National Eating Disorders Association helpline, which tried to replace human volunteers with a chatbot --and why the datafication and automation of mental health services are an injustice that will disproportionately affect the already vulnerable. Content note: This is a conversation that touches on mental health, people in crisis, and exploitation. This episode was originally recorded on June 8, 2023. Watch the video...
Take a deep breath and join Alex and Emily in AI Hell itself, as they take down a month's worth of hype in a mere 60 minutes. This episode aired on Friday, May 5, 2023. Watch the video of this episode on PeerTube . References: Terrifying NEJM article on GPT-4 in medicine “Healthcare professionals preferred ChatGPT 79% of the time” Good thoughts from various experts in response ChatGPT supposedly reading dental x-rays Chatbots “need” therapists CEO proposes AI therapist, removes proposal upon rea...
After a hype-y few weeks of AI happenings, Alex and Emily shovel the BS on GPT-4’s “system card,” its alleged “sparks of Artificial General Intelligence,” and a criti-hype heavy "AI pause" letter. Hint: for a good time, check the citations. This episode originally aired on Friday, April 7, 2023. You can also watch the video of this episode on PeerTube . References: GPT-4 system card: https://cdn.openai.com/papers/gpt-4-system-card.pdf “Sparks of AGI” hype: https://twitter.com/SebastienBubeck/sta...
Alex and Emily are taking AI to court! Amid big claims about LLMs, a look at the facts about ChatGPT, legal expertise, and what the bar exam actually tells you about someone's ability to practice law--with help from Harvard legal and technology scholar Kendra Albert . This episode was first recorded on March 3, 2023. Watch the video of this episode on PeerTube . References: Social Science Research Network paper “written” by ChatGPT Joe Wanzala, “ChatGPT is ideal for eDiscovery” Legal application...
Should the mathy-maths be telling doctors what might be wrong with you? And can they actually help train medical professionals to treat human patients? Alex and Emily discuss the not-so-real medical and healthcare applications of ChatGPT and other large language models. Plus another round of fresh AI hell, featuring "charisma as a service," and other assorted reasons to tear your hair out. This episode was first recorded on February 17th of 2023. Watch the video of this episode on PeerTube . Ref...
New year, new hype? As the world gets swept up in the fervor over ChatGPT of late 2022, Emily and Alex give a deep sigh and begin to unpack the wave of fresh enthusiasm over large language models and the "chat" format specifically. Plus, more fresh AI hell. This episode was recorded on January 20, 2023. Watch the video of this episode on PeerTube . References: Situating Search (Shah & Bender 2022) Related op-ed: https://iai.tv/articles/all-knowing-machines-are-a-fantasy-auid-2334 Piantadosi'...
Emily and Alex are joined by Dr. Jeremy G. Kahn to discuss the distressingly large number of evaluation metrics for artificial intelligence, and some new AI hell. Jeremy G. Kahn has a PhD in computational linguistics, with a focus on information-theoretic and empirical engineering approaches to dealing with natural language (in text and speech). He’s gregarious, polyglot, a semi-auto-didact, and occasionally prolix. He also likes comic books, coffee, progressive politics, information theory, lat...
Emily and Alex discuss MetaAI's bullshit science paper generator, Galactica, along with its defenders. Plus, where could AI actually help scientific research? And more Fresh AI Hell. Watch the video of this episode on PeerTube . References: Imre Lakatos on research programs Shah, Chirag and Emily M. Bender. 2022. Situating Search. Proceedings of the 2022 ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR ’22). UW RAISE (Responsibility in AI Systems and Experiences) Stocha...
Emily and Alex discuss Sam Bankman-Fried's Future Fund, its essay contest, and the problems with using AI for prediction and resource allocation--mere days before the collapse of FTX. Also, we introduce our "What in the Fresh AI Hell?" segment! This episode was recorded on November 9, 2022. Watch the video of this episode on PeerTube. References: Super Forecasting And AI Bill Howe - Applied AI in High-Expertise Settings or Curation as Programming Samir Passi and Solon Barocos - "Problem Formulat...
This episode delves into the complex debate surrounding AI-generated art, questioning whether algorithmically created images truly constitute "art." The hosts and guests discuss the lack of human intent and experience in AI art, its "asocial" nature outside established creative communities, and the economic motivations behind the widespread hype. The conversation also explores the anthropomorphization of AI robots and its implications for artist attribution, copyright, and the broader art market.
Technology researchers Emily M. Bender and Alex Hanna kick off the Mystery AI Hype Theater 3000 series by reading through, " Can machines learn how to behave ?" by Blaise Aguera y Arcas, a Google VP who works on artificial intelligence. This episode was recorded in September of 2022, and is the last of three about Aguera y Arcas' post. You can watch the video of this episode on PeerTube . Check out future streams on Twitch . Meanwhile, send us any AI Hell you see . Find our book The AI Con here ...
Technology researchers Emily M. Bender and Alex Hanna kick off the Mystery AI Hype Theater 3000 series by reading through, " Can machines learn how to behave ?" by Blaise Aguera y Arcas, a Google VP who works on artificial intelligence. This episode was recorded in September of 2022, and is the second of three about Aguera y Arcas' post. You can also watch the video of this episode on PeerTube . Check out future streams on Twitch . Meanwhile, send us any AI Hell you see . Find our book The AI Co...
Emily M. Bender and Alex Hanna launch "Mystery AI Hype Theater 3000" by meticulously critiquing a Google VP's article, "Can Machines Learn To Behave?" They debunk claims regarding AI "value alignment," the notion of machines "understanding concepts" or "states of mind," and the misattribution of moral accountability to AI rather than human creators. The hosts argue against the hype surrounding general-purpose AI and emphasize real-world ethical concerns over speculative existential risks.
