Just a year ago we released a two part episode titled An Overview of Technical AI Alignment with Rohin Shah. That conversation provided details on the views of central AI alignment research organizations and many of the ongoing research efforts for designing safe and aligned systems. Much has happened in the past twelve months, so we've invited Rohin — along with fellow researcher Buck Shlegeris — back for a follow-up conversation. Today's episode focuses especially on the state of current resea...
Apr 15, 2020•2 hr 21 min•Transcript available on Metacast The global spread of COVID-19 has put tremendous stress on humanity’s social, political, and economic systems. The breakdowns triggered by this sudden stress indicate areas where national and global systems are fragile, and where preventative and preparedness measures may be insufficient. The COVID-19 pandemic thus serves as an opportunity for reflecting on the strengths and weaknesses of human civilization and what we can do to help make humanity more resilient. The Future of Life Institute's E...
Apr 09, 2020•1 hr 27 min•Transcript available on Metacast Toby Ord’s “The Precipice: Existential Risk and the Future of Humanity" has emerged as a new cornerstone text in the field of existential risk. The book presents the foundations and recent developments of this budding field from an accessible vantage point, providing an overview suitable for newcomers. For those already familiar with existential risk, Toby brings new historical and academic context to the problem, along with central arguments for why existential risk matters, novel quantitative ...
Apr 01, 2020•1 hr 11 min•Transcript available on Metacast Lethal autonomous weapons represent the novel miniaturization and integration of modern AI and robotics technologies for military use. This emerging technology thus represents a potentially critical inflection point in the development of AI governance. Whether we allow AI to make the decision to take human life and where we draw lines around the acceptable and unacceptable uses of this technology will set precedents and grounds for future international AI collaboration and governance. Such regul...
Mar 16, 2020•1 hr 16 min•Transcript available on Metacast As with the agricultural and industrial revolutions before it, the intelligence revolution currently underway will unlock new degrees and kinds of abundance. Powerful forms of AI will likely generate never-before-seen levels of wealth, raising critical questions about its beneficiaries. Will this newfound wealth be used to provide for the common good, or will it become increasingly concentrated in the hands of the few who wield AI technologies? Cullen O'Keefe joins us on this episode of the FLI ...
Feb 28, 2020•1 hr 5 min•Transcript available on Metacast From Max Tegmark's Life 3.0 to Stuart Russell's Human Compatible and Nick Bostrom's Superintelligence, much has been written and said about the long-term risks of powerful AI systems. When considering concrete actions one can take to help mitigate these risks, governance and policy related solutions become an attractive area of consideration. But just what can anyone do in the present day policy sphere to help ensure that powerful AI systems remain beneficial and aligned with human values? Do to...
Feb 18, 2020•1 hr 11 min•Transcript available on Metacast Our perceptions of reality are based on the physics of interactions ranging from millimeters to miles in scale. But when it comes to the very small and the very massive, our intuitions often fail us. Given the extent to which modern physics challenges our understanding of the world around us, how wrong could we be about the fundamental nature of reality? And given our failure to anticipate the counterintuitive nature of the universe, how accurate are our intuitions about metaphysical and persona...
Jan 31, 2020•2 hr 45 min•Transcript available on Metacast In the 1984 book Reasons and Persons, philosopher Derek Parfit asks the reader to consider the following scenario: You step into a teleportation machine that scans your complete atomic structure, annihilates you, and then relays that atomic information to Mars at the speed of light. There, a similar machine recreates your exact atomic structure and composition using locally available resources. Have you just traveled, Parfit asks, or have you committed suicide? Would you step into this machine? ...
Jan 16, 2020•2 hr 3 min•Transcript available on Metacast Neither Yuval Noah Harari nor Max Tegmark need much in the way of introduction. Both are avant-garde thinkers at the forefront of 21st century discourse around science, technology, society and humanity’s future. This conversation represents a rare opportunity for two intellectual leaders to apply their combined expertise — in physics, artificial intelligence, history, philosophy and anthropology — to some of the most profound issues of our time. Max and Yuval bring their own macroscopic perspect...
Dec 31, 2019•1 hr 1 min•Transcript available on Metacast As 2019 is coming to an end and the opportunities of 2020 begin to emerge, it's a great time to reflect on the past year and our reasons for hope in the year to come. We spend much of our time on this podcast discussing risks that will possibly lead to the extinction or the permanent and drastic curtailing of the potential of Earth-originating intelligent life. While this is important and useful, much has been done at FLI and in the broader world to address these issues in service of the common ...
Dec 28, 2019•2 hr 39 min•Transcript available on Metacast Jan Leike is a senior research scientist who leads the agent alignment team at DeepMind. His is one of three teams within their technical AGI group; each team focuses on different aspects of ensuring advanced AI systems are aligned and beneficial. Jan's journey in the field of AI has taken him from a PhD on a theoretical reinforcement learning agent called AIXI to empirical AI safety research focused on recursive reward modeling. This conversation explores his movement from theoretical to empiri...
Dec 16, 2019•58 min•Transcript available on Metacast We could all be more altruistic and effective in our service of others, but what exactly is it that's stopping us? What are the biases and cognitive failures that prevent us from properly acting in service of existential risks, statistically large numbers of people, and long-term future considerations? How can we become more effective altruists? Stefan Schubert, a researcher at University of Oxford's Social Behaviour and Ethics Lab, explores questions like these at the intersection of moral psyc...
Dec 02, 2019•59 min•Transcript available on Metacast In this brief epilogue, Ariel reflects on what she's learned during the making of Not Cool, and the actions she'll be taking going forward.
Nov 27, 2019•5 min•Transcript available on Metacast It’s the Not Cool series finale, and by now we’ve heard from climate scientists, meteorologists, physicists, psychologists, epidemiologists and ecologists. We’ve gotten expert opinions on everything from mitigation and adaptation to security, policy and finance. Today, we’re tackling one final question: why should we trust them? Ariel is joined by Naomi Oreskes, Harvard professor and author of seven books, including the newly released "Why Trust Science?" Naomi lays out her case for why we shoul...
Nov 26, 2019•51 min•Transcript available on Metacast Most Americans believe in climate change — yet far too few are taking part in climate action. Many aren't even sure what effective climate action should look like. On Not Cool episode 25, Ariel is joined by Mario Molina, Executive Director of Protect our Winters, a non-profit aimed at increasing climate advocacy within the outdoor sports community. In this interview, Mario looks at climate activism more broadly: he explains where advocacy has fallen short, why it's important to hold corporations...
Nov 21, 2019•35 min•Transcript available on Metacast Defunding the fossil fuel industry is one of the biggest factors in addressing climate change and lowering carbon emissions. But with international financing and powerful lobbyists on their side, fossil fuel companies often seem out of public reach. On Not Cool episode 24, Ariel is joined by Ellen Quigley and Natalie Jones, who explain why that’s not the case, and what you can do — without too much effort — to stand up to them. Ellen and Natalie, both researchers at the University of Cambridge’s...
Nov 19, 2019•54 min•Transcript available on Metacast Wendell Wallach has been at the forefront of contemporary emerging technology issues for decades now. As an interdisciplinary thinker, he has engaged at the intersections of ethics, governance, AI, bioethics, robotics, and philosophy since the beginning formulations of what we now know as AI alignment were being codified. Wendell began with a broad interest in the ethics of emerging technology and has since become focused on machine ethics and AI governance. This conversation with Wendell explor...
Nov 15, 2019•1 hr 13 min•Transcript available on Metacast Though climate change and global warming are often used synonymously, there’s a different kind of climate change that also deserves attention: nuclear winter. A period of extreme global cooling that would likely follow a major nuclear exchange, nuclear winter is as of now — unlike global warming — still avoidable. But as Cold War era treaties break down and new nations gain nuclear capabilities, it's essential that we understand the potential climate impacts of nuclear war. On Not Cool Episode 2...
Nov 15, 2019•1 hr 3 min•Transcript available on Metacast Right before civil war broke out in 2011, Syria experienced a historic five-year drought. This particular drought, which exacerbated economic and political insecurity within the country, may or may not have been caused by climate change. But as climate change increases the frequency of such extreme events, it’s almost certain to inflame pre-existing tensions in other countries — and in some cases, to trigger armed conflict. On Not Cool episode 22, Ariel is joined by Cullen Hendrix, co-author of ...
Nov 13, 2019•36 min•Transcript available on Metacast The increase of CO2 in the atmosphere is doing more than just warming the planet and threatening the lives of many terrestrial species. A large percentage of that carbon is actually reabsorbed by the oceans, causing a phenomenon known as ocean acidification — that is, our carbon emissions are literally changing the chemistry of ocean water and threatening ocean ecosystems worldwide. On Not Cool episode 21, Ariel is joined by Libby Jewett, founding Director of the Ocean Acidification Program at t...
Nov 07, 2019•39 min•Transcript available on Metacast This summer, the world watched in near-universal horror as thousands of square miles of rainforest went up in flames. But what exactly makes forests so precious — and deforestation so costly? On the 20th episode of Not Cool, Ariel explores the many ways in which forests impact the global climate — and the profound price we pay when we destroy them. She’s joined by Deborah Lawrence, Environmental Science Professor at the University of Virginia whose research focuses on the ecological effects of t...
Nov 06, 2019•43 min•Transcript available on Metacast There exist many facts about the nature of reality which stand at odds with our commonly held intuitions and experiences of the world. Ultimately, there is a relativity of the simultaneity of events and there is no universal "now." Are these facts baked into our experience of the world? Or are our experiences and intuitions at odds with these facts? When we consider this, the origins of our mental models, and what modern physics and cosmology tell us about the nature of reality, we are beckoned ...
Oct 31, 2019•2 hr 31 min•Transcript available on Metacast Carbon emissions account for about 50% of warming, yet carbon overwhelmingly dominates the climate change discussion. On Episode 19 of Not Cool, Ariel is joined by Ilissa Ocko for a closer look at the non-carbon causes of climate change — like methane, sulphur dioxide, and an aerosol known as black carbon — that are driving the other 50% of warming. Ilissa is a senior climate scientist with the Environmental Defense Fund and an expert on short-lived climate pollutants. She explains how these non...
Oct 31, 2019•38 min•Transcript available on Metacast In many ways, the global carbon budget is like any other budget. There’s a maximum amount we can spend, and it must be allocated to various countries and various needs. But how do we determine how much carbon each country can emit? Can developing countries grow their economies without increasing their emissions? And if a large portion of China’s emissions come from products made for American and European consumption, who’s to blame for those emissions? On episode 18 of Not Cool, Ariel is joined ...
Oct 30, 2019•51 min•Transcript available on Metacast It’s time to get creative in the fight against climate change, and machine learning can help us do that. Not Cool episode 17 continues our discussion of “Tackling Climate Change with Machine Learning,” a nearly 100 page report co-authored by 22 researchers from some of the world’s top AI institutes. Today, Ariel talks to Natasha Jaques and Tegan Maharaj, the respective authors of the report’s “Tools for Individuals” and “Tools for Society” chapters. Natasha and Tegan explain how machine learning...
Oct 24, 2019•58 min•Transcript available on Metacast How can artificial intelligence, and specifically machine learning, be used to combat climate change? In an ambitious recent report, machine learning researchers provided a detailed overview of the ways that their work can be applied to both climate mitigation and adaptation efforts. The massive collaboration, titled “Tackling Climate Change with Machine Learning,” involved 22 authors from 16 of the world's top AI institutions. On Not Cool episodes 16 and 17, Ariel speaks directly to some of the...
Oct 22, 2019•1 hr 27 min•Transcript available on Metacast Despite the global scale of the climate crisis, its impacts will vary drastically at the local level. Not Cool Episode 15 looks at the unique struggles facing different communities — both human and non-human — and the importance of equity in climate adaptation. Ariel is joined by Astrid Caldas, a senior climate scientist at the Union of Concerned Scientists, to discuss the types of climate adaptation solutions we need and how we can implement them. She also talks about biodiversity loss, ecologi...
Oct 17, 2019•35 min•Transcript available on Metacast As the world nears the warming limit set forth by international agreement, carbon emissions have become a costly commodity. Not Cool episode 14 examines the rapidly expanding domain of carbon finance, along with the wider economic implications of the changing climate. Ariel is joined by Filippo Berardi, an environmental management and international development specialist at the Global Environment Facility (GEF). Filippo explains the international carbon market, the economic risks of not addressi...
Oct 15, 2019•41 min•Transcript available on Metacast What is ecosystem-based adaptation, and why should we be implementing it? The thirteenth episode of Not Cool explores how we can conserve, restore, and manage natural ecosystems in ways that also help us adapt to the impacts of climate change. Ariel is joined by Val Kapos, Head of the Climate Change and Biodiversity Programme at UN Environment’s World Conservation Monitoring Center, who explains the benefits of ecosystem-based adaptation along with some of the strategies for executing it. Val al...
Oct 10, 2019•32 min•Transcript available on Metacast Stuart Russell is one of AI's true pioneers and has been at the forefront of the field for decades. His expertise and forward thinking have culminated in his newest work, Human Compatible: Artificial Intelligence and the Problem of Control. The book is a cornerstone piece, alongside Superintelligence and Life 3.0, that articulates the civilization-scale problem we face of aligning machine intelligence with human goals and values. Not only is this a further articulation and development of the AI ...
Oct 08, 2019•1 hr 8 min•Transcript available on Metacast 
