Rationality is Systematized Winning, so rationalists should win. We’ve tried saving the world from AI, but that's really hard and we’ve had … mixed results. So let's start with something that rationalists should find pretty easy: Becoming Cool! I don’t mean, just, like, riding a motorcycle and breaking hearts level of cool. I mean like the first kid in school to get a Tamagotchi, their dad runs the ice cream truck and gives you free ice cream and, sure, they ride a motorcycle. I mean that kind o...
Apr 01, 2024•23 min
TL;DR Tacit knowledge is extremely valuable. Unfortunately, developing tacit knowledge is usually bottlenecked by apprentice-master relationships. Tacit Knowledge Videos could widen this bottleneck. This post is a Schelling point for aggregating these videos—aiming to be The Best Textbooks on Every Subject for Tacit Knowledge Videos. Scroll down to the list if that's what you're here for. Post videos that highlight tacit knowledge in the comments and I’ll add them to the post. Experts in the vid...
Apr 01, 2024•15 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/sJPbmm8Gd34vGYrKd/deep-atheism-and-ai-risk Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓...
Mar 20, 2024•47 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/h99tRkpQGxwtb9Dpv/my-clients-the-liars Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Mar 20, 2024•14 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/2sLwt2cSAag74nsdN/speaking-to-congressional-staffers-about-ai-risk Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Mar 10, 2024•24 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/Jash4Gbi2wpThzZ4k/cfar-takeaways-andrew-critch Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Mar 10, 2024•9 min
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.The following is a lightly edited version of a memo I wrote for a retreat. It was inspired by a draft of Counting arguments provide no evidence for AI doom, although the earlier draft contained some additional content. I personally really like the earlier content, and think that my post covers important points not made by the published version of that post. Thankful for the dozens of interesting conversations a...
Mar 09, 2024•20 min
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.TLDR: I’ve collected some tips for research that I’ve given to other people and/or used myself, which have sped things up and helped put people in the right general mindset for empirical AI alignment research. Some of these are opinionated takes, also around what has helped me. Researchers can be successful in different ways, but I still stand by the tips here as a reasonable default. What success generally loo...
Mar 07, 2024•40 min
Timaeus was announced in late October 2023, with the mission of making fundamental breakthroughs in technical AI alignment using deep ideas from mathematics and the sciences. This is our first progress update. In service of the mission, our first priority has been to support and contribute to ongoing work in Singular Learning Theory (SLT) and developmental interpretability, with the aim of laying theoretical and empirical foundations for a science of deep learning and neural network interpretabi...
Feb 29, 2024•12 min
This is a linkpost for https://bayesshammai.substack.com/p/contra-ngo-et-al-every-every-bayWith thanks to Scott Alexander for the inspiration, Jeffrey Ladish, Philip Parker, Avital Morris, and Drake Thomas for masterful cohosting, and Richard Ngo for his investigative journalism. Last summer, I threw an Every Bay Area House Party themed party. I don’t live in the Bay, but I was there for a construction-work-slash-webforum-moderation-and-UI-design-slash-grantmaking gig, so I took the opportunity ...
Feb 23, 2024•8 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/g8HHKaWENEbqh2mgK/updatelessness-doesn-t-solve-most-problems-1 Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 20, 2024•25 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/8yCXeafJo67tYe5L4/and-all-the-shoggoths-merely-players Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 20, 2024•22 min
Inspired by a house party inspired by Scott Alexander. By the time you arrive in Berkeley, the party is already in full swing. You’ve come late because your reading of the polycule graph indicated that the first half would be inauspicious. But now you’ve finally made it to the social event of the season: the Every Bay Area House Party-themed house party. The first order of the evening is to get a color-coded flirting wristband, so that you don’t incur any accidental micromarriages. You scan the ...
Feb 19, 2024•7 min
The Data 0. Population There were 558 responses over 32 days. The spacing and timing of the responses had hills and valleys because of an experiment I was performing where I'd get the survey advertised in a different place, then watch how many new responses happened in the day or two after that. Previous surveys have been run over the last decade or so. 2009: 166 2011: 1090 2012: 1195 2013: 1636 2014: 1503 2016: 3083 2017: "About 300" 2020: 61 2022: 186 2023: 558 Last year when I got a hundred a...
Feb 19, 2024•1 hr 53 min
Cross-posted with light edits from Otherwise. I think of us in some kind of twilight world as transformative AI looks more likely: things are about to change, and I don’t know if it's about to get a lot darker or a lot brighter. Increasingly this makes me wonder how I should be raising my kids differently. What might the world look like Most of my imaginings about my children's lives have them in pretty normal futures, where they go to college and have jobs and do normal human stuff, but with be...
Feb 18, 2024•8 min
This is a linkpost for https://seekingtobejolly.substack.com/p/no-one-in-my-org-puts-money-in-theirEpistemic status: the stories here are all as true as possible from memory, but my memory is so so. An AI made this This is going to be big It's late Summer 2017. I am on a walk in the Mendip Hills. It's warm and sunny and the air feels fresh. With me are around 20 other people from the Effective Altruism London community. We’ve travelled west for a retreat to discuss how to help others more effect...
Feb 18, 2024•15 min
This is a linkpost for https://www.narrativeark.xyz/p/masterpieceA sequel to qntm's Lena. Reading Lena first is helpful but not necessary. We’re excited to announce the fourth annual MMindscaping competition! Over the last few years, interest in the art of mindscaping has continued to grow rapidly. We expect this year's competition to be our biggest yet, and we’ve expanded the prize pool to match. The theme for the competition is “Weird and Wonderful”—we want your wackiest ideas and most off-the...
Feb 16, 2024•8 min
I'm trying to build my own art of rationality training, and I've started talking to various CFAR instructors about their experiences – things that might be important for me to know but which hadn't been written up nicely before. This is a quick write up of a conversation with Andrew Critch about his takeaways. (I took rough notes, and then roughly cleaned them up for this. I don't know "What surprised you most during your time at CFAR? Surprise 1: People are profoundly non-numerate. And, people ...
Feb 15, 2024•10 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/duvzdffTzL3dWJcxn/believing-in-1 Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 14, 2024•25 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/5jdqtpT6StjKDKacw/attitudes-about-applied-rationality Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓...
Feb 14, 2024•8 min
This is a hasty speculative fiction vignette of one way I expect we might get AGI by January 2025 (within about one year of writing this). Like similar works by others, I expect most of the guesses herein to turn out incorrect. However, this was still useful for expanding my imagination about what could happen to enable very short timelines, and I hope it's also useful to you. The assistant opened the door, and I walked into Director Yarden's austere office. For the Director of a major new feder...
Feb 14, 2024•16 min
This is a linkpost for https://garrisonlovely.substack.com/p/sam-altmans-chip-ambitions-undercut If you enjoy this, please consider subscribing to my Substack. Sam Altman has said he thinks that developing artificial general intelligence (AGI) could lead to human extinction, but OpenAI is trying to build it ASAP. Why? The common story for how AI could overpower humanity involves an “intelligence explosion,” where an AI system becomes smart enough to further improve its capabilities, bootstrappin...
Feb 11, 2024•7 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/PhTBDHu9PKJFmvb4p/a-shutdown-problem-proposal Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 09, 2024•12 min
People often parse information through an epistemic consensus filter. They do not ask "is this true", they ask "will others be OK with me thinking this is true". This makes them very malleable to brute force manufactured consensus; if every screen they look at says the same thing they will adopt that position because their brain interprets it as everyone in the tribe believing it. - Anon, 4Chan, slightly edited Ordinary people who haven't spent years of their lives thinking about rationality and...
Feb 04, 2024•9 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/GfZfDHZHCuYwrHGCd/without-fundamental-advances-misalignment-and-catastrophe Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 03, 2024•1 hr 41 min
Background Terminology: Counterfactual Impact vs “Leading The Parade” Y’know how a parade or marching band has a person who walks in front waving a fancy-looking stick up and down? Like this guy: The classic 80's comedy Animal House features a great scene in which a prankster steals the stick, and then leads the marching band off the main road and down a dead-end alley. That is not the guy who's supposed to have that stick.In the context of the movie, it's hilarious. It's also, presumably, not a...
Feb 02, 2024•17 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Source: https://www.lesswrong.com/posts/kcKrE9mzEHrdqtDpE/the-case-for-ensuring-that-powerful-ais-are-controlled Narrated for LessWrong by Perrin Walker . Share feedback on this narration. [Curated Post] ✓ [ 125+ Karma Post] ✓...
Feb 02, 2024•1 hr 4 min
I often encounter some confusion about whether the fact that synapses in the brain typically fire at frequencies of 1-100 Hz while the clock frequency of a state-of-the-art GPU is on the order of 1 GHz means that AIs think "many orders of magnitude faster" than humans. In this short post, I'll argue that this way of thinking about "cognitive speed" is quite misleading. The clock speed of a GPU is indeed meaningful: there is a clock inside the GPU that provides some signal that's periodic at a fr...
Feb 01, 2024•5 min
A pdf version of this report is available here. Summary. In this report we argue that AI systems capable of large scale scientific research will likely pursue unwanted goals and this will lead to catastrophic outcomes. We argue this is the default outcome, even with significant countermeasures, given the current trajectory of AI development. In Section 1 we discuss the tasks which are the focus of this report. We are specifically focusing on AIs which are capable of dramatically speeding up larg...
Jan 31, 2024•1 hr 57 min
This is a linkpost for https://rootsofprogress.org/the-block-funding-model-for-scienceWhen Galileo wanted to study the heavens through his telescope, he got money from those legendary patrons of the Renaissance, the Medici. To win their favor, when he discovered the moons of Jupiter, he named them the Medicean Stars. Other scientists and inventors offered flashy gifts, such as Cornelis Drebbel's perpetuum mobile (a sort of astronomical clock) given to King James, who made Drebbel court engineer ...
Jan 29, 2024•7 min
