At least, if you happen to be near me in brain space. What advice would you give your younger self? That was the prompt for a class I taught at PAIR 2024. About a quarter of participants ranked it in their top 3 of courses at the camp and half of them had it listed as their favorite. I hadn’t expected that. I thought my life advice was pretty idiosyncratic. I never heard of anyone living their life like I have. I never encountered this method in all the self-help blogs or feel-better books I con...
Nov 04, 2024•21 min
I open my eyes and find myself lying on a bed in a hospital room. I blink. "Hello", says a middle-aged man with glasses, sitting on a chair by my bed. "You've been out for quite a long while." "Oh no ... is it Friday already? I had that report due -" "It's Thursday", the man says. "Oh great", I say. "I still have time." "Oh, you have all the time in the world", the man says, chuckling. "You were out for 21 years." I burst out laughing, but then falter as the man just keeps looking at me. "You me...
Nov 04, 2024•30 min
Claim: memeticity in a scientific field is mostly determined, not by the most competent researchers in the field, but instead by roughly-median researchers. We’ll call this the “median researcher problem”. Prototypical example: imagine a scientific field in which the large majority of practitioners have a very poor understanding of statistics, p-hacking, etc. Then lots of work in that field will be highly memetic despite trash statistics, blatant p-hacking, etc. Sure, the most competent people i...
Nov 04, 2024•3 min
This is a link post.We (Connor Leahy, Gabriel Alfour, Chris Scammell, Andrea Miotti, Adam Shimi) have just published The Compendium, which brings together in a single place the most important arguments that drive our models of the AGI race, and what we need to do to avoid catastrophe. We felt that something like this has been missing from the AI conversation. Most of these points have been shared before, but a “comprehensive worldview” doc has been missing. We’ve tried our best to fill this gap,...
Nov 01, 2024•4 min
There are two nuclear options for treating depression: Ketamine and TMS; This post is about the latter. TMS stands for Transcranial Magnetic Stimulation. Basically, it fixes depression via magnets, which is about the second or third most magical things that magnets can do. I don’t know a whole lot about the neuroscience - this post isn’t about the how or the why. It's from the perspective of a patient, and it's about the what. What is it like to get TMS? TMS The Gatekeeping For Reasons™, doctors...
Oct 31, 2024•11 min
Epistemic status: model-building based on observation, with a few successful unusual predictions. Anecdotal evidence has so far been consistent with the model. This puts it at risk of seeming more compelling than the evidence justifies just yet. Caveat emptor. Imagine you're a very young child. Around, say, three years old. You've just done something that really upsets your mother. Maybe you were playing and knocked her glasses off the table and they broke. Of course you find her reaction uncomf...
Oct 28, 2024•29 min
This post includes a "flattened version" of an interactive diagram that cannot be displayed on this site. I recommend reading the original version of the post with the interactive diagram, which can be found here. Over the last few months, ARC has released a number of pieces of research. While some of these can be independently motivated, there is also a more unified research vision behind them. The purpose of this post is to try to convey some of that vision and how our individual pieces of res...
Oct 27, 2024•11 min
1. 4.4% of the US federal budget went into the space race at its peak. This was surprising to me, until a friend pointed out that landing rockets on specific parts of the moon requires very similar technology to landing rockets in soviet cities.[1] I wonder how much more enthusiastic the scientists working on Apollo were, with the convenient motivating story of “I’m working towards a great scientific endeavor” vs “I’m working to make sure we can kill millions if we want to”. 2. The field of alig...
Oct 25, 2024•3 min
This summer, I participated in a human challenge trial at the University of Maryland. I spent the days just prior to my 30th birthday sick with shigellosis. What? Why? Dysentery is an acute disease in which pathogens attack the intestine. It is most often caused by the bacteria Shigella. It spreads via the fecal-oral route. It requires an astonishingly low number of pathogens to make a person sick – so it spreads quickly, especially in bad hygienic conditions or anywhere water can get tainted wi...
Oct 24, 2024•32 min
This is a link post. Part 1: Our Thinking Near and Far 1 Abstract/Distant Future Bias 2 Abstractly Ideal, Concretely Selfish 3 We Add Near, Average Far 4 Why We Don't Know What We Want 5 We See the Sacred from Afar, to See It Together 6 The Future Seems Shiny 7 Doubting My Far Mind Disagreement 8 Beware the Inside View 9 Are Meta Views Outside Views? 10 Disagreement Is Near-Far Bias 11 Others' Views Are Detail 12 Why Be Contrarian? 13 On Disagreement, Again 14 Rationality Requires Common Priors ...
Oct 23, 2024•9 min
Of all the cognitive tools our ancestors left us, what's best? Society seems to think pretty highly of arithmetic. It's one of the first things we learn as children. So I think it's weird that only a tiny percentage of people seem to know how to actually use arithmetic. Or maybe even understand what arithmetic is for. Why? I think the problem is the idea that arithmetic is about “calculating”. No! Arithmetic is a world-modeling technology. Arguably, it's the best world-modeling technology: It's ...
Oct 22, 2024•12 min
This post starts out pretty gloomy but ends up with some points that I feel pretty positive about. Day to day, I'm more focussed on the positive points, but awareness of the negative has been crucial to forming my priorities, so I'm going to start with those. It's mostly addressed to the EA community, but is hopefully somewhat of interest to LessWrong and the Alignment Forum as well. My main concerns I think AGI is going to be developed soon, and quickly. Possibly (20%) that's next year, and mos...
Oct 15, 2024•25 min
This post focuses on philosophical objections to Bayesianism as an epistemology. I first explain Bayesianism and some standard objections to it, then lay out my two main objections (inspired by ideas in philosophy of science). A follow-up post will speculate about how to formalize an alternative. Degrees of belief The core idea of Bayesianism: we should ideally reason by assigning credences to propositions which represent our degrees of belief that those propositions are true. If that seems like...
Oct 15, 2024•18 min
As humanity gets closer to Artificial General Intelligence (AGI), a new geopolitical strategy is gaining traction in US and allied circles, in the NatSec, AI safety and tech communities. Anthropic CEO Dario Amodei and RAND Corporation call it the “entente”, while others privately refer to it as “hegemony" or “crush China”. I will argue that, irrespective of one's ethical or geopolitical preferences, it is fundamentally flawed and against US national security interests. If the US fights China in ...
Oct 14, 2024•18 min
I think that most people underestimate how many scientific mysteries remain, even on questions that sound basic. My favourite candidate for "the most basic thing that is still unknown" is the momentum carried by light, when it is in a medium (for example, a flash of light in glass or water). If a block of glass has a refractive index of <span>_n_</span>, then the light inside that block travels <span>_n_</span> times slower than the light would in vacuum. But what is the ...
Oct 14, 2024•19 min
How can we make many humans who are very good at solving difficult problems? Summary (table of made-up numbers) I made up the made-up numbers in this table of made-up numbers; therefore, the numbers in this table of made-up numbers are made-up numbers. Call to action If you have a shitload of money, there are some projects you can give money to that would make supergenius humans on demand happen faster. If you have a fuckton of money, there are projects whose creation you could fund that would g...
Oct 09, 2024•25 min
This post is probably hazardous for one type of person in one particular growth stage, and necessary for people in a different growth stage, and I don't really know how to tell the difference in advance. If you read it and feel like it kinda wrecked you send me a DM. I'll try to help bandage it. One of my favorite stories growing up was Star Wars: Traitor, by Matthew Stover. The book is short, if you want to read it. Spoilers follow. (I took a look at it again recently and I think it didn't obvi...
Oct 03, 2024•12 min
This is a description of my work on some data science projects, lightly obfuscated and fictionalized to protect the confidentiality of the organizations I handled them for (and also to make it flow better). I focus on the high-level epistemic/mathematical issues, and the lived experience of working on intellectual problems, but gloss over the timelines and implementation details. The Upper Bound One time, I was working for a company which wanted to win some first-place sealed-bid auctions in a m...
Oct 03, 2024•8 min
[Meta note: quickly written, unpolished. Also, it's possible that there's some more convincing work on this topic that I'm unaware of – if so, let me know] In research discussions about LLMs, I often pick up a vibe of casual, generalized skepticism about model-generated CoT (chain-of-thought) explanations. CoTs (people say) are not trustworthy in general. They don't always reflect what the model is "actually" thinking or how it has "actually" solved a given problem. This claim is true as far as ...
Sep 30, 2024•22 min
I've been wanting to write a nice post for a few months, but should probably just write a one sooner instead. This is a top-level post not because it's a long text, but because it's important text. Anyways. Cryonics is pretty much money-free now—one of the most affordable ways to dispose of your body post-mortem. In the west coast in the USA, from Oregon Brain Preservation, as of around May 2024 I think: Our research program is open to individuals in Washington, Oregon, and Northern California. ...
Sep 30, 2024•3 min
Quarterly Performance Review, Autumn 1983 Colonel Yuri Kuznetsov looked out the window anxiously. The endless gray landscape did little to soothe his nerves. He only had one employee review left to get through, but he’d saved the hardest one for last. He wasn’t upset about having to dismiss Lieutenant Colonel Petrov—he couldn’t wait to be rid of the little shit—but he couldn’t shake the feeling that something was amiss. He took a swig from his flask. “Stanislav, you can come in now,” Yuri shoute...
Sep 29, 2024•9 min
I’ve claimed that Willpower compounds and that small wins in the present make it easier to get bigger wins in the future. Unfortunately, procrastination and laziness compound, too. You’re stressed out for some reason, so you take the evening off for a YouTube binge. You end up staying awake a little later than usual and sleeping poorly. So the next morning you feel especially tired; you snooze a few extra times. In your rushed morning routine you don’t have time to prepare for the work meeting a...
Sep 29, 2024•16 min
For a long time, when I heard "slow takeoff", I assumed it meant "takeoff that takes longer calendar time than fast takeoff." (i.e. what is now referred to more often as "short timelines" vs "long timelines."). I think Paul Christiano popularized the term, and it so happened he both expected to see longer timelines and smoother/continuous takeoff. I think it's at least somewhat confusing to use the term "slow" to mean "smooth/continuous", because that's not what "slow" particularly means most of...
Sep 29, 2024•3 min
A common claim among e/accs is that, since the solar system is big, Earth will be left alone by superintelligences. A simple rejoinder is that just because Bernard Arnault has $170 billion, does not mean that he'll give you $77.18. Earth subtends only 4.54e-10 = 0.0000000454% of the angular area around the Sun, according to GPT-o1.[1] Asking an ASI to leave a hole in a Dyson Shell, so that Earth could get some sunlight not transformed to infrared, would cost It 4.5e-10 of Its income. This is lik...
Sep 23, 2024•19 min
A year ago, I started trying to deliberate practice skills that would "help people figure out the answers to confusing, important questions." I experimented with Thinking Physics questions, GPQA questions, Puzzle Games , Strategy Games, and a stupid twitchy reflex game I had struggled to beat for 8 years[1]. Then I went back to my day job and tried figuring stuff out there too. The most important skill I was trying to learn was Metastrategic Brainstorming[2] – the skill of looking at a confusing...
Sep 21, 2024•13 min
After the release of Ben Pace's extended interview with me about my views on religion, I felt inspired to publish more of my thinking about religion in a format that's more detailed, compact, and organized. This post is the first publication in my series of intended posts about religion. Thanks to Ben Pace, Chris Lakin, Richard Ngo, Renshin Lauren Lee, Mark Miller, and Imam Ammar Amonette for their feedback on this post, and thanks to Kaj Sotala, Tomáš Gavenčiak, Paul Colognese, and David Spivak...
Sep 19, 2024•14 min
There's a popular story that goes like this: Christopher Hitchens used to be in favor of the US waterboarding terrorists because he though it's wasn't bad enough to be torture.. Then he had it tried on himself, and changed his mind, coming to believe it isn't torture. (Context for those unfamiliar: in the decade following 9/11, the US engaged in a lot of... questionable behavior to persecute the war on terror, and there was a big debate on whether waterboarding should be permitted. Many other pu...
Sep 17, 2024•13 min
Midjourney, “Fourth Industrial Revolution Digital Transformation”This is a little rant I like to give, because it's something I learned on the job that I’ve never seen written up explicitly. There are a bunch of buzzwords floating around regarding computer technology in an industrial or manufacturing context: “digital transformation”, “the Fourth Industrial Revolution”, “Industrial Internet of Things”. What do those things really mean? Do they mean anything at all? The answer is yes, and what th...
Sep 15, 2024•18 min
[Conflict of interest disclaimer: We are FutureSearch, a company working on AI-powered forecasting and other types of quantitative reasoning. If thin LLM wrappers could achieve superhuman forecasting performance, this would obsolete a lot of our work.] Widespread, misleading claims about AI forecasting Recently we have seen a number of papers – (Schoenegger et al., 2024, Halawi et al., 2024, Phan et al., 2024, Hsieh et al., 2024) – with claims that boil down to “we built an LLM-powered forecaste...
Sep 14, 2024•18 min
This is a link post. --- First published: September 12th, 2024 Source: https://www.lesswrong.com/posts/bhY5aE4MtwpGf3LCo/openai-o1 --- Narrated by TYPE III AUDIO ....
Sep 13, 2024•22 sec
