The Less Wrong General Census is unofficially here! You can take it at this link. It's that time again. If you are reading this post and identify as a LessWronger, then you are the target audience. I'd appreciate it if you took the survey. If you post, if you comment, if you lurk, if you don't actually read the site that much but you do read a bunch of the other rationalist blogs or you're really into HPMOR, if you hung out on rationalist tumblr back in the day, or if none of those exactly fit y...
Dec 13, 2023•2 min
If you are interested in the longevity scene, like I am, you probably have seen press releases about the dog longevity company, Loyal for Dogs, getting a nod for efficacy from the FDA. These have come in the form of the New York Post calling the drug "groundbreaking", Science Alert calling the drug "radical", and the more sedate New York Times just asking, "Could Longevity Drugs for Dogs Extend Your Pet's Life?", presumably unaware of Betteridge's Law of Headlines. You may have also seen the coo...
Dec 13, 2023•10 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated Crossposted from Otherwise Parents supervise their children way more than they used to Children spend less of their time in unstructured play than they did in past generations. Parental supervision is way up. The wild thing is that this is true even while the number of children per family has decreased and the amount of time mothers work outside the home has increased. Source: https://www.lesswrong.com/posts...
Dec 13, 2023•13 min
In the course of my life, there have been a handful of times I discovered an idea that changed the way I thought about the world. The first occurred when I picked up Nick Bostrom's book “superintelligence” and realized that AI would utterly transform the world. The second was when I learned about embryo selection and how it could change future generations. And the third happened a few months ago when I read a message from a friend of mine on Discord about editing the genome of a living person. W...
Dec 12, 2023•1 hr 8 min
I was asked to respond to this comment by Eliezer Yudkowsky. This post is partly redundant with my previous post. Why is flesh weaker than diamond? When trying to resolve disagreements, I find that precision is important. Tensile strength, compressive strength, and impact strength are different. Material microstructure matters. Poorly-sintered diamond crystals could crumble like sand, and a large diamond crystal has lower impact strength than some materials made of proteins. Even when the load-b...
Dec 11, 2023•11 min
In May and June of 2023, I (Akash) had about 50-70 meetings about AI risks with congressional staffers. I had been meaning to write a post reflecting on the experience and some of my takeaways, and I figured it could be a good topic for a LessWrong dialogue. I saw that hath had offered to do LW dialogues with folks, and I reached out. In this dialogue, we discuss how I decided to chat with staffers, my initial observations in DC, some context about how Congressional offices work, what my meeting...
Dec 05, 2023•30 min
Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated You can’t optimise an allocation of resources if you don’t know what the current one is. Existing maps of alignment research are mostly too old to guide you and the field has nearly no ratchet , no common knowledge of what everyone is doing and why, what is abandoned and why, what is renamed, what relates to what, what is going on. This post is mostly just a big index: a link-dump for as many currently activ...
Dec 04, 2023•1 hr 3 min
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Quintin Pope & Nora Belrose have a new “AI Optimists” website, along with a new essay “AI is easy to control”, arguing that the risk of human extinction due to future AI (“AI x-risk”) is a mere 1% (“a tail risk worth considering, but not the dominant source of risk in the world”). (I’m much more pessimistic.) It makes lots of interesting arguments, and I’m happy that the authors are engaging in substantive ...
Dec 02, 2023•23 min
Any community which ever adds new people will need to either routinely teach the new and (to established members) blindingly obvious information to those who genuinely haven’t heard it before, or accept that over time community members will only know the simplest basics by accident of osmosis or selection bias. There isn’t another way out of that. You don’t get to stop doing it. If you have a vibrant and popular group full of people really interested in the subject of the group, and you run it f...
Nov 30, 2023•9 min
The author's Substack: https://substack.com/@homosabiens Support ongoing human narrations of LessWrong's curated posts: www.patreon.com/LWCurated You know it must be out there, but you mostly never see it. Author's Note 1: In something like 75% of possible futures, this will be the last essay that I publish on LessWrong. Future content will be available on my substack , where I'm hoping people will be willing to chip in a little commensurate with the value of the writing, and (after a delay) on ...
Nov 28, 2023•1 hr 6 min
Crossposted from the AI Alignment Forum. May contain more technical jargon than usual.Summary. You can’t optimise an allocation of resources if you don’t know what the current one is. Existing maps of alignment research are mostly too old to guide you and the field has nearly no ratchet, no common knowledge of what everyone is doing and why, what is abandoned and why, what is renamed, what relates to what, what is going on. This post is mostly just a big index: a link-dump for as many currently ...
Nov 28, 2023•1 hr 17 min
Status: Vague, sorry. The point seems almost tautological to me, and yet also seems like the correct answer to the people going around saying “LLMs turned out to be not very want-y, when are the people who expected 'agents' going to update?”, so, here we are. Okay, so you know how AI today isn't great at certain... let's say "long-horizon" tasks? Like novel large-scale engineering projects, or writing a long book series with lots of foreshadowing? (Modulo the fact that it can play chess pretty w...
Nov 25, 2023•8 min
Support ongoing human narrations of curated posts: www.patreon.com/LWCurated Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well. TL;DR Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happ...
Nov 23, 2023•6 min
Previously: OpenAI: Facts from a Weekend. On Friday afternoon, OpenAI's board fired CEO Sam Altman. Overnight, an agreement in principle was reached to reinstate Sam Altman as CEO of OpenAI, with an initial new board of Brad Taylor (ex-co-CEO of Salesforce, chair), Larry Summers and Adam D’Angelo. What happened? Why did it happen? How will it ultimately end? The fight is far from over. We do not entirely know, but we know a lot more than we did a few days ago. This is my attempt to put the piece...
Nov 22, 2023•20 min
Approximately four GPTs and seven years ago, OpenAI's founders brought forth on this corporate landscape a new entity, conceived in liberty, and dedicated to the proposition that all men might live equally when AGI is created. Now we are engaged in a great corporate war, testing whether that entity, or any entity so conceived and so dedicated, can long endure. What matters is not theory but practice. What happens when the chips are down? So what happened? What prompted it? What will happen now? ...
Nov 20, 2023•17 min
This is a linkpost for https://openai.com/blog/openai-announces-leadership-transitionBasically just the title, see the OAI blog post for more details. Mr. Altman's departure follows a deliberative review process by the board, which concluded that he was not consistently candid in his communications with the board, hindering its ability to exercise its responsibilities. The board no longer has confidence in his ability to continue leading OpenAI. In a statement, the board of directors said: “Open...
Nov 18, 2023•1 min
You know it must be out there, but you mostly never see it. Author's Note 1: I'm something like 75% confident that this will be the last essay that I publish on LessWrong. Future content will be available on my substack, where I'm hoping people will be willing to chip in a little commensurate with the value of the writing, and (after a delay) on my personal site. I decided to post this final essay here rather than silently switching over because many LessWrong readers would otherwise never find ...
Nov 17, 2023•53 min
Support ongoing human narrations of curated posts: www.patreon.com/LWCurated I'm sure Harry Potter and the Methods of Rationality taught me some of the obvious, overt things it set out to teach. Looking back on it a decade after I first read it however, what strikes me most strongly are often the brief, tossed off bits in the middle of the flow of a story. Fred and George exchanged worried glances."I can't think of anything," said George."Neither can I," said Fred. "Sorry."Harry stared at them.A...
Nov 17, 2023•14 min
Here's a recent conversation I had with a friend: Me: "I wish I had more friends. You guys are great, but I only get to hang out with you like once or twice a week. It's painful being holed up in my house the entire rest of the time."Friend: "You know ${X}. You could talk to him."Me: "I haven't talked to ${X} since 2019."Friend: "Why does that matter? Just call him."Me: "What do you mean 'just call him'? I can't do that."Friend: "Yes you can"Me: Source: https://www.lesswrong.com/posts/2HawAteFsn...
Nov 17, 2023•2 min
Support ongoing human narrations of curated posts: www.patreon.com/LWCurated How many years will pass before transformative AI is built? Three people who have thought about this question a lot are Ajeya Cotra from Open Philanthropy , Daniel Kokotajlo from OpenAI and Ege Erdil from Epoch . Despite each spending at least hundreds of hours investigating this question, they still still disagree substantially about the relevant timescales. For instance, here are their median timelines for one operati...
Nov 17, 2023•1 hr 18 min
It’s fairly common for EA orgs to provide fiscal sponsorship to other EA orgs. Wait, no, that sentence is not quite right. The more accurate sentence is that there are very few EA organizations, in the legal sense; most of what you think of as orgs are projects that are legally hosted by a single org, and which governments therefore consider to be one legal entity. Source: https://www.lesswrong.com/posts/XvEJydHAHk6hjWQr5/ea-orgs-legal-structure-inhibits-risk-taking-and-information Narrated for ...
Nov 17, 2023•8 min
habryka Ok, so we both had some feelings about the recent Conjecture post on "lots of people in AI Alignment are lying" , and the associated marketing campaign and stuff . I would appreciate some context in which I can think through that, and also to share info we have in the space that might help us figure out what's going on. I expect this will pretty quickly cause us to end up on some broader questions about how to do advocacy, how much the current social network around AI Alignment should co...
Nov 17, 2023•40 min
1. There's a supercharged, dire wolf form of the bystander effect that I’d like to shine a spotlight on. First, a quick recap. The Bystander Effect is a phenomenon where people are less likely to help when there's a group around. When I took basic medical training, I was told to always ask one specific person to take actions instead of asking a crowd at large. “You, in the green shirt! Call 911!” (911 is the emergency services number in the United States.) One habit I worked hard to instill in m...
Nov 16, 2023•10 min
Support ongoing human narrations of curated posts: www.patreon.com/LWCurated (You can sign up to play deception chess here if you haven't already.) This is the first of my analyses of the deception chess games. The introduction will describe the setup of the game, and the conclusion will sum up what happened in general terms; the rest of the post will mostly be chess analysis and skippable if you just want the results. If you haven't read the original post , read it before reading this so that y...
Nov 09, 2023•17 min
Support ongoing human narrations of curated posts: www.patreon.com/LWCurated This is a linkpost for https://transformer-circuits.pub/2023/monosemantic-features/ Text of post based on our blog post as a linkpost for the full paper which is considerably longer and more detailed. Neural networks are trained on data, not programmed to follow rules. We understand the math of the trained network exactly – each neuron in a neural network performs simple arithmetic – but we don't understand why those ma...
Nov 09, 2023•8 min
Recently, I have been learning about industry norms, legal discovery proceedings, and incentive structures related to companies building risky systems. I wanted to share some findings in this post because they may be important for the frontier AI community to understand well. TL;DR Documented communications of risks (especially by employees) make companies much more likely to be held liable in court when bad things happen. The resulting Duty to Due Diligence from Discoverable Documentation of Da...
Nov 09, 2023•5 min
I guess there’s maybe a 10-20% chance of AI causing human extinction in the coming decades, but I feel more distressed about it than even that suggests—I think because in the case where it doesn’t cause human extinction, I find it hard to imagine life not going kind of off the rails. So many things I like about the world seem likely to be over or badly disrupted with superhuman AI (writing, explaining things to people, friendships where you can be of any use to one another, taking pride in skill...
Nov 09, 2023•1 min
davidad has a 10-min talk out on a proposal about which he says: “the first time I’ve seen a concrete plan that might work to get human uploads before 2040, maybe even faster, given unlimited funding”. I think the talk is a good watch, but the dialogue below is pretty readable even if you haven't seen it. I'm also putting some summary notes from the talk in the Appendix of this dialogue. Source: https://www.lesswrong.com/posts/FEFQSGLhJFpqmEhgi/does-davidad-s-uploading-moonshot-work Narrated for...
Nov 09, 2023•50 min
I examined all the biorisk-relevant citations from a policy paper arguing that we should ban powerful open source LLMs. None of them provide good evidence for the paper's conclusion. The best of the set is evidence from statements from Anthropic -- which rest upon data that no one outside of Anthropic can even see, and on Anthropic's interpretation of that data. The rest of the evidence cited in this paper ultimately rests on a single extremely questionable "experiment" without a control group. ...
Nov 09, 2023•42 min
A common theme implicit in many AI risk stories has been that broader society will either fail to anticipate the risks of AI until it is too late, or do little to address those risks in a serious manner. In my opinion, there are now clear signs that this assumption is false, and that society will address AI with something approaching both the attention and diligence it deserves. For example, one clear sign is Joe Biden's recent executive order on AI safety [1] . In light of recent news, it is wo...
Nov 09, 2023•16 min
