Gwern has answered my prayers and taught GPT-2 poetry . GPT-2 is the language processing system that OpenAI announced a few weeks ago. They are keeping the full version secret, but have released a smaller prototype version. Gwern retrained it on the Gutenberg Poetry Corpus , a 117 MB collection of pre-1923 English poetry, to create a specialized poetry AI. I previously tested the out-of-the-box version of GPT-2 and couldn’t make it understand rhyme and meter. I wrongly assumed this was a fundame...
Mar 17, 2019•17 min•Ep. 172
Here’s a graph of US air pollution over time: During the discussion of 90s environmentalism, some people pointed out that this showed the Clean Air Act didn’t matter. The trend is the same before the Act as after it. This kind of argument is common. For example, here’s the libertarian Mercatus Institute arguing that OSHA didn’t help workplace safety: I’ve always taken these arguments pretty seriously. But recently I’ve gotten more cautious. Here’s a graph of Moore’s Law, the “rule” that transist...
Mar 15, 2019•7 min•Ep. 171
[ Related to: Book Review: Albion’s Seed ] [ Epistemic status: Not too serious] I realize I’ve been confusing everyone with my use of the word “Puritan”. When I say “That guy is so Puritan!” people object “But he’s not religious!” or “He doesn’t hate fun!” I don’t know what the real word for the category I’m calling “Puritan” is. Words like “Yankee”, “Boston Brahmin”, or “Transcendentalist” are close, but none of them really work. “Eccentric overeducated hypercompetent contrarian early American ...
Mar 14, 2019•17 min•Ep. 170
I. Albion’s Seed by David Fischer is a history professor’s nine-hundred-page treatise on patterns of early immigration to the Eastern United States. It’s not light reading and not the sort of thing I would normally pick up. I read it anyway on the advice of people who kept telling me it explains everything about America. And it sort of does. In school, we tend to think of the original American colonists as “Englishmen”, a maximally non-diverse group who form the background for all of the diversi...
Mar 13, 2019•1 hr 11 min•Ep. 169
Last week the FDA approved esketamine for treatment-resistant depression. Let’s review how the pharmaceutical industry works: a company discovers and patents a potentially exciting new drug. They spend tens of millions of dollars proving safety and efficacy to the FDA. The FDA rewards them with a 10ish year monopoly on the drug, during which they can charge whatever ridiculous price they want. This isn’t a great system, but at least we get new medicines sometimes. Occasionally people discover th...
Mar 12, 2019•15 min•Ep. 168
[Title from this unrelated story or this unrelated essay ] Last week I wrote about how conspiracy theories spread so much faster on Facebook than debunkings of those same theories. A few commenters chimed in to say that of course this was true, the conspiracy theories had evolved into an almost-perfect form for exploiting cognitive biases and the pressures of social media. Debunkings and true beliefs couldn’t copy that process, so they were losing out. This sounded like a challenge, so here you ...
Mar 06, 2019•10 min•Ep. 167
[With apologies to Putnam, Pope, and all of you] Two children are reading a text written by an AI : The hobbits splashed water in each other’s faces until they were both sopping wet One child says to the other “Wow! After reading some text, the AI understands what water is!” The second child says “It doesn’t really understand.” The first child says “Sure it does! It understands that water is the sort of substance that splashes. It understands that people who are splashed with water get wet. What...
Mar 02, 2019•6 min•Ep. 166
The Verge writes a story (an exposé?) on the Facebook-moderation industry. It goes through the standard ways it maltreats its employees: low pay, limited bathroom breaks, awful managers – and then into some not-so-standard ones. Mods have to read (or watch) all of the worst things people post on Facebook, from conspiracy theories to snuff videos. The story talks about the psychological trauma this inflicts: It’s an environment where workers cope by telling dark jokes about committing suicide, th...
Mar 01, 2019•14 min•Ep. 165
Imagine a black box which, when you pressed a button, would generate a scientific hypothesis. 50% of its hypotheses are false; 50% are true hypotheses as game-changing and elegant as relativity. Even despite the error rate, it’s easy to see this box would quickly surpass space capsules, da Vinci paintings, and printer ink cartridges to become the most valuable object in the world. Scientific progress on demand, and all you have to do is test some stuff to see if it’s true ? I don’t want to deval...
Feb 28, 2019•9 min•Ep. 164
[Epistemic status: I am basing this on widely-accepted published research, but I can’t guarantee I’ve understood the research right or managed to emphasize/believe the right people. Some light editing to bring in important points people raised in the comments.] You all know this graph: Median wages tracked productivity until 1973, then stopped. Productivity kept growing, but wages remained stagnant. This is called “wage decoupling”. Sometimes people talk about wages decoupling from GDP, or from ...
Feb 28, 2019•52 min•Ep. 163
[This post is having major technical issues. Some comments may not be appearing. If you can’t comment, please say so on the subreddit.] I. I Come To Praise Caesar, Not To Bury Him Several years ago, an SSC reader made an r/slatestarcodex subreddit for discussion of blog posts here and related topics. As per the usual process , the topics that generated the strongest emotions – Trump, gender, race, the communist menace, the fascist menace, etc – started taking over. The moderators (and I had been...
Feb 23, 2019•40 min•Ep. 162
I was going back over yesterday’s post , and something sounded familiar about this paragraph: A very careless plagiarist takes someone else’s work and copies it verbatim: “The mitochondria is the powerhouse of the cell”. A more careful plagiarist takes the work and changes a few words around: “The mitochondria is the energy dynamo of the cell”. A plagiarist who is more careful still changes the entire sentence structure: “In cells, mitochondria are the energy dynamos”. The most careful plagiaris...
Feb 22, 2019•4 min•Ep. 161
A machine learning researcher writes me in response to yesterday’s post , saying: I still think GPT-2 is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked. I resisted the urge to answer “Yeah, well, your mom is a brute-force statistical pattern matcher which blends up the internet and gives you back a slightly unappetizing slurry of it when asked.” But I think it would have been true. A very careless plagiari...
Feb 21, 2019•27 min•Ep. 160
Last week OpenAI announced its latest breakthrough . GPT-2 is a language model that can write essays to a prompt, answer questions, and summarize longer works. For example (bold prompt is human-written, all other text is the machine-generated response): Prompt: For today’s homework assignment, please describe the reasons for the US Civil War. It is easy to identify why the Civil War happened, because so many people and so many books and so much television and films tell us that it was the cause,...
Feb 21, 2019•17 min•Ep. 159
I. Chefs. Hundreds of them. Tall chefs, short chefs, black chefs, white chefs. I pushed forward through them, like an explorer hacking away at undergrowth. They muttered curses at me, but I was stronger than they were. I came to a door. I opened it. Sweet empty space. I shut the door behind me, sat down in the chair. “Hello,” I said. “Detective Paul Eastman, pleased to make your acquaintance.” “Doctor Zachary LaShay,” said the man behind the desk. His little remaining hair was greying; his eyes ...
Feb 16, 2019•44 min•Ep. 158
SSRIs are the most widely used class of psychiatric medications, helpful for depression, anxiety, OCD, panic, PTSD, anger, and certain personality disorders (Why should the same drug treat all these things? Great question!) They’ve been pretty thoroughly studied, but there’s still a lot we don’t understand about them. The SSC Survey is less rigorous than most existing studies, but its many questions and very high sample size provide a different tool to investigate some of these issues. I asked f...
Feb 09, 2019•14 min•Ep. 157
I. I don’t know much about gay history, but the heavily mythicized version of it I heard goes like this: At first open homosexuality was totally taboo. A few groups of respectable people with hilariously upper-class names like The Mattachine Society and The Daughters Of Bilitis quietly tried to influence elites in favor of more tolerance, using whatever backchannels elites use to influence one another. They had limited success, but they comforted themselves that at least they were presenting a l...
Feb 07, 2019•12 min•Ep. 156
I. Zero To One might be the first best-selling business book based on a Tumblr. Stanford student Blake Masters took Peter Thiel’s class on startups. He posted his notes on Tumblr after each lecture. They became a minor sensation. Thiel asked if he wanted to make them into a book together. He did. The title comes from Thiel’s metaphor that ordinary businessmen like restaurant owners take a product “from 1 to n” (shouldn’t this be from n to n+1?) – they build more of something that already exists....
Feb 02, 2019•42 min•Ep. 155
At the beginning of every year, I make predictions . At the end of every year, I score them . So here are a hundred more for 2019. Rules: all predictions about what will be true on January 1, 2020. Any that involve polling will be settled by the top poll or average of polls on Real Clear Politics on that day. Most predictions about my personal life, or that refer to the personal lives of other people, have been redacted to protect their privacy. I’m using the full 0 – 100 range in making predict...
Jan 26, 2019•14 min•Ep. 154
Lots of people have asked me to recommend them a psychiatrist or therapist. I’ve done a terrible job responding: it’s a conflict of interest to recommend my own group, and I don’t know many people outside of it. So now I’ve put together a list (by which I mostly mean blatantly copied a similar list made by fellow community member Anisha M) of mental health professionals whom members of the rationalist community have had good experiences with. So far it’s short and mostly limited to the Bay Area....
Jan 25, 2019•2 min•Ep. 153
At the beginning of every year, I make predictions. At the end of every year, I score them. Here are 2014 , 2015 , 2016 , and 2017 . And here are the predictions I made for 2018. Strikethrough’d are false. Intact are true. Italicized are getting thrown out because I can’t decide if they’re true or not. Please don’t complain that 50% predictions don’t mean anything; I know this is true but there are some things I’m genuinely 50-50 unsure of. US: 1. Donald Trump remains president at end of year: 9...
Jan 24, 2019•20 min•Ep. 152
Thanks to everyone who commented on the review of The Structure Of Scientific Revolutions . From David Chapman : It’s important to remember that Kuhn wrote this seven decades ago. It was one of the most influential books of pop philosophy in the 1960s-70s, influencing the counterculture of the time, so it is very much “in the water supply.” Much of what’s right in it is now obvious; what’s wrong is salient. To make sense of the book, you have to understand the state of the philosophy of science ...
Jan 20, 2019•43 min•Ep. 151
In 2010, Ben Tilly of the blog Random Observations wrote Analysis Vs. Algebra Predicts Eating Corn? , which said: I like learning about odd connections between disparate things. This probably is the oddest example that I know. Broadly speaking, mathematicians can be divided into those who like analysis, and those who like algebra. The distinction between the two types runs throughout math. Even those who work in areas that are far from analysis or algebra are very aware of the difference between...
Jan 17, 2019•11 min•Ep. 150
[Content warning: References to anti-Semitic and anti-Catholic canards] I feel deep affection for Gary Allen’s None Dare Call It Conspiracy , a bizarre screed about the Federal Reserve/Communist/Trilateral Commission plot for a one world government. From its ridiculous title to its even-more-ridiculous cover image , this is a book that accepts its own nature. In the Aristotelian framework, where everything is trying to be the most perfect example of whatever it is, None Dare Call It Conspiracy h...
Jan 16, 2019•44 min•Ep. 149
Thanks to the 8,171 people who took the 2019 Slate Star Codex survey. Some of the links below will say 13,171 people took the survey, but that’s a bug – sometimes Google just adds 5,000 to things. You can: – See the questions for the SSC survey. – See the results from the SSC survey. I’ll be publishing more complicated analyses over the course of the next year, hopefully starting later this week. If you want to scoop me, or investigate the data yourself, you can download the answers of the 7000 ...
Jan 14, 2019•2 min•Ep. 148
Related to: Book Review: The Structure Of Scientific Revolutions Every good conspiracy theorist needs their own Grand Unified Chart; I’m a particular fan of this one . So far, my own Grand Unified Chart looks like this: All of these are examples of interpreting the world through a combination of pre-existing ideas what the world should be like (first column), plus actually experiencing the world (last column). In all of them, the world is too confusing and permits too many different interpretati...
Jan 12, 2019•6 min•Ep. 147
When I hear scientists talk about Thomas Kuhn, he sounds very reasonable. Scientists have theories that guide their work. Sometimes they run into things their theories can’t explain. Then some genius develops a new theory, and scientists are guided by that one. So the cycle repeats, knowledge gained with every step. When I hear philosophers talk about Thomas Kuhn, he sounds like a madman. There is no such thing as ground-level truth! Only theory! No objective sense-data! Only theory! No basis fo...
Jan 10, 2019•41 min•Ep. 146
This post is about the 2019 SSC Survey. If you’ve read at least one blog post here before, please take the survey if you haven’t already. Please don’t read on until you’ve taken it, since this post could bias your results. 1. Can we confirm or disconfirm different corn-eating profiles of algebraists vs. analysts ? 2. Can we replicate the study showing that people who eat more beef jerky are more likely to be hospitalized for bipolar mania? 3. Are there differences in side effects among SSRIs? (t...
Jan 06, 2019•8 min•Ep. 145
0. Introduction I grew up in the 90s, which meant watching movies about plucky children fighting Pollution Demons. Sometimes teachers would show them to us in class. None of us found that strange. We knew that when we grew up, this would be our fight: to take on the loggers and whalers and seal-clubbers who were destroying our planet and save the Earth for the next generation. What happened to that? I don’t mean the Pollution Demons: they’re still around, I think one of them runs Trump’s EPA now...
Jan 03, 2019•55 min•Ep. 144
Please take the 2019 Slate Star Codex Survey . The survey helps me learn more about SSC readers and plan community events. But it also provides me with useful informal research data for questions I’m interested it, which I then turn into interesting posts. My favorite from last year was Fight Me, Psychologists: Birth Order Effects Exist And Are Very Strong , which I think made a real contribution to individual differences psychology and which could not have happened without your cooperation. The...
Jan 01, 2019•5 min•Ep. 143
