Misinformation Machines with Gordon Pennycook – Part 2

Hello and welcome to the Behavioral Design Podcast. This season we're diving into the intersection of behavioral science and AI. We want to make sense of the state of AI, from understanding how humans interact with intelligent systems to using AI to do behavioral design itself. I'm Aline Holsworth, a health tech advisor specializing in AI and product design. Over the past 15 years, I've been crafting human centered products with behavioral science

at the core. At Apple, I LED Behavioral Science for Health AI, designing and launching AI powered features to help users reach their health goals. And I'm Samuel Sultzer, your second Co host. I'm a behavioral strategist specializing in hybrid formation and designing products that drive long term baby change. I work with leading tech organizations integrating AI to scale behavioral design for good.

And I'm also the founder of Baby Bites, a dedicated community on behavioral science and AI. Quick word on Nuance Behavior where we help organizations build impactful digital products using behavioral design. We only take on a few clients at a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our

website, nuancebehavior.com. Hey, Lynn.

Hey, I was really excited to actually share something that I realized very recently and we covered obviously in the previous episode with Gordon Pennycook. You know how to think about misinformation. And I don't know if this fully qualifies, but it was certainly something that I was not aware of and had a poor information around, which was to maybe no surprise based on our previous

conversation. But The Wizard of Oz is a film and you had a really great dress up I saw on Halloween with the theme of Wizard of Oz. And we talked about that in our Halloween themed episode as well. And I actually ended up watching Wizard of Oz. No, wow, you're now informed. Wonderful. So yeah, how did it go? It was good. It was a it was a good film. I can see why why it has

received such kind of acclaim. And yeah, I'm happy to finally say that I've now watched that film and I can even more so tell you that your costumes were amazing. So kudos to you and your family. You guys really pulled off an amazing Halloween. That means a lot.

Thank you. Yeah. So we spoke to Gordon in Part 1 episode that came out just a few days ago about the state of AI fuels misinformation and how to think about misinformation in this kind of modern context and especially as it relates to elections. And So what does the landscape look like, What are we dealing with and so on. And so with that foundation now in place, we now in today's episode in Part 2, really want to tackle what can we do about it, What can we do to combat or

manage or reduce misinformation. Yeah. And actually our our listeners might recall we have already stepped into this topic in the past. We had Lindsay Juarez on the show a while back and she talked about an intervention that she did on TikTok in partnering with TikTok to slow down the spread of misinformation there.

And their intervention really fell into this category of a nudge, as we'll as we'll get into with Gordon, where they added an accuracy prompt and then some friction to sharing. And they had some some really outsize effects. They reduce the shares of something that was sort of dubiously true by 24%. And they also reduce the likes and views by some additional percentage points. So very impressive work from Lindsay.

And we will get into maybe what you might think of as a higher level view of stopping the spread of misinformation with Gordon. Yeah. And so that episode is hidden under existential questions with her, which is final title for that episode. I forgot that we call that episode that, but we did talk about a lot of existential questions as well with her. So yeah, that's a great episode to check out.

But yeah, again, we're speaking with Gordon Pennycook and we welcome him as we feel is really important to during especially the current time makes sense of misinformation elections and how to think about that from a behavioral science perspective.

And we're really lucky to have Gordon because he's a professor of psychology at Cornell. He has extensively done many different types of studies and research around where we can go wrong in our reasoning and how we can support people in reducing the risk to either be faced with misinformation or if faced with it, how they can better make sense of it or debunk it, for example. And so again, in this episode, we go head on into what we can do about misinformation.

And So what can people look forward to? So much Sam. So first we take this high level view of different strategies for combating misinformation. So comparing and contrasting them, really, really putting Gordon on the spot to answer all of our many, many questions about these different strategies. What works, what doesn't, you know, how do we compare the, the efficacy and you know, the, the practical effect size so on. And in particular, looking at these three categories of

nudges. So like, like what we described with Lindsay boosts and educational interventions and then the third category of refutation strategies. So we were really dive in deep into those. And then of course, staying with our theme of AI this season, we take a look at the debunking chatbot that Gordon created along with his co-authors Thomas Costello and David Rand, which they call their debunk bot. And finally, we close with Gordon's most controversial opinion about.

AI actually, most importantly, we should mention that you should also expect to learn what does Grand Theft Auto teach us about misinformation? I knew you were going to go there, Happens to. Mugatroid. Wow. OK, happy to say welcome back

Gordon to the Beetles and podcast. Thanks for having me back. Yeah. And in the spirit of kind of misinformation and so on, is there anything that you said last time that you feel like you regret or if you like, could potentially have some form of fake news? No, and I I can honestly say that I don't recall what I said or thought about it after the conversation. Smart. That's that's the smart thing to say. Best way to live? Yeah, no regrets. Exactly. I live in the present.

Also, I can say that I don't regret one thing and I just want to like underline this is that I, I feel like I gave you the the short end of the stick of, of really appreciating your knowledge of bullshit. Like I knew of your knowledge of behavioral science as the BS, but also like clearly I should have really also known about how well you know about bullshit, especially around this. So I just want to again take my hat off to you for like, knowing the BS of BS and yeah.

Truly unnecessary, and I'm not sure that's a fine point of distinction anyway. So, but thank you I guess.

Cool. Last time we we spoke about understanding and the state of misinformation, and I guess today we'll speak about how we can deal with it. Yeah, let's let's jump right in. Gordon, you have written extensively and researched quite a bit on what to do about misinformation. So in one of your papers that you were one tool in the box of tools, you categorized some

different strategies. And so you and your co-authors said if we were to break down the different strategies, we might put them into three major categories. So #1 nudges, targeting behaviors, many ways of of doing this, like accuracy prompts, using friction, some labels or two boosts, and educational interventions. So these target people's competencies. Are you able to identify misinformation and know what to question and so on, like media literacy tips, that sort of

thing. And then the third category, refutation strategies. So these target beliefs, things like debunking and rebuttals, more different kinds of warnings and fact checking labels. So can you tell me, given this whole toolbox of ways to squash the spread of misinformation, but how do you sort of think of these in relation to one another? Can we compare and contrast the different categories? Great question.

One side note about that paper. So it was really Anastasia Kozariva who put together the toolbox, which was organizing a bunch of people doing research on interventions against misinformation to kind of like write out details about the different types of interventions that they've worked on and also about other people's work that they've done and so on. And then they created these kind of categories, types of interventions to help policy makers decide what to do and how

to contrast them. So yeah, so you mentioned three different kind of broad categories that have different strengths and weaknesses. So let's start with the reputational strategies, which is I think they kind of standard fair. Like when people usually are thinking about how to deal with misinformation, they're thinking about fact checking it and debunking it. And like a lot of there's a huge amount of work on that sort of stuff. And we published work on fact

checking also. And the, the kind of primary issue there is that it's really just a Band-Aid solution. Like if you're going to refute falsehoods, you have to make sure that you have the right reputation. You have to be able to get that to people who saw the falsehood, which is never really possible to catch everybody. So it's really just mitigating loss. And so there's other types of interventions that try to kind of get ahead of misinformation, build up people's business.

And there's these boosts or educational interventions are like things that give people the knowledge or the information that they need so that they will be better able to like identify or would they be less likely to share misinformation if they see it online, for example? So like you said, lateral reading and media literacy, all these types of things, that's important. That's not even inherently about

misinformation. That's just like education is a like literally just going to school and getting educated or listening to podcasts maybe is a sort of boost. It's just teaching people things. And then there's like this more kind of behavioral thing, which

is nudges, which is just trying to change something about the architecture of the way that people are engaging online so that it improves the choices. And so you mentioned friction. We came up with actually prompts. I think it might be worthwhile talking about the actually prompt thing for a second because it does sort of evade easy categorization. And we'll highlight some of these differences. So like an actually prompt is reminding people about the importance of accuracy in

various subtle ways. And so it's categorized as a nudge because it's focusing on like a targeting of behavior, like you're, you're trying to get people not to share falsehood by thinking about whether something is true or false and you're not giving them kind of any new information. So it doesn't seem that educational. Yeah, just telling them to reflect. Yeah, but you're not telling them to you just you're not saying please think about accuracy.

That's like a directive. It's more like you ask someone, hey, do you think it's important to only share accurate things and then they'll share more accurate things after that, you know what I mean? Or like it's just like an ad. We've done this with ads before where you like, we just bought a bunch of ads on Twitter that were about like reminding people about accuracy and how it's important cuz everyone agrees with that. It's not even connected to a

particular piece of content. It's just like believe in true things. Yeah. Like, do you want to share things that are true? And the people are like, yeah, that's great, I would love to share true things. But sometimes people share things that they don't like. What we found in our experiments is that people will share things without even thinking about whether they're true or false. And so it's just targeting that like mode of behavior, but it's ultimately targeting a thought

process. So the reason why that's kind of hard to categorize is because it's not changing the information. It's not like changing the way the architecture of the choice, which is a typical kind of nudge. It's just changing the way that people are thinking about how they're using the medium. So it's kind of unique it, but it ultimately has the same general bend which is just trying to target a specific behavior with a very simple sort of message.

Would you say that it kind of falls into like the priming box or how would you think about it in relation to something like that? So it's more just a reminder or yeah. That's an interesting question. Also, we debated about this because people who like priming did put it in the priming box, but it doesn't. It's not obviously priming either, because it's not like. Like it's not subtle enough to be priming. Part of that is people already care about accuracy.

It's just like something that isn't top of mind when they're kind of mindlessly scrolling on social media. And so we're just kind of like bringing something that's there up to have more importance in a decision. And so that's a lot different than like a giving someone hot coffee and making them feel warm, like psychologically, you know what I mean, These. Classic, classic. Yeah, maybe it's more of a reminder than.

Yeah. So it's essentially a reminder, you know, is a stop sign or a yield sign priming, you know what I mean? Like people might not even notice it and they'll still react to it. Yeah, it's more like a stop sign or like a yield sign that are just like a priming. So now we sort of have these general categories. How would you compare them

against each other in terms of efficacy? Do some of these strategies work better than others? You. Know what's funny about this is that so there was this big mega study that was done. I'm not sure if you're familiar with it, but what happened was so people have been doing all this research on different types of interventions, but they always use different outcomes and different stimuli like different measures.

And so it's very hard to know based on the literature whether like a warning label is as effective as a lateral reading intervention, you know, and like also these are such Apple and the origins, it's hard to know. And so there was this big mega study that was we had a bunch of people involved. Everyone had different interventions, but the same stimuli, the same material, and usually the same sort of outcomes. And they were all basically the

same. Everything worked about the same, like decently well, you know. Yeah. And what is decently well, can you translate that into something that something like a practical measure that everyone might understand? No, I can't, sorry. The problem is this. So we run an experiment and what happens is in these studies, there's for example, we, we create a set of stimuli which is like true and false news headlines, OK.

And often in these studies, because we want a good estimate of both, whether people are like believing false things less or true things more, we have like an equal amount of true and false things. And the false things are like fact checked as false things. And so we can say as researchers that these are things that definitely fit into that category. In that set of experiments, we also had things that were categorized as misleading, but they're often also very clearly misleading.

And so that's like you have to quit these clear categories. And then you might find that like, say for the action prompt thing, sharing a misinformation is like 20% lower. And then you say, OK, well, then if I put that out in the world, then people will share 20% less false things. But like, you don't know that because not only did we not run it in the world, we ran it with this balanced set of true and false things that were a particular type of false

category. And so I could give you some answer of like, but it's, it's not, it's not going to be legitimate. It's like we know that the psychology behind it is legitimate. We know that these things have an effect when scaled. We don't really know what the effect would be.

We think that there's reasonable expectation that they would have some effect, but it really depends on what people are engaging with, what their information environment is like, how much attention they're paying to the intervention. You know, like prior exposure to these things. It's hard to say. Right. And you're also doing this outside of the context of, of

recommender systems, right? Where in the real world you might be on a social media platform that, that gives you more misinformation, the more misinformation that you view and your peers are viewing misinformation and, and so on. And that might magnify the, the, you know, the effect more than these other strategies of countering the misinformation.

Yeah, although I think it actually in many respects it's the opposite, like in the sense because we give people the feed that they would get essentially in one of these studies, because we want a good estimate of both belief in false and true stuff is like equal true and false things. But that's not what people see. Like, you don't imagine if half of your feed was misinformation. That would be insane.

Like that's a lot. Now something there are some people who do see that much falsehood, but it's a small fraction of the of people. Most people see generally mainstream fine things, unless you're at a rally I guess, but that's not typical. What is typical? Could Do you know what percentage? So again, this is really difficult because it depends on how you're defining whether something is true or false.

So if you, if we define false things as like things that are fact checked as being false, then like most people see like less than 5% of the stuff is of news content is fact checked as false for like the vast majority of people. It's like a small fraction. But there's like stuff that you might categorize as misleading and that's a major category. It's hard to define whether something is misleading.

So it's harder. And then there's like the sort of thing that's not news content, that's false, like something a politician might say about cats and dogs or whatever. And then a lot of people see that if someone, if someone tells you that they can quantify the amount of truth and falsehood that an individual sees, they are themselves saying a falsehood. Like if this is something that we we can't, it's not easy to find whether something's true or

false anyways. And then to like try to capture the amount of information that people are engaging with. It's just a gigantic. So it's really hard to know, but there's I could give like an explicit example of why this matters in the context of interventions, if that would be OK. I know I've had a long point of just me talking, but you know, we're, we'll break podcast rules for a second because I like ranting about this thing.

So, OK, so let's take, let's talk about psychological inoculation for a second, a particular kind of thing that I find interesting. So the idea behind psychological inoculation, this is one of the boost for educational interventions that you mentioned. It's pre bunking, it's not a way of saying it. Well, inoculation is a subcategory pre bunking.

So pre bunking is you could tell somebody before the election, which is presently what we're talking, you might say, well, this is the sort of falsehood that you're going to see. And certainly people are going to see things like ballot boxes being stolen and like ballots being thrown in the garbage and that kind of stuff. We can sort of know what sort of falsehoods spread during the election. And so you can say, well, just be mindful.

This is something that people have lied about in the past and it works really well. Like people, that makes sense. People can like take that specific information and apply it to the stuff that they see. Inoculation is a little bit different. It tries to go one step broader than that. It says here are the sort of tactics that people use when they create misinformation. So it might be something like they'll use emotional language to try to grab your attention, OK?

They'll say it's disgusting or whatever. And this is like a common tactic in the misinformation. And so there's like a bunch of research on like, if you teach people about those sorts of tactics and then you give them stimuli that contain those tactics directly afterwards, they'll be like, Oh yeah, that contains it. And so they can learn the thing that you taught them in the

inoculation thing. The question then is, does that mean that if you give someone this inoculation, this like set of information about avoiding emotional language, how will that impact their everyday kind of detection of that thing? And that's a very different question. And part of the problem there is that like things that are true that are not misinformation also use emotional language. You know, sometimes things are disgusting actually, like there

are disgusting things. And so if I say, if I see the word disgust, that does not mean that things are false. And we've done some research on this, like if you give people the inoculation, it doesn't help them distinguish between us true and false. It just helps them be more wary

of emotional words. And if it happens to be the case that people see way more true things than false things, then probably what they're doing is if it insofar as working in the real world, they're believing truth true things less because of the inoculation. And they aren't believing false things less necessarily because they're not seeing that much false things, they're seeing way more true things.

Do you see what I mean? So like how it's applied depends on the information environment that people are engaging with. So it's hard to say in these experiments. There's a bunch of studies about inoculation being effective. What that means is people can learn the inoculations. We don't know what that means. They are using that appropriately in the world because those experiments

haven't been. Run. Yeah. But it it makes sense because like we're still fallible to something like, let's say motivated reasoning in terms of like even if you taught someone, for example, what ad hominem attacks are and and how they work in terms of political kind of debates, often times can lead to people kind of attacking individual character rather than

the policy they have. Even if they know that they're probably much more likely to notice if someone is using kind of ad hominem attacks against the politician they like rather than maybe if the politician that they like using some against. Like, I guess that's kind of what you're speaking to right as well.

It's like it's just. Yeah, I mean, actually that's that's an additional issue in the sense that like if the world is noisy anytime you teach anybody anything, it's going to be impossible for them to adopt it with perfect accuracy.

And that's just a general issue. What I'm saying that even if you did have perfect accuracy in the context of some of these inoculations like emotional language, it would still lead to people believing through things less and then they are believing false things less because they see more true things that have

emotional language. Like I think about this way, if I had a magic wand and I could inoculate people against emotional language to say every time you see emotional language, don't believe that thing, which is what essentially these things are trying to do. The next thing that they see that has emotional language is more likely to be true than false. Right, Yeah, it's when you're teaching it, you're sort of ignoring the base rate and so. Like it may be a great neglect, yeah.

Exactly. And then do you say, OK, well then you have to teach them to look out for emotional language in so far as there is falsehood also included. But now you're back to prebunking and you're not eoculating anymore. It's about teaching people actual facts about the world. And that's just regular prebunking. And that's fine.

But it's basically the the conceit is this, it is not possible to, I don't think to find simple markers that you can teach someone in a minute and 1/2 that will help them distinguish between what's true and false. The world is too complicated for that. And so we need to actually just teach people about facts.

And the educational interventions that are more directed towards the things that help us identify true versus false things are the ones that are going to be most effective when it comes to misinformation at least. But one side note is that you could make the claim that it's good to teach people about the tactics regardless of whether this misinformation evolves. Like, we want people to be able to identify ad homom attacks. That's fine. It's just not intervention against misinformation.

It's intervention against ad homom attacks. And that's a different thing. That's more nuanced argument than that. But yeah, sorry, a bit of a digression, but it's a topic I find interesting. Yeah. No, it should be interesting. And I think that's why we have this conversation is kind of getting some of the weeds and it's always is the case with payable science stuff that it depends. And I think that is also, you know, appreciated that you're like not trying to oversimplify things here.

So I, I think that's good. And I guess when it comes to then thinking about, you know, again, it depends. We have 3 categories, many different types of interventions you could do. Is there any advice you would have or any thinking you would have around kind of like how if someone is trying to do work in in this realm where they can start thinking about like, OK, all of these various types of things we could do, where should we start depending on where we

are? Yeah. I want to add to that as well because I think one of the the mantras of behavioral design and especially for practitioners is like, you know, we don't want to focus on changing beliefs. We want to focus on changing behavior. That's what counts, that's what matters. And so we often push people towards these strategies that change the behavior. So of these three categories,

that would be the nudges. But now you're telling me these are equally effective across the board and you know, with the nuance and the caveats, of course. But what would you say to that sort of general rule of thumb? Does it apply here? I think that's more of a choice about what you want. That's less about efficacy I would assume and more about like how you want to be engaging with. I mean, it really depends.

So if you take debunking, so if I was able to, in theory every person who sees the falsehood gets a debunk, then that will be effective to some extent. But of course, in reality, I'm not going to be able to do that. If I do a nudge, I don't need to know what the person saw for that to have efficacy.

Now. It's only going to be effective insofar as they see things like if if I give someone a nudge that will get them think more about accuracy, that's not going to have an impact if no one ever sees falsehood, but that's fine because it doesn't. You have to waste anything. That's fine. There's nothing wrong with that. If I'm debunking something, if I debunk it but the person hasn't seen it, then like that's fine too.

But what strategy you use depends on what you're trying to do. And there is no like, oh, this category is better than that category. It's just like they're different things and people like the reputation, like the ones that target beliefs in particular. Some people would want to avoid that because they don't want to be the ones who say this is what's true and this is what's

false. But if you're nudging you, you don't have to be the one who decides that you just like, here's good information about better sources, you know, there you go. And then people can take it or leave it and whatever. Do you see what I mean? OK, yeah. And and so where do our sort of sticky topics of freedom of speech and the censorship and all these sorts of things, how do they play into these different strategies? Are some I don't know closer to infringing on that right than others?

I guess so, although I mean on a social media platform, there is always filtering, there is no, it's not freedom of speech in the sense that that's freedom of reach, I guess. And different people have different platforms and there is an algorithm and that is decided by people and then computers also or whatever. But like there's choices that are put into these things that determine what you see.

And so if you were to say the social media company decides we're going to wait a source quality wait for news content on the platform, that's an editorial choice that does not impact freedom of speech. It's going to impact what people see you can. Say whatever you want there, it just might not be seen by anyone else.

Yeah, exactly. And it and if you have a source that is blacklisted by the social media company for whatever reason, like you're, you're free, it's still free to say whatever you want, but you don't have the opportunity to broadcast what you want to say to that company's proprietary, you know, to the people who are on the thing. So, you know, that's the complication with the free

speech issue. And then you always, but I mean, it is the case though that social media companies do are kind of really concerned about that or these perceptions of freedom of speech. And so they don't, I think they're just afraid of basically getting a New York Times article written that they're trying to target conservatives and stuff like that, which is usually what's happening. And they often overcorrect. And that's a bigger issue.

But yeah, I mean, the other thing is when it comes to fact checking and warning labels and source credibility labels and all this type of stuff, someone does have to make a choice about whether something is fact X is false, whether a source is credible or whatever. And often these are like you'll rely on Fact Check professional fact checkers, journalists who are kind of experts in the domain. But you know, a lot of people don't trust that.

And these are not simple issues to to deal with. Yeah. Well, one other lens to look at this that I usually use for various types of kind of interventions is to think about it as either focusing on, call it, proactive strategies versus called reactive strategies. And I think it applies relatively well here as well where you know some of the things you have kind of talked about fits in the box of things that you might do practically.

Like certainly the pre banking is a good example of that where you're proactively getting ahead of things to see how you can do something to increase people's ability to distinguish maybe fake versus true and so on. And there's very various other things around probably boosting that would also qualify there in

general. But then obviously as we're talking about, like there's this thing where in the moment people will be exposed to various things and it will be requiring more like quality reactive strategies where they will be very keen to share something. And then maybe that fits into the box of maybe I'm trying to think about which box would fit where you get an alert saying like, hey, are you really sure you want to share this? Have you read it?

Like I don't know what what that box would fit in. Yeah, I don't think that counts as a nudge sort of friction. Yeah. It's kind of like a the targets of behavior, at least, yeah. Yeah. So that's kind of like a more reactive strategy. So I'm just interested, do you have any sense of, you know, all else being equal, like where would you usually think about, you know, attacking or like is it both equally important? Yeah, I think they're both equally.

I mean, yeah, I think, I mean, essentially they're both equally important, or at least it's not important to know whether 1 is better than the other. It's kind of like saying this is a toolbox. So like they're just for different things. Like what's better, a hammer or a screwdriver? It's like it depends what you need it for. Probably a hammer is. I mean, if we're going to have the argument, I think probably hammers generally have a broader usefulness than a screwdriver,

but it doesn't really matter. Like it just if you have a screw then you need a screwdriver. You can always hammer in a screw. Yeah, you could. Eventually hammer it in. If you're really. Desperate. But it's a purely kind of academic discussion, right? So yeah, they're just trying to do different things. And so it's good.

This is why it's a toolbox. It's like depends what the person's trying to do. You can pull in the toolbox and see what is like, what are some ways in which you can deal with that particular issue, which is like if it's a for debunking, for example, it might be there's a kind of like common falsehood that people are believing or something that's being spread. And so it's too late to do prebunking.

You have to do debunking. You have to kind of try to catch up. But if you're like right before an election, there's going to be certainly a bunch of like false claims about election fraud. Yeah, now now's the time to do some pre bunking and try to make sure that people know what good, what good sources are and bad sources are and so on.

And so like these. Yeah, it depends on what you're trying to do. Yeah, and we can shout out the also website that exists for this toolbox, really useful for anyone who has checked out. It offers a lot of chances to look at the various tools and also papers attached to them. So shout out to that website. But one thing I just wanted to touch upon as well because we we haven't directly explored it and

I don't currently see how maybe these tools as well maybe addresses this as you study of echo chambers in terms of with the Internet increasingly being fragmented, there's a bunch of small subreddits and WhatsApp groups and various things where, you know, honestly, it seems like misinformation thrives more than ever in those kind of cases. Well, how do you think about dealing with that in terms of maybe from this toolbox in general?

Like how do you think about that challenge with misinformation online? Yeah. It's an interesting question. I'm not sure the toolbox is all that helpful for echo chambers. The echo chambers is a structural problem. It's about the way the networks are formed. And that's the kind of a social

media problem. I mean although I guess in theory if you are like really successful in altering people's beliefs for example, they might remove themselves from an echo chamber because they're like wait this is all junk. Like why am I hanging out on the subreddit? It's worth knowing that like online echo chambers are not as bad as people think they are. That is, our real life echo chambers tend to be worse than that.

If you think about just homophily in neighborhoods and so on, you actually are exposed to way more diversion opinion online than you would be otherwise. But you know, it still is the case that people find if you have a particular kind of set of fringe beliefs, you can find a community that reinforces those things that you would not find in your real in your like day-to-day life.

And that's a major problem. But that's kind of more of a fringe issue, but still is a it is an issue among people on the fringe, a smaller proportion, but a very large issue because even a few small people who are in like, for example, like violent, you know, groups of individuals is not a good thing, obviously. So 1 aspect of this toolbox is

that all the strategies, I think you could argue are focused on the individual interventions that really get the individual to stop and think, do something differently, believe something differently. How do you think those sorts of individual focused interventions compare against what you might call your more standard policy toolkit? So things like, you know, regulating information, punishing the perpetrators, right? How do you think of those things together?

Yeah, I would think that they don't fare well compared to them to policy meaning that like. So I think it's important to try to improve individual level decision making, obviously, and that, you know, regardless of whatever the the other, the problem is that I can't influence thought. I'm not a policy maker. I but I am a psychologist and so I can do research on like individual level interventions,

but I'll give like a example. Like we can spend a lot of time working out like the best way to remind people think about accuracy. And this is going to have a tiny fraction of effect relative to like a social media company putting more emphasis on showing people stuff that's accurate. You know what I mean? So you know. You actually get to the the root of the problem.

Yeah, exactly. In that case, I don't own social media company, so I I can do what I can do so that, you know, 11 important note is that tech companies, social media companies do like the individual interventions because it means we'll let the people solve it. We're not going to solve it ourselves. The users will solve it here user. Here's some here's some ways to deal with the shit that we're showing you on our platform. You know, good luck with that. You know, so that's an issue.

But so we're in some sense, we're all kind of useful as idiots for the social media companies, but we like, what else am I going to do? I have to figure out something. We have to find ways to make things better. So hopefully that doesn't take pressure off the social media companies to actually make improvements on the platforms.

Well, OK, so you said you don't own a social media company. But but let's say that you did. Imagine that Elon Musk hands X over to you. It's yours to overhaul. You can redesign it entirely. It still has to be a social media platform where information is shared. What would you do if you had this infinite power to basically start from scratch with social media? How would you design it differently if you would decide

it differently? Yeah, I mean, so most of my recommendations would be to hire way more content moderators and trust and safety people. And like all of the people that were fired, for example, by Elon Musk, you know, I bring the experts back in to help make better informed choices on this. Sort of basically what you need is better filtering of quality. And this does not need to be misaligned with people's

preferences. It's not like I want to determine what the information environment is for people and be a big sensor and determine what's true or false that most people don't want to see junk. Like, as an example, clickbait. People actually hate clickbait. It works. People will engage with it and that really does like push forward on algorithms, but people don't want it.

So you can make principal choice as someone who owns a social media company to say we're not going to deal with that kind of stuff. And there's like, you know, Twitter's got all sorts of spam and everyone has noticed the quality kind of decrease. Yeah. There's just levers you can pull to make that better in terms of,

like, how to change algorithms and, you know, maybe find ways to incentivize accuracy more. Like, I, I there's things that I would change, but probably they would also people would stop using the platform. And I'm not sure if I would be the right person to make those sorts of choices. So I mean, the first thing I would do is just simply hire more people who are experts in contact quality essentially and try to improve the stuff that people are seeing.

Would you let users choose what percentage of true content they were shown so you could get like a, you know, hey user from 0% to 100%, how much misinformation would you like? That's interesting. The most users would be people who say I don't agree with you but what's misinformation so that everyone wants 100% true stuff but we just. Can't agree on what the truth is. Exactly who gets to determine what's what's true and what

isn't? So I mean, one thing that is worth noting, community notes on Twitter, which is like, you know, a kind of consensus built almost kind of democratic Wikipedia esque fact checking people like that. I mean, there's not very, you don't really see that much dispute about the notes. They're always quite good and accurate. And so you could scale that up. I think it could be scaled up more, but that's a success story

I think. I think there's a few things that creates as much Glee and Freud and Freud as someone re sharing community notes. I think there's even like one of the most popular accounts is like destroyed by community notes or something. Yeah, yeah. And they'll put community notes on like Musk's his own tweets and stuff like that. And that's always fun. So yeah, that kind of stuff is great. Well, I want to know, what would you guys do? You're not allowed to say that. No, we can.

Yeah, I I am a big fan as well of in some ways giving kind of

power to the people in terms of I would say that my platform of choice is probably Reddit these days because I have an opportunity to both attach myself to certain subreddits that have an interesting. But it's also kind of creates quite a lot of variability at the same time. But importantly, it kind of automatically has an inbuilt system where anything that is spam or click based and all these things is usually downloaded.

So you never see it. And the things that is actually inside full and useful is usually uploaded. And so it kind of. Punishes people that are trying to spread misinformation kind of purposefully harmful because it's almost like a virus, like how you say like antibodies where the kind of platform or the community itself notices, OK, this person is just trying to be a troll and this community, we're against trolling. And so we're going to downvote this stuff.

And I think also on Reddit is where you've seen things spawn like this community for having kind of prove me wrong, basically like certain communities where they're trying to have ways to like, hey, prove me wrong, where that kind of has worked. Like that's one of the few cases where I can see people thoughtfully engaged, become more insightful around certain topics. It's not true for all subreddits. There's some shit shows subreddits for sure.

But I I I do like some of those aspects with Reddits that you enable people to also be part of the solution. Yeah, that's great. And I know that, like, the operators of subreddits do a lot of work to, you know, enforce dorms and stuff like that. There's some really interesting work on that. Nathan Matthias, who's my colleague at Cornell, does work on Reddit.

Yeah. OK, I think he is a moderator on a subreddit or he's studying it. He's studying it like he works with moderators to do. Like it's some, it's almost ethnographic stuff, but like, yeah, it's all about creating trust within communities and so on. Yeah, that's cool. And I think that's something that I find fascinating where there's some extreme version that's where people have some form of a special server version of Grand Theft Auto where people can have real jobs.

Or like there's also a Second Life and some of these games where people spend a lot of time. Same thing with Wikipedia. Like, you know, some of the people that contribute to Wikipedia, there's amazing what people are willing to kind of do on the free time to protect the community or be part of something and so on. So if you can harness that for good, I think that's amazing. But yeah. Is this the first time that Sam has mentioned DTA on the podcast? It is. I think it is, yeah.

Finally, yeah. Wow. I'm glad I'm here for it. Amazing.

So there's one strategy that I think really can help with the scaling problems. So maybe help with not having to hire all the fact checkers back, but some of them. And this is a strategy that was developed by Bill Adair. So he's the creator of Politi Fact. And So what one thing that they've developed or are developing is what he's calling the half baked pizza.

And so they have a just some generative AI take a first pass at content and say, like, we have a pretty good idea that this is, you know, flaming pants on fire, very untrue. Or, you know, wherever on the scale it falls. And then everything has to have a human review passed to like really make the final decision. But it already does a lot of the work for them to sort of, I don't know, scale it, make it easier, that sort of thing. Yeah, that makes sense.

I mean, adding these tools, I need the person in the loop, but that's yeah, we need to be able to leverage actual new technology to make things better also. Oh, that seems like a nice segue. Have you heard of anyone leveraging new technology like generative AI to combat misinformation?

Oh my God, it's funny that you mentioned that because I also have done that. Oh yeah, and also and well, so my my favorite GTA game is oh, sorry, I'm just joking. Sound got excited. Oh no. So we have a recent paper in science that we use AI to have evidence based dialogues with people who believe conspiracies. OK. And the kind of background for this is everyone has this idea of conspiracy theorists as being like beyond help when it comes to evidence.

Like they're down the rabbit hole and there's like no way they're coming back out of it essentially. But the same time, if you think about how we do these sort of experiments, like if you want to like, say, debunk a false or unsubstantiated conspiracy, you have to kind of make guesses about what people believe and then make guesses about what information would be the most useful that would convince them

not that thing is not true. And but usually the people who are making guesses are not the people who believe it, you know what I mean? And they have no, and they're like, maybe ivory tower academics or something that are like trying to figure that out. And so maybe we don't know if we're good at guessing, but we don't need to guess anymore because we can, as I said, leverage new technologies to

make these things better. And so we did is we create an experiment where people come in, they tell us about a conspiracy that they believe in. Now, we don't say like, what conspiracy do you believe in? We kind of like we define the term a secret plot with malevolent actors and so on. And then we we say some people call these conspiracy theories with like air quotes. Well, I mean, it was written text was like literal quotations. I guess it's a podcast. I'm making air quotes.

That's what you can see. And then so they say a conspiracy in their own words and they tell us why they believe it. And then we give that to the AI and the AI is being told, we kind of prompted to persuade them not to believe it using evidence. And then what happens is they have this evidence based conversation where the person will say make some claims, the AI will come back with like every claim refuted with details in like sites and stuff like that.

And then the person will say, well, what about this? And they don't refute that. And they do that like three times. The conversation last like 8 minutes. And the AI is really good at like, counteracting most conspiracies that we saw people talk about. And what we found is people

after the conversation believed the conspiracy. Like they decreased their certainty by 20%. Like fully 1/4 of the people who believed the conspiracy did not believe it after the conversation. They changed their mind. Like I said, it was like 8 1/2 minutes long. 20% said this isn't true at all. 25 percent, 25% said they believed it before the conversation. They did not believe it after the conversation. So 1/4 of them changed their mind after the conversation.

No one I've told this research about has failed to be surprised by that. We did not predict it. Also, people were surprisingly responsive to evidence. And that's because this is the best, like the strongest test of actual, like whether evidence can change someone's mind because we've given them better evidence than they've gotten. Because we used generative AI to do that. And it's really effective at that. Yeah, it's really interesting finding.

And I can actually say I had a fun moment with a group of friends the other week where we were grabbing a beer after work. And one of my friends works in AI. He's like, hey, did you know? And then he basically talks about these findings and I'm like, well, I'm speaking to the guy who's behind this on our podcast. And I got a lot of cool points from my friends because they're. Like oh wow, right on. So thanks. Thanks for that. I appreciate it.

Yeah, it was almost like the time that you wrapped GTA. I'm not going to bring it up again, I'm sorry. Wow, that's a thermal trick all. That almost almost the time. But also I think interesting here is that it's persisted right? Like this leaves seems to

persist. We asked them again 10 days later, and two months later, not only was the effect still there and had not decayed, meaning that people were at the same level of decreased belief in the conspiracy overall two months later as they were directly after the conversation, which means they really did change their minds. Like it wasn't just like a reactant, sort of like, oh, yeah, I guess you're right. And then they like went back to where they were. The people were like, you made a

good argument. And if you read the conversations, which you can access if you go to the paper, people were like, that was great. In fact, we find some studies that people trust AI more after the conversation. You know, they find it interesting. We did a bunch of follow-ups too where we like told people explicitly the AI is going to try to persuade you not to believe it. We get the same effect. We tell people you should debate the AI, convince the AI that it's wrong.

You still get the same effect. The only thing that stops the effect from happening is if you tell the AI, don't give evidence, just try to persuade them, but don't provide like specific counter evidence. And what happens is what they'll say is like, oh, this is really harmful. This is something harmful to believe, or it makes them like appeals to the empathy or whatever. That doesn't work. You have to give people specific

evidence. Is that the problem with humans when we try and persuade our relatives that their conspiracy beliefs are crazy is. That is that what we're doing wrong? I know what we're doing wrong. Well, maybe, But I think what we're doing wrong is that we don't have the evidence. Like we don't have enough access to. We don't have access in the moment to all of the Internet to try to like find compelling counter arguments to why the moon landing was a hoax.

And also the thing about conspiracies, I mean, the people's beliefs are so idiosyncratic about each individual conspiracy. But like the specific like things that they might have heard about. It's like the Gish gallop, you know, we're like, you have an argument with somebody who believes this fringe thing and they just jump from argument to argument. And then it's so hard to refute each one.

It's no problem for the AI. They'll just, if you can see it like a dozen things and it's going to give you a dozen responses within like a few seconds. It's really remarkable to see actually. It's such, so you can check it out yourself if you want to. It's on debunkbot.com. So you can go and like do the experiment yourself. I mean, a lot of people who are doing it don't like actually believe in a conspiracy. So they're kind of like pretending and then seeing what

it would say. If you could do with somebody who actually believes a conspiracy and then see what they think, you'll see how useful it couldn't be. Yeah. It's perfect timing ahead of Thanksgiving and so on for anyone who's like, you know, gearing up towards like, not wanting to have certain arguments about certain things. Yeah, I mean, what you could do is just literally load it up on your phone and be like, here, listen, I don't know anything about this, but hey, I've heard

about this thing. Hey, you want to like answer what this says and I'm curious what it's going to say and then try to get see what it'll produce interesting responses and maybe that'll spark conversation, who knows.

Yeah, and it's also interesting. I've been working on this piece or writing this piece around kind of like there's certain things that we take for granted that we obviously understand that AI is is really good at like information processing or maybe memory in certain regards and so on.

But there are other things that we provided a lot of virtue to humans to like in terms of what we see as really good human traits that AI is like superhuman, You know, they're kind of like superhumanly patient. We talked about patients being a virtue and it is regarding what you speak to is that they have like an AI will never not be patient. They were just like, you can tell it a million stupid stuff and it'd be like, oh, that's so

interesting. And it does that in these conversations, it's always like, oh, that's really interesting and whatever. Or like, I can see why you might think that, you know, which is just total fluff in these conversations. It doesn't like, as far as we can tell it, that part doesn't play a gigantic role.

It doesn't hurt certainly. But also like the other thing that's interesting about it is that like somebody could could admit to an AI that they're wrong, which they might not, they probably would not be able to do to a relative that they've been debating about things forever. There's no social costs, yeah. There's no social costs. There's a lot of, there's a lot of benefit to it.

Now, one thing I should say though is that we used it specifically here to try to provide people good information that's accurate about conspiracies. And we did Fact Check it. We had a fact checker look at the claims, a subset of the claims from the studies and it was 99% accurate. And so this was not a situation where the AI has difficulty finding information that's good. That doesn't mean that's always going to be the case. But in the context of like popular conspiracies, it's

pretty good. But I mean, it is possible for

someone to create an AI that would convince people to believe things that are false, too. I'm not sure that it would be as effective necessarily. Like if you think about it's easier to convince someone that the Earth is not flat than convincing them that is flat because you can imagine how hard it would be to construct a good argument for something that's so kind of timely false. But in less fringe cases, you can imagine this being used for bad purposes also.

So that's a concern, but just something we focused on. And I think more research is required on that. We have to kind of like, cuz since we didn't like build the bomb, like as psychologists we have to kind of figure out how big the radius is. So yeah, that's another element of it as well. Because I wanted to ask if you had any like pet conspiracy theory that you were really keen to debunk you through the bunk part yourself or that you learned that was wrong through the bunk part.

Have you done some self experimentation? Yeah, I did. It was the Epstein conspiracy, not killing himself. That one seems like I'm like, well, I don't know, it's a little fishy, don't you think? But I hadn't like looked up things like, I didn't really have any reasons. I just was like, I'm not so sure about that. And then it was pretty good at it, like gave pretty good information about it. I don't actually remember the details of it.

You have to look at yourself because I don't actually care or whatever. But it was that was one where I was like, I'm sure about. I need more information. That was really good at providing that information. Yeah. I mean, the one thing that's worth noting is that because it so just because it's a conspiracy doesn't mean it's false. Like there are some conspiracies do happen there.

Like people can conspire. Watergate was a conspiracy, and there were a small number of cases where people did have a conversation about actual conspiracies. One example is MK Ultra. Are you familiar with this? This was, Yeah, the American government was giving people drugs. Crazy, crazy. It's a really crazy true story. So look up MK Ultra. Yeah, it's hallucinating drugs. The LLC they're giving people that was discussing, yeah, 1.2% of people have that

conversation. It did not decrease people's belief in that. So it wasn't like convincing people that true conspiracies were false. So that's that's what. You would hope for. That's what you'd hope for yeah. So it yeah, but it wasn't built to do that. Like if you could build an AI that would could maybe like it wasn't cuz it's not gonna make things up cuz we used GPT 4 turbo and it just won't generate falsehoods. Yeah, at least it on purpose.

It wouldn't. But you did say that according to the fact checkers, it was 99% accurate. So what was what were the conspiracies that fall into the 1% were inaccurate? This is at the level of the claim. So like one specific thing it said was not. It didn't say that it was false. The fact checker said it was misleading. I can't remember what it was, but it was also like, I'm not sure that I would classify that as misleading. It was kind of really nitpicky, misleading things.

So that was it. It didn't say anything obviously false. Or problem is the fact checker. Is that what you're saying? Yeah, exactly. Yeah, Fact checker is also. It wasn't. Human error, the 1% error. Yeah, OK, OK. Yeah, it didn't hallucinate anything. It didn't fabricate sources or like create information that doesn't exist, but like it could present arguments that were made by people that seemed credible but that maybe a particular individual would consider misleading.

So controversial opinion. Let's talk about that because that's our final kind of question of this system is to ask everyone who's a guest basically when it comes to AI if to have certain controversial opinion or like what is your most controversial opinion about AI? Yeah. And and bonus points if it's an actual conspiracy theory that you hold.

My most controversial opinion about AI is that I don't think it's going to lead to the end of everything. Is that a controversial opinion? I'm a skeptic about AGI. Like I think that already we're seeing a kind of plateau of the capacities of AI. And like now the amount and the length and the energy requirements for like subsequent trainings is also becoming beyond what is feasible or reasonable. I think what we have now is if we look back 10 years, we're going to be dealing with pretty

similar AI agents. That's my prediction. We'll look back in 10 years and I'll be hilariously wrong on this and I've been wrong in the past. So, well, I guess it's also interesting because that can be true, but can also be true like that the current systems are maybe more powerful than people like currently can really understand, and that it can still 'cause some harm without it becoming a guy. Or how do you think about that?

Part I think that the harm that you know, AI can do and probably will do will be specifically because humans are using what is available for nefarious purposes and not because of some you know I robot situation where the AI gets consciousness and starts controlling everything. You know what I mean? Or, you know, like using it to, I mean, it's already sort of happening. People use artificial intelligence to make it easier to create misinformation.

But we're also like, as Elaine mentioned, we can use it to like help debunk falsehoods. And so like, it's a tool that people use and like, it can be used for good or bad, but like ultimately the our demise will be up to us.

One more So it you're talking about the relative harm versus good produced by AI in terms of misinformation specifically, where do you think those sort of fall if you take all the good and all of the bad created by AI in the domain, specifically of misinformation, like how much good versus how much bad is AI

doing now? That's it, because I want to answer it by expanding the like misinformation because like I think it's definitely more good in the sense that a lot of people use it for things that are helpful, that relate to information like. OK, yeah, in the in the space of information, let's say. Yeah, yeah. And like it's like it's a good information source. Like perplexity is an AI that's like a, it's a really good

search engine people use that. It's like, and they aren't at present built to create misinformation. Like most of the AI that people are using are have lots of good filters and generally they're producing decent information. They don't hallucinate a gigantic right? So therefore the balance is good. And also it's not difficult to make up shit. Like it's not we don't really need AI to create falsehoods. I mean, it might create the amount, but like that also has

not really been an issue. Like it's really easy to make up misinformation and just like create a website and like stuff will get spread if it hits. So you don't really need AI for that. And I don't know that it's really helping. Like there's not like if you think about it, the the primary things that like the that you would consider misinformation. Since AI became a big thing, this has not changed. Like, it's not like it's. Deepfakes I think are are maybe uniquely.

Yeah, but they're not. But like, there's not very many cases of deepfakes really having a consequence. And part of that's because people can identify, but it's not, you know, in the grand scheme of things. There's been a lot of concern about deepfakes that has not really panned out, I don't think. I think it's much more concerning to be part of some for a second that people with gigantic platforms, yeah, you know, say explicit falsehoods repeatedly over and over again.

And, you know, that doesn't seem to matter that much. That's not an AI problem. That's not a big problem. It's just, yeah, a big. Problem that doesn't that doesn't matter in the sense that people aren't worried about it or people aren't doing something about it or that. It doesn't matter. It doesn't matter that spewing obvious falsehoods make someone unfit for office or changes their capacity to be elected in a country that is one of the most powerful in the world. That's a problem.

Yeah, and that's the topic for a different podcast. Yeah. Well, it's timely. It's certainly timely. But for now, I I would say this was honestly really a gift. These two episodes we've done with you. We really appreciate you taking the time, obviously, but also, you know, all of the great work you've been doing in this field and everyone should go to the bunk bots to to try it out themselves as well. And obviously, again, we'll link to the toolbox and yeah, all the

great things they're doing. So really appreciate it. Thanks for the time. My great pleasure, thanks for having me. And that's a wrap. You've been listening to the Behavioral Design Podcast, brought to you by Habit Weekly and Nuance Behavior. Sam and Alene tell me This season is packed with incredible insights about behavioral design and AI, so be sure to subscribe and share the podcast with your friends. Though you might want to keep it away from your enemies.

In case you haven't noticed, I'm an AI voice. Yep, pretty crazy. Quite the improvement since last season's AI outro, don't you think? If you'd like to collaborate with us at Nuance Behavior, where we use behavioral design to craft digital products with Nuance, e-mail us at hello@nuancebehavior.com or book a call directly on our website, nuancebehavior.com.

A special thanks to the amazing Dave Pizarro for our show music and to Mei Chen Yap and April English for their help in producing and publishing this episode. Thanks again for tuning in. We'll be back soon with another exciting conversation where behavioral design and AI intersect. Oh. Is this the first time that Sam has mentioned DTA on the podcast? It is. I think it is. Yeah, finally. Wow. I'm glad I'm here for it. Amazing. Are you DTA player?

No, no, we're not going there. Guys, guys, guys. We've got a timeline here. A few moments later. Putting aside a discussion on potential risks of what could cause at the end of the world and so on, I guess the most important question I think we should come back to as a final thing, Grand Theft Auto. Yeah, yeah, there. We go I'd actually I I've played Grand Theft Auto. I'm not that big into Grand Theft Auto, but I do think it was funny that you mentioned it. It's a good game.

And finally, when will they release the next one? What's going on, Sam? I'll episode for another podcast. We definitely need to bring you back and talk about that.