Behavior Change Algorithms with Susan Murphy

Hello and welcome to the Behavioral Design Podcast. This season we're diving into the intersection of behavioral science and AI. We want to make sense of the state of AI, from understanding how humans interact with intelligent systems to using AI to do behavioral design itself. I'm Aline Holsworth, a health tech advisor specializing in AI and product design. Over the past 15 years, I've been crafting human centered products with behavioral science

at the core. At Apple, I LED Behavioral Science for Health AI, designing and launching AI powered features to help users reach their health goals. And I'm Samuel Sultzer, your second Co host. I'm a behavioral strategist specializing in hybrid formation and designing products that drive long term baby change. I work with leading tech organizations integrating AI to scale behavioral design for good.

And I'm also the founder of Baby Bites, a dedicated community on behavioral science and AI. Quick word on Nuance Behavior where we help organizations build impactful digital products using behavioral design. We only take on a few clients at

a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our website, nuancebehavior.com. So Eileen, I want to ask you something. How often do you speak with an AI on a daily basis? Like how often are you conversing with an AI? Conversing as in with my voice or not at all? I have not done that yet actually. That's a lie.

That's so funny all the time. Oh my gosh. I'm thinking of these really innovative new versions of like ChatGPT voice. I have Siri set a timer for my coffee every single morning. I ask Alexa to play music constantly. Yeah, I know. My my digital voice assistants are a regular part of my life. I don't know how I just got that. Yeah, that's interesting. You know, like I assumed that was probably true. And so I was kind of surprised when you also said like, no, never.

OK, well that that makes a lot of sense. So you've then got quite used to using, as you said, like serial 4 timers and so on Alexa, but you haven't used ChatGPT in terms of the voice mode. I have not know you know I'm still on the free plan. Is it cheapo? So no, I have not gotten access to that yet. OK. Because in that case, that actually is in a way perfect. Because I have been using the various voice modes of Chachi, BT and other solutions quite a

bit over the last few years. And one anticipated thing has been that Open AI was going to launch their Scarlett Johansson version of their, you know, basically advanced voice mode, they call it, and it was going to come out in the spring or the summer. Then Scarlett Johansson sued them because it was actually using her voice, basically. Without her consent? What?

Why did? Without her consent, very super sneaky and super arrogant to, as you say, like without her consent to assume that they could do it. But they initially asked her and she said no. And then they decided let's just do it anyway. That was the decision. What? You know, this shows Silicon Valley or something like this. Parody of what Silicon Valley people do and use the. Hubris. Hubris.

Yeah, Yeah, it is unbelievable. But anyway, as a mere call it innocent consumer in this situation, I was still looking forward to using a more advanced version of this. And especially as these kind of videos was coming out of people testing this advanced mode and noticing that it can do so many cool things with it or disturbing things as well. I was really excited to get my hands on it.

And this would date episode a little bit would when I say this, but a few weeks ago or a week ago or so, they said, hey, we now is going to give this access to everyone. Everyone's going to get it. And I was refreshing, you know, on a daily basis, like, OK, when I'm going to get it and so on. Finally, the day comes when they realize it's not coming to Europe because of EU law, because of EU law. They basically it's assumed because this is trying to

register a bit of your emotions. AI in EU can't currently get access to users emotions as part of their kind of engagement with the user. And so for that reason this new version for better or worse can't be rolled out currently as it is in EU, but it is available in in America. So I, I am curious if you're up for it, if maybe we can find a way to get you to test this, try it out and report back to me in a few episodes and see like, you know, give your take.

Like, is it worth the hype? Was it disturbing? Was it fun? Was it, you know, not useful at all? Would you be up for doing some form of version of this? Yeah, sure. I, I can give it some different emotions and see if it accurately, I don't know, like catalogs them.

So related I think is, is there's, there's been a lot of hype around Google's notebook LM, which of course, probably everyone else in the world knows by now can create a podcast, basically a dialogue between two people discussing the, you know, the topic that you ask it to summarize. And I was what I I don't usually fear for my job as a result of AI, but now as a podcast host, I do wonder, like, are we becoming obsolete? What do you think?

Well, I think actually this is something that we're going to explore later also in the season probably we're going to actually put this to tests that is currently in our internal schedule is to have an episode that tests this a little bit. And yeah, we'll see. I guess the listener will have to decide at some point and be the judge.

You know, to be fair, I do think it's very impressive in terms of like how they've been able to create this very, you know, again, emotive and engaged sounding hosts that I think the most viral clip is kind of two AIS kind of coming to the realization that they're not real. And they're grasping with the fact that like, wait, like what I'm calling my wife, you know, is she actually there or like, you know, and you hear, have them have that conversation in a

yeah. But I as a human would never have those doubts, so that's how you know the difference. Well, that separates you from me. I am still, you know, never sure if I'm in a simulation. Or not. That's because you are, as in a simulation. OK, I'm in your simulation, little did you know. This is the episode where I talk to an AI. Yeah, yeah, that would have been the perfect Plot West. But no, what are we actually doing this episode? This episode we're talking to

Susan Murphy, and I'm a big fan of Susan Murphy. I've been a big fan of Susan Murphy for a long time. Actually, in 2018, I went to a webinar that she gave on her micro randomized trials. So she talked all about how she was using them in her famous heart steps study. We'll include the heart Steps study in the show notes because it is just so cool and it's gone through a lot of iterations over time, yeah. A pre COVID webinar. Wow. Yeah, they did exist before

COVID, believe it or not. But one, you know, one of the things we'll talk about with Susan and what I just think is so cool is she really developed the micro randomized trial.

And you know, maybe it's worth just getting into a little bit about what the difference is between a micro randomized trial and sort of what, what we behavioral scientists think of as a standard RCT. And you know, I, I don't, I probably don't need to describe what our, our standard RC TS are, but just for the sake of the contrast that, you know, participants, you get assigned to a treatment group, one or two or or however many groups that you're comparing.

And then you take your dependent variable at time 1 and time 2. And you say, or you know, what's the difference here? So for example, if you want to understand the effect of a reminder to stand, this is this is sort of an easy example. Everyone in Group one might get a reminder. Everyone in Group 2 might not get a reminder. That could be your control group. And then at the end of the experiment, you say like, OK,

who stood more? Was it the group with the reminders or the group without the reminders? But if you think about that as an experiment, you know, say everyone with the reminders stood more than the people without the reminders, that's super, super boring. It's so non specific. You, you don't have much information at all about, you know, what does this work on an individual level, under which conditions and and so on.

And so one of the contributions of Susan was to develop this micro randomized trial where all participants are randomized and maybe throughout the day, many times throughout the experiment to either get a standard notification or not. And so now with other version of a randomized trial, you can start to understand these contexts where the notification works and really on an individual level, what works for

each person and when it doesn't. And I remember in this 2018 webinar thinking, Oh my gosh, like why isn't everyone doing this? This is so cool. And then a couple years ago I was fortunate enough to get first hand experience actually using Susan's methods. And yeah, I'm hoping some of that actually gets published, then I can share it. But yeah, can now count myself as one of the lucky few who have been able to be engaged in using

micro randomized trials. I think I agree with you in terms of like it's so kind of take it for granted that, you know, RCT is in form of golden standard in many ways. Where of course in some contexts, I think especially in the pharmaceutical world, maybe it's often times like you kind of forced to do something like that because, yeah, you don't have the ability to test, you know, various drugs at various

times of the day over. It becomes a little bit complex when you do something traditional like that, but in our world, the different. Yes, there's a lot it doesn't. You can't use an MRT for everything, but there's you can do a lot. So I would encourage our fields to do more. So let me introduce our guest

today, Susan Murphy. She is professor of statistics and computer Science at Harvard and is best known for, you know what we just described her innovative research methods for building personalized adaptive treatments with AI. And really anytime anyone asks me about using AI to do behavioral science research, Susan is the first person that I recommend. And her heart step study is always the first paper that I share. Yeah. And in the episode, we explore

much of this. And if you hear us saying, Jed Dice, a good thing to remember is that stands for Just in Time, Adaptive Intervention. So it's purposefully mispronounced to sound cooler, but actually, if you pronounce it as it should or like as it's written, it's but yeah, don't be confused. It's it's quite straightforward.

That's what Jedi's are. And so we'll look at that and how they contrast and build upon micro randomized trials to personalize interventions in real time to to have the highest effectiveness. And this goes into, you know, looking at specifically, we cover how we can use these ideas to support people in being more engaged or tailoring solutions based on personal context, like their activity levels or

external factors. And we, as with many episodes, this is and we we challenge Susan as well on our quick fire round, which is to AI or not to AI. We ask her about her most controversial opinion on AI and so much more. And finally, Susan explains why behavioral science should be seen as the frontier of science today, which is obviously very fun to hear. And hearing her talk about why it's such an exciting field to be in is something I think we all kind of need to hear.

Maybe right now it's it's really nice. So with that said, let's listen to our episode with Susan Murphy. Happens to Murgatroyd. I'm excited to say welcome Susan to the Behavioral Sign Podcasts. How are you today? I'm good. It's great to be here. Awesome. Yeah, we're really excited and yeah, been in front of your work for a long time. That's great.

As have I welcome, Susan, I I'm so excited to dive into this and really get at some of the the methods that you've both developed and advanced and are really exciting. You're really at the forefront at this intersection between AI and behavioral science. And however, I think some of our behavioral scientists may not be aware of them.

So I would love if you wouldn't mind giving a little bit of an explainer on some of your work in terms of developing just in time adaptive interventions and micro randomized trials. And maybe you know, when once you've sort of given this the very introist intro, maybe we can try and understand the distinction between the two and where 1 ends and when where the other starts. Yeah. So I think many of the listeners are familiar with manualized

therapies. Often there's a manual that you follow in order to maintain consistency and delivery of different types of content, behavior change, support and so on. So when you go to a digital intervention where the individual is going about their life, for example, maybe they want to increase their physical activity, the behavior change goal here is to help them, assist them in doing that. So what do we mean? What's the analog of a manualized intervention?

And that's this just in time adaptive intervention. So the reason why we do manualized interventions, again, it's just to help us maintain a certain standard and a certain quality of delivery of our behavioral intervention. And it's the same thing here. The idea is that the just in time adaptive intervention, it has multiple components, 1 component or what we call tailoring variables. These are this is information about the individual.

So in the case of someone who's trying to increase their physical activity, it might be their current location, for example, are they at home or work or out and about? It might be the weather, is it crummy? Is it cold? So on. It might also be whether or not they've been monitoring their physical activity recently, how engaged they've been. So all of these variables might influence how responsive the person could be to a tip about how they might be more active in

the next 30 minutes. Like it could be a, it's really a brief tip. You know, it's like suggestions about taking a walk, maybe walking next time you have to go to the restroom, walk to the furthest point in the building and back just to add a little bit of physical activity to your routine. So when is it useful to send

these messages? So the just in time adaptive intervention provides essentially something akin to a manual that says, well, if the weather is really bad, better to have a tip that's related to walking in the building as opposed to walking outside. Or if the weather's really bad, the tip should reframe, attempt to help the person reframe what it means to go outside in that crummy weather and exercise. Maybe it's not as bad as you think.

You know, that sort of thing. So the way in which the justice time adaptive intervention operates is it takes those tailoring variables and there's a decision rule. The decision rule inputs the tailoring variables and outputs what kind of whether to provide support such as this activity suggestion or activity tip and whether to provide it, and then what type of activity suggestion to provide. So would you say the just in time adaptive intervention is really more of a fixed set of rules?

Exactly. It's a fixed set of rules. Now, if you and I, Elaine, were in the same, you know, suppose both of us want to improve our physical activity and in at any given moment, I might be provided a very different tip from you, but that's because our tailoring variables are

different. So the just in time adaptive intervention is personalized in the sense that it takes into account whether I'm at homework, you know, so on to determine whether or not it's really useful for me to receive this particular type of activity suggestion.

Right. And I happen to have for whatever reason both an or ring and Apple Watch and both of them give me this notification in some variation of saying like, hey, you have been sitting for a long time, it's time to stand or looks like you may be time for a walk. Would that be an example of this? That would. Be. I can't speak to exactly how these two devices work. You know what software is driving these devices. Often it is some sort of decision rule based hopefully.

Anyway. I do recall I can't remember what kind of wrist tracker I was using at the time, but I was trialing one out and I sprained my ankle and the tracker kept telling me I needed to go for a walk. Oh dear. Yeah, it was awful for me, you know, because I was already pretty upset and I didn't want to get reminded, you know? So I don't know to what extent, how sophisticated their rules are, right? They may be more or less sophisticated.

Well, certainly the the standard notification on the Apple Watch just goes off at 50 minutes at the hour, every hour. So that's an example. Of decision rules. You know, you might prefer a better tailoring variable here besides the time of the day. For example, we had one study where we were trying to disrupt sedentary behavior, but you had to be one of the tailoring variables was how long you had been sedentary. And you had to have been sedentary for a certain amount of time.

And you had to have not received a message recently in order for us to even consider sending you another anti sedentary message. You know, something like, why don't you stand up and roll your shoulders or something just to disrupt sitting, right? So that's the just in time adaptive intervention. But I I do have to comment. You know it is called a Jedi, right? Why is it called a Jedi?

Because when we first went down this one of my colleagues, so I partially work in computer science and one of my colleagues also ambush Torari works in computer science and we were running a workshop. And so and there's a many behavioral scientists there. And it was all about constructing just in time adaptive interventions. And we coined that phrase because we were everything was after after the Jedi's. Yeah, we looked for the words that would go. Yeah, you come up with the

acronym first. Yeah, and you notice how it's mispronounced, right? Well, JIT, but I still called Jedi. Called the Jedi. OK, well going forward I will correct my pronunciation. It's not a Jedi, it's a Jedi. Awesome. And how much of yours Lucas and what not been involved in like is there any? Did he give it the green light? Well, we're not trying to make any money. We're just trying to do science and help people. So I don't think they have anything to worry about.

Us. The people at the, the, the little workshop where this phrase was coined, they really liked it. You know, we had Jedi's all over the place. What's the name of the little bitty character that trains the Jedi? A Yoda, a little. Yoda. Yoda Yeah, we had little Yodas everywhere. Baby Yodas. We just didn't, it took us a while, you know, to start thinking about how to write a paper about Jedi's. And by the time we wrote a paper, some of the behavioral scientists had already written a

paper. And we're we're using the term. It's great. Wow, right? We were thrilled. Of it, yeah. So the Jedi's, of course, are these more fixed interventions, but a lot goes into creating them. And so that's really where micro randomized trials come in. So can you tell us a little bit about what micro randomized trials are, how they work, What are the nuts and bolts of this

intervention? Yeah. So suppose we don't really know whether or not it's useful to send an activity suggestion when you're at work in the morning and the weather's not so good. Suppose we don't have a good feeling. I mean, remember, the activity suggestion can just be walking around in the building you're in, right? And we want to learn whether or not it would be useful to you, to people, if we would send an activity suggestion at that time.

So how do you learn? Well, the way you learn an AI, the dominant way we learn an AI is by what we call exploration. And what does that mean? That means that sometimes in the morning at work, when the weather's not so good, you get an you receive an activity suggestion. And sometimes in the morning at work, when the weather's not so good, you don't get a activity suggestion.

And the question is at which of those times, if I compare when you get an activity suggestion to when you don't, which one appears to help you in terms of being more active? And that's what micro randomized trials are all about. It's to provide that exploration so that we can learn how to form

those decision rules. So if the distinction or maybe the metaphor that you're making is really one of explorer versus exploit where micro randomized trails are the exploration and then the Jedi's are the exploitation. Yeah, they're the Jedi's are are implicitly saying that we have enough knowledge, domain expertise, data science knowledge to be able to exploit that knowledge and know what we should do in the morning at work when the weather's not so good.

Whereas the micro randomized trial is saying, well, at times at which you don't have that level of expert knowledge. Let's explore and let's see which one works. So for example, in this one physical activity study that I was part of with Petya Klashnia, we knew that if the sensors detected that you might be operating a vehicle at that moment, we did not want to interrupt you and send an

activity suggestion. We exploited our expert opinion that we should not be interrupting you when you might be operating a vehicle and ask you to go for a walk. And on top of that, it doesn't make any sense anyway, you know. So there are many times at which there is sufficient information about your current setting so that you know you should not

bother that person. Or even we decided if you'd received an activity suggestion or any other kind of suggestion within the last hour, we shouldn't bother you. And we just decided that was just, that's our domain expertise. We don't want to overburden people. We don't want to produce habituation where they are no longer responsive to the suggestions. So that constrains the

exploration. So in a micro randomized trial, you're still exploiting domain scientific expertise and you're exploring among the options that are still you don't have a good, you don't have a good understanding of which is best. So it's not complete exploration. It's constrained and it has to be ethical, right? That was, that's relate to the point I made about driving. You also don't want to overly burden people.

That's why you don't want to send a suggestion too frequently and so on. Also in that trial, individuals could turn off the notifications for a certain number of hours. So if you turned it off, there's no way we're going to send you, you know, we're just no exploration, man.

Yeah. I want to push on the methodology just a little bit because I I find this the two-part process very interesting and I want to understand better what is necessary about having the Phase 1 micro randomized trial kind of leading into as an input to the Jedi. It how much of that is essential that one phase feeds into the other versus you could imagine some combined intervention where the final product has more of that experimentation and learning all kind of, you know,

smushed into one product. Actually, what you're doing is

you're going into our second topic about reinforcement learning because that combines let's do it, but to what? Just go back to the micro randomization. So there, what's weird about it or odd seemingly odd about it is that when you talk about sending, for example, sending activity suggestions to someone as they go about their life, there's many times you could do that per day and there's many days over which that could happen.

So like in our in the study I've been talking about, this is hard steps. The first study each person could be micro randomized 210 times, and in further studies it was over 500 times. And the number of times at which you might explore just has to do with the number of instances at which you want to learn what should be done at that moment. You know what kind of support should be provided or whether you should even, you know, provide any attempt to provide

any support. So micro randomized trials have what we would view as randomizations. But in fact, we really should be thinking of it as exploration like you were doing. Elaine. That's a much more accurate statement. But at the time we developed it, we were focusing on individuals, scientists who were used to randomized trials. And so we were connecting everything to randomized trials. So we framed it from a randomized trial perspective.

Nowadays, it's still framed from a randomized trial perspective, which as we all know is one of the best ways. It is probably the best way to obtain evidence about one thing versus another, send an activity suggestion versus not in a particular setting. But now we have a more subtle understanding. So back to the trying to help people improve their physical activity. The society is always changing, right?

And there may be, you know, we have new drugs that come out out that help people who are overweight. We have pandemics that occur. There's crises in communities, both good and bad things that happen to our society or our community as time goes on. And the kinds of people who might be interested in improving their physical activity change

in unknown ways. They may look similar age group, similar working habits and so on, but they may be in a different state, for example, or a different ethnic group, so on.

OK so reinforcement learning combines the micro randomized trial with the Jedi. And So what it does is that at each time it trades off this exploration that is learning, trying to obtain information about whether or not it's useful in our case to send activity suggestions or not a versus should you exploit the information you've already

gained. So it trades these two at every single time as you go through many times experiencing a digital intervention, you the algorithm is trading off well, is there enough evidence right now to exploit it and make a, a decision with certainty that you should send an activity suggestion or send a different kind of support? Or is there still some uncertainty and so we need to explore a little bit to make sure we improve our delivery of

this support. My personal opinion is because society is changing, it's always changing, that this exploration can never end. We have to have a more of a viewpoint of that we're going to continually and you're changing. If you're experiencing a digital intervention, you know you're changing. You know you're having things are happening in your life. Your children maybe get the flu and all of a sudden it's much harder to move around. You know you don't have time, you're too busy caregiving or

you may sprain your ankle. Sprain your ankle? Exactly. So you have to, In reinforcement learning, we the, we have an algorithm that operates, it's either operating on the phone or in or on the cloud, but communicating with the phone, the wearable or your iPad. And it it does some mixture of exploration and exploitation of its current understanding of what's best.

Yeah. So, so it's never really fixed, even if, you know, say for four months or so, you don't respond to an invitation to get up and walk around while it's crummy outside and you're at work. But you know, say that was when your ankle was sprained. And then four months later, there could still be a reason to try that again in case. But with very low probability, right, with very low, because you don't want to bother, you don't want to discourage the

individual, right. But. And you also like in a really good digital intervention, you'll be able to provide some feedback. You'll be able to indicate that this suggestion was completely inappropriate for you in your life. You know, it's just like maybe. Well, in our case, we weren't

giving suggestions to swim. But if you don't like swimming and you got a suggestion to swim, you should have the right to be able to indicate to the little coach on your phone, IE the algorithm in our case that this is just not a suitable suggestion. It is, yeah. And if the algorithm ever wants to ask you again about it, it should phrase that in a way. Well, in the past, I understand you weren't, you didn't really want a suggestion about swimming. But perhaps might, you might

consider it now or something. You know, it has to take into account your feedback. It has to use your feedback to improve its learning about you, about what can be done to help you. Yeah, And I guess I'm keen to maybe take a step back because I think one thing that comes up quite often, I think when I'm trying to communicate around behavioral science in general, you know, a lot of people are keen for a long time forever, maybe for the forever how how

long humanity has been around. People have been very happy to say and tell people what they're supposed to do, like do this, do that to tell them what to do. And I'll, you know, been trying to communicate around behavioral science that is important to understand like what we want people to do. But oftentimes we think that, OK, if someone is not, you know, let's say fit enough, they they, the what is go to the gym or eat

vegetables. But it's actually quite hard to understand, like what is the right what? That's tricky. But then I also talked about like, you know, it's not about what it said, but how it's said and when it's said and even by whom sometimes in terms of like who's the messenger? And so suddenly we're having a lot of different colored like variables of sort to consider. Well, it's not only that. It's like you the person you

should have agencies. So like you have decided, say for example, you are newly diagnosed with stage 1 hypertension. So you need to improve your sleep, you need to eat better, you need to be physically more active, right? At least these three things. Well then you should be able, you should decide what you want to work on. And if it's like, if the

algorithm, so in our in the world I live in, what we want is an algorithm, We want to build these algorithms so that if it notices that you're becoming increasingly disengaged, IE non responsive, then it might suggest, well, would you like to switch things up and try working on your sleep instead? Or would you I'd like to take a vacation. Sounds good to me. Yes, yeah, let's go with the vacation. Yeah. I mean, in other words, you want the AI agent, that's what I'll call it. That's Yoda.

You want that to be your partner, right? A partner. It's not. And that's the ideal. That's what we're striving towards is that this artificial agent will be a partner and you have agency. You know, we're all about helping you achieve your goals. Not our goals, your goals.

Yeah. And I think that's really important and sticking to the idea of like, just still weighing some of these things because the shared goal is that, you know, this person wants to prove their health and they have sought out help to increase their chances.

And so knowing that like this feels like, it feels like Asian science, but it's, it wasn't too long ago that, you know, interventions in behaval science were just only thinking about, you know, one like, let's say 5 different text messages that has

files and framing. So it's only about like how something is said and it only testing how these five different messages are going to get people to take it like, say, on average, on average or when like we're just going to remind people at certain times of the day and then yeah.

Quarter till the hour. Yeah. And so, you know, in terms of, you know, we're getting into reinforcement learning, we talked a little bit about and how we think about like how, how would you start, you know, make sense of these different variables? Like what is more important to weigh?

What is the core thing to to focus on, you know, how do you tackle that conundrum of like, should we get the algorithm to focus a lot on like, how would just reframes messages rather than how it sends at certain times of the day or, you know, trying different behaviors, like what people supposed to do rather than when they're doing it? Like, how do you think about that conundrum? Yeah. Right. So the issue, the big challenge, OK, so I'm one of the developers

of these algorithms, right? And these reinforcement learning algorithms, they essentially have two parts. The first part is it's usually some statistical method. It's trying to learn about you, how responsive you're likely to be to these types, different types of support in a particular setting. So that's the first part of the reinforcement learning algorithm.

The second part is how do you use that information to decide whether or not to send a particular type of support or not send any, not bother the person at all. OK, so we know like many of the people who have a behavioral science background, they have had some statistics and they know that there's a lot of noise in these problems. You know, there's, and when you're talking about the digital intervention space, there's this is we're, we're providing support as someone goes about their life.

So all kinds of things are happening. You know, someone almost hit you when you were driving to work today or your boss is not in a good mood or you stubbed your toe when you walked in the door. You know, there's just one shock after another. So these are very noisy settings. And when there's a lot of noise, it's much harder for that first part of the algorithm, the part that's learning about you to learn because it has to separate

the signal from the noise. And So what we normally do in terms of building these algorithms, at least in so far, we'll use domain science to come up with a whole host of different types of support, but the algorithm will personalize whether or not you get bothered at that moment in that setting. And why do we do that? Because first of all, the the biggest signal is usually between sending something and not doing anything at all, something active versus nothing.

It's like providing a medication versus not giving any medication at all. That's where you normally see. So the chance of this algorithm learning whether or not it's worthwhile to interrupt you. Is much higher if we focus on that one decision then we try to the extent we can to use behavioral science to ascertain what kind of support should be sent if a decision is made that support should be sent what kind of support. And now most of the time in our

in the studies I'm in, we micro randomize among those different kinds of support. So say if there's ten different ways in which we could provide support at that moment, we one, we've probably if the algorithm says send support, it's worthwhile to the person's receptive, they're probably, you know, they're more likely to be helped by any kind of support, then we'll send any one of these supports with chance 110th. So you can learn a lot more than just the optimization of timing.

You can understand what kind of content at what. Time. But how do we learn that? What we, we learned that after you have an implementation period where you implement, So the reinforcement learning algorithm is operating while you're experiencing the intervention. It's learning and selecting whether or not to send different types of support or in our case whether to send support or not. It's learning about that, but it's just doing pure exploration among the kinds of support.

OK, so you finish an implementation, say two months have gone by, you look at all the data from all the individuals and now you've explored on those individuals. Every time you send support, you explored among those ten options. Now you do analysis to to determine what was one of those options better in a particular setting. And then now you go to the implementation and you can use that information.

So in that setting, the algorithm would just learn whether or not perhaps to provide support and if so with high probability would provide the support you learned from the analysis. So there's two types of learning going on. There's the algorithm is attempting to learn as you experience the intervention. That's the first type that's for you. And me just on an individual level, right and then? After a group of individuals, they've experienced the

intervention. Now you have a data, pretty rich data actually, on a whole group of people. Now you have a chance to get rid of some of that noise. You have a lot of people. Now you're going to incur bias because people are different. You know, we're not the same, but there may be some overall commonalities between us. Like for example, when we're at work and we're in a meeting. You really shouldn't be bothered, I mean.

You should, right? So anyway, you know, so there may be certain commonalities between all of us that can be learned between implementations. So what we have now is a hierarchical setting in which within an implementation, you're trying to learn on each person, is it worthwhile to interrupt them or not to try and help them out. And then between implementations you're trying to learn, well, what kind of help is useful in which kind of setting?

And actually in our world of digital interventions, this is sort, this is really a good idea because technology is changing so fast. So between implementations, a new kind of tracker may have come out. How do you integrate the tracker into your digital intervention? There may be, you know, there's societal changes. You need to think about the societal changes that are going on or there may be a new type of intervention that's come out.

And like, you know, right now there's a lot of interest in positive psychology. I know there's been interest for a while, but it's a renewed interest in positive psychology. So maybe you didn't have anything about positive psychology in the last implementation, but now you want to try and explore a little bit among positive psychology messages in the next implementation.

So in a world in which the environment in which we're learning and trying to personalize is evolving really rapidly, it makes sense to have this hierarchical approach, learning within a person over time as they experience the intervention, and then learning across groups of people between

implementations. And that's essentially what we do. What would you say is the most interesting or novel finding that you've found at the population or sample level across groups that that doesn't rely on individual idiosyncrasies, but something that you've found from this rich data that goes across so many? I think many of your listeners will be acutely aware of this,

particularly if they downloaded a mobile app, but we didn't realize this at 1st. And that was that disengagement and habituation is really a big problem in the digital intervention space. So it's very hard to continue to persevere when you're trying to change your behavior. It's just really hard. That's all there is to it. It's just really hard. And so usually people at first they can stay engaged, say for, you know, 3 weeks, 2 weeks, and then their responsivity starts to decrease.

So seeing that across many people, we began to realize now you have to have interventions which are about helping people stay motivated, helping them. It's not like you're giving them ideas about how to be more active. It's about helping them keep their mind going, you know, keep on task and like if you, so that's one of the nice things about if you have multiple things you want to work on, you can switch, you can take a vacation.

This these are ways that you might be able to help someone stay motivated and giving people breaks is really useful. The vacation idea, that was a big one and it's obvious now to me, but I have to say, it wasn't obvious at the time. I just thought because no one could ever, in the studies we had run, no one could ever get the same message twice. They could never get the same suggestion twice.

So I thought, well, why would their responsivity to these suggestions versus not getting a suggestion decrease with time? They're not getting the same thing. They're never getting the same thing. But that's not it's just it's behavior change is hard. Yeah, and and I'm sure there's a bit of notification fatigue. Right when you. Start to ignore all the

notifications. Yeah, Yeah, exactly. And so we also try in terms of your current context, we try and keep track of how many times you've been notified recently because if after you get to a certain level, it may just not be worth it, it you may, you may need help, but it's not worth the price of trying to reach out and provide. It's not going to be helpful. Yeah, I wonder how much of it also has to do with users when they're getting lots of small

recommendations. It feels like maybe just the weight of how much there is to do because they haven't formed a habit. They haven't set a routine in place that makes these things. It really happened with automaticity. It's that everything, every message is one more thing that they have to do as opposed to, oh, now I just have a habit where I go for a walk every morning. And it doesn't feel like cognitive burden. Yes, you know, you have to cognate about everything. It's true.

But so you know, I mean, this is one reason why we break goals, for example, big goals into many small sub goals, right, is to try and help people and then we try and help them. Like for example, right before you, you eat, try and go for a 5 minute walk or you try and anchor certain behaviors to patterns that are already there in your life.

You know, we try and these are all part of that toolbox of way of supports that I was talking about is trying to help the person find the right way to keep up. And in the case I've been talking about, you know, improving their physical activity. Yeah. Well first of all my wrist is vibrated, so it means that you need to. Stand up. Stand up, but but it's also maybe we're going to soon get into our kind of quick fire

round. But I wanted to just wrap up with a kind of final question on your research, which is, you know, you've kind of LED a very data LED approach to understanding human behavior. And and I guess it'll be interesting is to hear what you've learned about humans from your research so far. Does anything kind of generalize at all at population or, you know, bigger level?

Because I'm just interested if you like had to say, you know, in short, like one thing about humans, if you're some aliens that kind of like studying humans from your research, what would you say that you have learned?

Well, I think the the thing I've learned most, and it's actually I've not only learned it, but it's a big attractor is I view behavioral science, behavior change as the edge of science. It's the frontier science and it's very exciting to be part of a frontier science. You know, it's not like, well, I don't want to diss any other area of science, but there's a lot of areas of science of which there's been a hundreds of years of work, you know, and they have all these mechanistic laws, but

we're not. Yeah, I didn't want to say that. You know, it's not simple. We're really a frontier science. We're on the very edge, you know, brain science in general and why people do what they do and how they can change their behaviors and how they might be maintain changes. It's just it's very much on the frontier. And I think we have to always, at least the thing I've learned is always have to remember that. I can't get discouraged. I can't say, oh, there's so much noise.

It's hopeless. This is a hopeless problem. No, I have to remember when you're on a, you're working in a frontier area, frontier science. This is the way it is. This is, you know, it's a, there's going to be, you know, you're going to take two steps forward, one step back, 2 steps forward, one step, right. And that's just the way it is. And that's because we're working on the edge. It's great and then you throw a throw in a dash of AI and you can't get any more cutting edge than that.

Well, AI is supposed to be of assistance to behavioral. Space, right, Right. It's a tool. It's a tool in service of behavior.

That, I believe, is the perfect segue to a little game that we're going to play. Hey. I hope I can do it. You can. So this we have a little quick fire round called to AI or not to AI. So we're just going to present a bunch of tasks to you, and then you're going to tell us whether it is or isn't well suited for AI. For AI to try to solve or tackle. Yes, to perform. To perform. Are you ready? I'm.

Going to do my best. All right, to AI or not to AI nudge commuters to change their mode of transportation in order to optimize traffic flow. Oh definitely to AI. Nice. What about help someone break a bad habit? To AI, definitely use to a. All right, pick your bedtime. I think you could use AI for that too. You could help me decide what my bedtime should be. Assisting, not deciding. Yeah, assist, not decide. Nice. OK, what about on the spot therapy for someone experiencing

high stress? Yeah, you could use AI. It would be good if you can tailor the support to the context the person is in. Are they by themselves? Are they in a social net setting? Is it in the middle of the night, like at 3:00 AM or, you know, so on? Is the sun shining? That's sort of it. You know, maybe they could go and put their face in the sun and feel a little bit better. You know it. Get a vitamin D lamp. Yeah, get a vitamin D. OK, purchase and deliver

groceries. We're already using AI for that. I'm sorry you got to give me something that AI doesn't have a chance of helping us. We'll get there. Yeah, we'll get there. But this one is still a little bit of like adjacent to what I guess and somewhat is happening. But we can splice it up with saying that it's from a virtual A or AI Yoda teaching algebra to 6th graders. Oh, definitely. You can use that. I mean, there's already a lot of work on this already.

Online math tutoring, teaching. Yeah, definitely. And is there something that 6th graders shouldn't be taught by my guy? Things their parents don't want them to learn. How to assemble a machine gun? Or whatever you know their parents decide. But yeah, I think in all of these cases, AI can be an assistant. I don't want to say AI can take over from a human, right? And in my experience, humans, good therapists are always better than AI. But AI cannot be better than a really bad therapist.

And AI can be there with you at 3:00 AM and the stairwell in the middle of the night. Very true, but can an AI, this is our next one, perform a pap smear? Not right now, but give it time. OK. There is AI assisted surgery, you know that? I'm sure you know that, yeah. Harder than a pap smear in my opinion. Well, the patient is asleep with the surgery and the patient's not asleep with the pap smear. I don't know. It's a little bit more complicated there, we. Go, OK, you don't realize.

Well, so far, so final one, final one, to AI or not to AI? Give someone a haircut. Not at this time, but that's just a matter of time. Well, at least for me, I I don't have much to cut, so yeah, you. Don't have to worry about how they can help you with your beard. Yeah. No, I mean, I think this is coming. It's not at this time that I know of. Robotic haircuts. Yeah. Yeah, no, I wouldn't do it. I don't know, I might. Why not? I'd try it once. Yeah.

I mean, you already have this thing where you go to a website and you tell them your size and you've visually, you know, you, you see yourself trying on different clothes, you try and see how you would look with different clothes on and stuff like that. So I don't know. But you can always, when you actually try on the clothes, you can return them if they don't actually fit in real life, right? The haircut, you're not getting that back. That's right.

I know exactly what you mean. That's happened to me before. I've gotten a haircut, but this time, you know, from a human. And I thought, Oh no, there's no way I can go back in time. Oh my gosh. All right. We are almost out of time. So I'm going to throw our very last question at you, which we ask all of our guests. What is your most controversial opinion about AII?

Don't know. It depends on who sees it as controversial, But I do think this idea that we can develop algorithms that actually learn as a person experiences an intervention, that is controversial.

It is. And it's controversial among data scientists because of the enormous amount of noise going on. And there's a lot of concern that the noise just swamps any kind of signal. And we've worked on a whole variety of ways to support that learning in order to combat that what I would call controversial idea. So you're describing some form of like synthetic users or synthetic kind? Of right, you can develop digital twins that's you can try and do that. But you can also like using data

from other individuals. You can initialize your algorithm. You can warm start your algorithm so that even though the algorithm is not making, it's still having to do more exploration because you're a new person. You're not the same as all those other people. It's closer to where it's closer to finding what would be a good solution to you than if it was just a cold start and it hadn't used data on anybody else.

So we do a lot of that. We try to use all the prior implementations on similar people to warm start each algorithm so that you can detect the signal in this sea of noise. Yeah. Yeah. And I saw maybe it was this this spring, I saw some, I think they were actually quite young students at Stanford.

But they basically had this a generated some form of, I think it was some representation of maybe some city in California like Palo Alto or something like this, where they basically had like generative agents that represented individual humans to basically understand what the optimal transportation patterns and so on. Yeah. And how they interact with each other. So they're modeling like multiple decision making agents,

right? Each person is a decision making agent and how do they interact and how does the behavior of one agent influence the other agents? And yeah, yeah. Yeah. So you, I guess you're then quite pro to what can be done in that domain like in terms of exploring that further and and especially as you say maybe early days in terms of like reducing anatomic consequences as well. Maybe is that what you're describing like in terms of before before launching an intervention?

Right. At human level to have some form of way to. Right. So remember earlier in the this interview I mentioned that you don't always do complete exploration. It's very constrained. And everything I do is embedded in a behavioral science team. So if there's any, you know, there's ethical constraints, there's burden constraints. And I don't the algorithm is only permitted to explore within those constraints. You have to have equipoise for

exploration. And so the ethics there, they really come to the fore because we're embedded in a behavioral science team. I'm not sure if I'm really answering your question. This is critical. And so of course, when you build a digital twin of us, say people who have stage 1 hypertension, you build a digital twin of that subpopulation. It's akin to what you were

talking about. You're trying to understand, well, how do people who first get diagnosed with stage 1 hypertension, how might they respond to this digital intervention in various contexts and what rate at which they might start to experience overburden and habituation. Try to understand all of that before you put the intervention out to you know you provide it to real people.

And I know, Elisa, there was last question, but now I feel like I want to ask one word relating to this little So so there's a Black Mirror episode, I think it's called, I'm trying to think, is this called Hang the DJ or something like this? It's actually one of the few Black Mirror episodes that is kind of a positive. Like it's a little bit non dystopian, but actually kind of positive. You see this kind of like two people that are dating and they

have in this dating mechanisms. They have some form of devices that tells them how long they can try to kind of date together and see how compatible they are. And then if the time runs out, they would kind of be much to someone else. And then, OK, this is spoiler alert for anyone who hasn't seen this episode. But in the end of the episode, you realize that actually what whatever what happened that you saw was actually within a higher order device of real people.

And what you have seen was just kind of digital twins of the real people that were kind of, yeah, they were basically like test dating other digital twins of other users. And then recently there was, I think the CEO of, I think it was Bumble, the dating of Bumble, who basically were kind of like teasing like, oh, wouldn't this be some interesting feature that

we could launch? Or I don't know exactly if it was more tongue and chic or if it was actually something that she suggested that they would try the lunch at some point. You see, this scares me. This sort of thing scares me

really bad. There could be all kinds of massive unintended consequences and you could end up like you could end up nudging someone towards just people like themselves when they might have ended up with a much richer life if they'd ended up with someone who was quite different. And so I don't know, I don't know, slot my cup of tea. Doesn't doesn't, no no AI like we had our game before us. This is not AI. To me, when I hear that this is the problem with AI that is extracted away from the

behavioral science. AI needs to stay completely in touch, grounded by behavioral science. That's my opinion. Oh yeah, not argue with that, yeah. Well, yeah, this was amazing. Thank you so much, Susan, both for joining today, sharing your amazing research and, and, and work and, and thoughts with us. And yeah, and also for your, again, your great work. It's been really lovely to to follow your work and, and now to speak to you as well. And yeah. And remember, just don't get discouraged.

We're all working in a frontier science every time. Anybody you know, in physics, blah, blah, blah about how we're not so great? No, no, no. They're not in a frontier science. I mean, what can you say? I want to be. I want to go where no one has gone before. And that's a wrap. You've been listening to the Behavioral design podcast brought to you by Habit Weekly and Nuanced Behavior. Sam and Eileen tell me.

This season is packed with incredible insights about behavioral design and AI, so be sure to subscribe and share the podcast with your friends. Though you might want to keep it away from your enemies. In case you haven't noticed, I'm an AI voice. Yep, pretty crazy. Quite the improvement since last season's AI outro, don't you

think? If you'd like to collaborate with us at Nuance Behavior, where we use behavioral design to craft digital products with Nuance, e-mail us at hello@nuancebehavior.com or book a call directly on our website, nuancebehavior.com. A special thanks to the amazing Dave Pizarro for our show music and to Mei Chen Yap and April English for their help in producing and publishing this episode. Thanks again for tuning in.

We'll be back soon with another exciting conversation where behavioral design and AI intersect. Happens to Murgatroyd. Oh. And remember, just don't get discouraged. We're all working in a frontier science every time. Anybody you know in physics, blah, blah blah about how we're not so great. No, no, no, they're not in a frontier science.