Behavior Change Score with Roos van Duijnhoven

Hello, hello, hello. Hello, this is Samuel and welcome to another episode of Behavioral Design Podcast. This episode is part of our ongoing miniseries where we get a chance to speak with the colleagues we have at Nuance Behavior. And here there are unique perspectives on what it means to design for baby change in practice. So Nuance is home of some fantastic people and today's guest is no exception.

I'm very thrilled to welcome Rouge von Dynehoffen to the podcast. With a background in baby change and neuropsychology, Rouge specializes in applying behavioral science to design human centric solutions for some of society's most pressing challenges. Her experience spans diverse sectors from cyber resilience and digital literacy to mental health and habit formation, always with the focus on making a positive impact.

We had a really fun conversation and discussion exploring some interesting topics including the Behaviour Change Core framework, how it's been applied in digital health and what it reveals about designing effective digital products like what makes for strong onboarding and why retention remains such an

ongoing challenge. We also talk about Big E versus Little E engagement, where we're really diving to why real world baby change matters so much and why we should maybe favorite a bit more than just focusing on E nap engagement, but also how wearables and other data can really make that easier.

And of course, I can resist putting rules on the spot with a controversial question, this one specifically personal for someone from the Netherlands. So All in all, you can expect a fun and wide-ranging conversation, particularly focused on the digital side of the signing for Baby Change. So without further ado, let's dive into this episode with Roosh Happens. To. Murgatroyd, I'm very excited to say welcome Roos to the Behavioral Design Podcast. Thanks for every.

Yay, this has been a long time coming. If you like. We've had a lot of conversations but none of them on a podcast before. This is fun. Yeah, it's going to be fun. I think this is going to be the first episode that I'm not going to listen to myself. Yeah, of course. Yeah, I know I can relate. And I feel like I've just come to the point where I just had to learn to suffer through sometime

having listened to myself. And yeah, but yeah, it's it's up to you if whether you want to listen to it, but you have no choice but to participate. So, so, yeah, I think given that this conversation is always built a little bit of the last podcast we had with Susan Murphy. So yeah, I just maybe to start, I'm just curious, what's your impression of that? Episode. Yeah, I thought it was super interesting episode.

And especially that she, I think at some point made a comment about that it's just really difficult to stay engaged over a longer period of time. It's sometimes relatively easier to get people to start something new, but then to actually maintain that over time, it's just really difficult. So I noticed a few things that she said that kind of overlapped with the work that we're doing and the findings that we have

found. Yeah, I agree. And I think it's fun in some ways above because part of me was thinking about you a little bit because we did some work I think maybe a year ago or so where we actually work with kind of messaging specifically and we're working with this kind of large insurance provided. We wanted to basically help with getting people to take a test to see their cardiovascular risk. And I wanted to like find ways to develop a messaging strategy

to to ensure that that happened. And that was fun because we applied in some ways a lot of those principles in that example. And we were able to actually increase with like 40 or 50% the people who did this. That was like one of those like interventions that went really well. That was that was fun. But yeah, it reminded me of what we did there.

But also like taking that maybe even to a further degree with specifically using some of these kind of AI elements that we didn't I think use as much at that time. Yeah, yeah. I especially find it super cool that you have these tailoring variables, right, that she was talking about. And then you can use this to see how responsive a person is going to be to the intervention, but then also it adapts to the behavior over time.

So what we did was this framework of, OK, we have these different, maybe we even made like a behavioral persona group or I'm not actually sure if we did that for that one. But you can either segment your audience and then send those tailored messages. But then when they don't respond to it, OK, then you kind of, yeah, need to adapt. But now with Jedi, that kind of does that adapting for you, which is super cool that that's that that's possible.

And hope, yeah, also way more effective than doing everything yourself. Yeah, Yeah, I think that's true. And I think you like, that was also a good example where, you know, we probably would have liked to do more advanced things, but maybe used as a reporting from the front lines of paper science. Like sometimes he was also working with a large insurance company that has certain internal tools that they prefer to use that are maybe, yeah, not

as AI driven, let's say. So yeah, you have to make the rest of the contacts you're in, but but yeah, that's, that's cool. So I guess to to build on this, is there anything else you wanted to take away from that conversation or like highlight as an important kind of insights? Well, that AI needs to stay grounded in behavioral science, right? We're pioneering. I think that was a really nice thing that that got stuck in my head.

When you sometimes get frustrated when something is just so complex and you're trying to figure it out, but then you can't really find the information or just take so long to figure out and you have a lot of ups and downs. Of course, they have a science is super cool and I get super excited about it, but it can also give you a headache if you don't know how to sell something. And she already made this nice point about, well, we're at a

frontier science. It's very logical that there are some questions that we don't have the answers to and it's just really cool to work in an industry and contribute to trying to figure out the answers for those questions that are giving you a headache. So I think that was a really nice one. And yeah, that's that's stuck in my head. So that's that's a take away.

Yeah, yeah. I think that's definitely one of those things like I was tempted to put as my alarm clock or something as like in the morning being like you're a part of a frontier science. We're doing frontier work, you know. Yeah. Well, to, to get to get that kind of motivational doing, we're doing good stuff. We're doing important stuff because. Yeah, definitely. It's nice to hear that. And I agree as well.

And I guess to stay on track of engagement, we had this report that we published together with everyone in Nuance, which was called the Baby Change Score report, focusing on digital health products and interventions, an app specifically. And basically we introduced this baby change score. And so yeah, maybe you want to, you want to explain what what the heck, What is the baby change score and how does it work? Sure. Yeah. So the baby change score like you said.

In summary, we published the first report which was specifically about fitness apps, but the whole idea is that there is just so much information about behavioral science and behavioral design and behavior change techniques and behavioral strategies. And they give all these terms. And for someone who is maybe new to the field but does know like that behavioral science is maybe valuable for their product, it's sometimes quite hard. Like where do you even start?

And it's one thing to have all the theory, but then how do you apply that to an actual product that you're working on, right? For example, you may know that giving reward is important to do as a strategy, but then people start putting like badges. You can collect badges for all sorts of behavior that you have in your app, like when we reviewed all these apps to create a behavior change score in the end. But one app that I reviewed then gave me a badge for uploading my

profile picture. Yeah, great, that's cool. But what is that going to actually? How is that going to contribute to me doing the actual behavior that I want to change? Wait wait, so you're saying that you didn't share the badge with your partner or what? Like you didn't share with your woohoo? Got my first badge. No, sadly I didn't. But yeah, no, that's just not really related to the actual

behaviors. Just sometimes when you're done looking at so many apps that are supposed to help you change your behavior. And in this case, it wasn't about fitness, so about physical activity. You're supposed to or you have a goal. That's why you even open it up in the 1st place. You want to be more fit or go to the gym more often and then you open these apps and then these apps are supposed to help you

get there, right? But it feels like nowadays there are sometimes behavioral quote UN quote strategies just sprinkled on top. I heard someone say this once. I found that a really nice quote, but sprinkled on top without much strategic thought because like a bench for my profile picture. Really. How is that going to help me get to the gym? So yeah, the whole goal of the Behavior James Score report is to, on one hand, score different apps.

And we just now looked at the most popular fitness apps and see how they're doing in terms of behavioral science and behavior change techniques. So what we did is go through lots of behavioral research and models and frameworks and have it forming technology, all that kind of stuff. And we then categorized the different behavioral elements that are important when you want to form a habit and engage people, maintain that engagement.

And then we mapped that out across the whole user journey, so from onboarding, activation to engagement and retention. And we just started scoring like, how well are apps using these behavioral techniques that are needed to, for example, have a successful onboarding? Yeah. And I think one big part of why we ended up creating this Bayesian score is kind of alluding to what you were saying in many ways in terms of that product teams are in a tricky

situation. Like it's not easy to develop products and it's it's kind of tricky space where you know, it's easy to look at competitors. It is to get excited about what features to build and what things to do. But it's easy also then to get carried away and building things. Even though if you have this motivation to like wanting to use more behavioral driven features or product psychology in your product or app, that's great.

That's like a great intention. But at the same time, many product teams struggle there because as you mentioned, they kind of like they don't really know how to actually always put that in place. And so in many ways, we wanted to put together something that kind of meet product teams where they are, where they can kind of get some understanding of how this all of these hundreds of concepts we cover in behavioral science, how they relate to specifically their app at hand

or their product at hand. And how we then based on that can move from first understanding like what doesn't work? What are the things that maybe

are areas for improvements? And then kind of identifying those things instead of just kind of getting into the trap where I think a lot of product team struggle is like they just hear about all these exciting people saying strategies like you said, and they just want to like add them all, but they don't really know which problem they're trying to solve for in there in the rest way. Yeah, Yeah.

So basically when you boil it down to like just a few main questions is actually OK, what things are important when it comes to behavioral factors? What is important to say form habit if you want to build a habit, what is important from behavioral science to form a habit and how do you translate that to a product? So which element should be present in your habit forming product?

And then also we looked at what are the specific journey phases that people go through, So like from onboarding and activation to like engagement and retention and like what happens after a failure state. So when people stop using the app, like how do you get them back, all that sort of stuff. And then for this report, we looked at fitness apps specifically. So what we did is we per app, we had at least two people reviewing it. So we actually used the apps. We were very fit for summer.

We actually used these FS and then we started scoring them on all these, all these elements as we were going through all phases of uses. And yeah, by by scoring them, we actually looked at, OK, these variables are necessary to have say, an effective onboarding. And does this app include these elements, yes or no or like to what extent? Because it's also possible that it's like somewhat there, but maybe not fully in the right way.

So we were able to say things like across all the apps, certain certain behavioral elements were more or less present or some things were used a lot other things were completely overlooked. But also per app, we could say, oh, you're really doing well on onboarding. But we see that on retention, there do not really seem to be a lot of elements or features that seem to be behaviorally informed or that are actually reaching out to us when we drop off and

do not use this app anymore. So we could see an overall trend in behavioral techniques in fitness apps in this case, and we could zoom into one app specifically and see, hey, you're doing well on this front, but hey, on on this other thing, there is actually some room for improvement. Yeah, it's funny. In terms of that we, we chose fitness apps. I think it was in good timing because we were looking at starting the first report with some few different types of categories.

But I guess we we land on fitness apps there, the state of preference was because we wanted to look at like hard to change behavior that is like long lasting, like something related to fitness in this case. But I guess maybe also there was some kind of behind the scenes preferences for people to get fit in the team. So I think we've kind of done our best to kind of support each other.

It's kind of meta actually, because we were kind of our we, we were like the whole team was each other's social commitment buddy or something. We were really committed to doing these workouts. It was, it was really funny. I remember with with Jared, Jared and I were were working out with the same map and it was like we had quite a high degree of competitiveness in the app, like it was made to compare with your friends and so on. So it was fun to see and compare

ourselves. And yeah, I want to reveal who won here. That would be unfair to Jerry. I'll, I'll save that for when, when Jerry comes on the podcast. But yeah, obviously the report is free, unavailable on our website and you can find it easily. But he has to be interesting to maybe to talk about.

One interesting thing from the report would probably be that we looked at these 4 stages of the usage journey and we mapped the scoring of these apps against how well they performed across from kind of on boarding all the way to activation, engagement and retention. And we can see both individually how these apps were performing at these stages, but also we can see some trends how they were performing and so on.

Maybe you can share some some of those trends that we noticed from those kind of yeah journey stages. Yeah, sure. So overall, we saw that on boarding perform the best of across all the journey stages. And while retention towards the end of the journey got way less attention and we were thinking about like why this would be the case. And we thought, well, during onboarding, there's a lot of data that you can actually quite

easily collect, right? You can see how long people spend on doing a sign up or like going through an onboarding flow or where they may drop off. Like you have quite a lot of data points to collect there, but at the same time, first

impressions matter, right? So during the onboarding, I want to impress people a little bit, but also you, you kind of need to collect information from the user to be able to personalize the journey later on. And because there is just way more, yeah, data to actually measure. It's just that's maybe the reason why the onboarding performed best, whereas retention is so much can happen in someone's life that makes them drop off or not come back to an app. And it's just really hard to

figure out why that is the case. And what actually was that barrier? Why did people drop off? Was it, was it me, was it about the app or was it something in their lives? I don't know, maybe they got a kid, they don't have time anymore to go to the gym or maybe they're just really busy with work. Like there's so many unknowns there and therefore it's also quite hard to then design for that. So I think that might be the reason why on Boarding perform so much better than retention. Yeah.

No, I think it's so true. There's so many kind of incentives to do it as well in terms of just like everyone wants to get people to sign up for that and pay for that or whatever it is. And then like you say, I think we obviously try to beat the drum often times about experimentation as payable scientists. And I think a lot of teams to their credit these days, like there's a lot of experimentation and AB testing and various types

of testing that's being done. But obviously depending on your amount of users you have and so on, it can be like kind of tricky if you wanted to test some form of feature related to your retention, because let's say you wanted to test your 60 day retention. Well, it will take at least 60 days to find the answer of whether it worked or not. And that's in the case of a lot of users. In case if you don't have enough users, it might take several months to collect enough data on what worked.

Whereas if you do some form of thing to improve the onboarding and 1st day of of usage, you get that data straight away. And it's it's so much more data and immediate data. So just easier as well to I think experiment. So maybe one thing is like, I think it's also because of the difference between in app behavior and outside of the app behavior. Like and onboarding is really like in the app, right? You need to fill in questions

that are in the app. You need to go through like, I don't know, a tutorial of how the app works. Maybe you need to fill in or experiment or try out a few of the features. But then where when you're actually using the app, and in this case it's still use fitness apps as a as an example, like you need to actually either do a homework at work, out at home, or you need to go to gym and perform the workout, do the work. And that is something that's maybe using outside of the app,

which is then harder to measure. And yeah, therefore you also maybe see less features about it because it is so hard to

measure. Yeah, I think that leads us perfectly into something I wanted to discussed related to this as well, which is talking about Biggie and literally engagement. So we had actually the author of the paper that introduced this framework on the podcast before I had the code Lewis. So if you want to listen to a whole discussion about this, check out that podcast.

But basically it's this idea of understanding digital health behaviors with these lens of defining them as either being kind of biggie engagement or small engagement like you're that's what you're talking about. And I think it's really great to to get a chance to talk about that. So I guess what we see often times, as you mentioned, with kind of like this small E in that behavior often times is that, as you say, it's easy to

measure. But then the big elephant in the room, outside of the room, I don't know what to call it, but like, Oh my God, it's. A big elephant. It's an E. Yeah, that's true. The Big E elephant, that's true, I tell you. Well, mind blow, I didn't. That was not on purpose. That was a subliminal or something. But the Big E, the big elephant is that, as you say, there's so much stuff happening in the real world.

And I think you probably have this experience as well where you kind of work on product teams and everything kind of, it seems to go great until you're like, well, what about what people actually do in the real world? And they're like. Don't know, don't know and.

Especially when we talk about digital health or digital sustainability or like the stuff that we work with is basically all of these kind of often times context and industries and sector where people kind of have to do something in the real world. It's not enough that they just log into the app, for example. So, so yeah, that is the challenge, how to better understand and think about this kind of Big E behavior wraps in the real world.

So yeah, how do you think about? That yeah, Yeah, good question. So the little ES is about in app behaviors, right? And it's about the not only using the app, but also interacting with the behavior change features or the intervention, because only through the intervention, you will actually help people to do the big, big E the the target behavior that you actually care about.

South for fitness, you can in the app have a feature, maybe have a commitment with a partner that you can can go for runs together, but then actually going for the run together is then the big E. So yeah, it is sometimes to be hard to measure this Big E in the real world because of the fact that it's outside of the app. So it's hard to track that within the app. Or you can ask people to log their run, but then the logging the run becomes a behavior in itself.

So then people need to and go for the run and log the run because otherwise it doesn't count. So yeah, it is really difficult. But what you can do sometimes is to look for metrics or proxy metrics, and a combination of different and carefully thought out proxies can then approximate the target behavior that you actually care about. And I was thinking about this the other day when my partner's dad had to go to a sleep lab to measure things for his sleep

right. Like when you kind of you're, you're basically you're plugged into these machines and you sleep overnight to measure and stuff. Exactly. So you had to go there and all this like stuff stuck on his head and his face and they don't you sleep there and then they observe you and they do all these measures and that's how you measure sleep and sleep

quality, right. But if you, if you build a product that's actually helping people improve their sleep, it's just not really feasible to invite everybody over to one of those labs and just sleep there. It's not a hotel. It's like it's just too expensive to to do that, right? So instead what you can do is break it down into little things that are related to sleep quality. So you can look at things like how long did it take someone to fall asleep? Or like how long did they sleep?

How often did they wake up? If you have a wearable or so you can also say like how much movement was there or something. And then together, even though they may not directly measure the outcome goal or measure the quality of sleep, but like still a combination of these things and ideally also a combination of different types of data. So like behavioral data, but also attitudinal data. Ask someone when they wake up, how do you feel? Do you feel rested or not?

All those things combined and wrapped up in a nice little package together, like they could still give you an indication if your product is working or not. So there's one way to go about the big elephant. Is the. Big elephant, Yeah. No, I think that's, that's such a good way of talking about it.

It really, you know, something that you highlight that these days it is actually more possible than a lot of product teams think when it comes to understanding and measuring Big E as well and understanding kind of this, this Big E behaviors. And I hate to do this, but we do have actually like a a episode that all about this coming up quite soon as well, later probably this year, early next year.

So as a teacher, if you're really interested in measuring Big E and stuff, we have an expert on specifically this topic, which kind of blew my mind in terms of how you can kind of find ways to measure people. So so yeah, that is beautiful. I think we covered so much interesting stuff around this, and I think there's so much more to probably say about Biggie and Small E, But again, I'll recommend the episode with Heather Co Lewis on that.

And then again, if you want to learn about the Asian Square port, I'm sure we'll link to it in the show notes, but also you can find it on the new one's website to download. And yeah, thanks for sharing your wisdom on those topics. Cool. This has been so much fun and I

could talk many more hours to you about many of these subjects, but I want to ask you a controversial question or a conversial opinion basically because that's the theme of this season is controversial opinions about AI. But I kind of want to do a little bit tweaked version of this because I know of course that you are Dutch and you're very proud of that. And therefore, I want to kind of like tweak this question to be a little more kind of close to home for you, quite literally so.

Super curious. With bicycles, there's a lot of bicycles in in Netherlands, but there's increasingly much, much more electric bicycles. The conversion question here is like, what do you think about electric bicycles? Is it a good thing or like overrated or underrated basically? Oh, that's a difficult one. It depends. You can't say the offense that's the that's a Babel scientist in you. That's. OK, then Overrated. OK, overrated. Yeah, because let me explain.

So I never had an electric bike myself and my partner did. And I really love that because in Amsterdam, sometimes when it gets windy, it gets super windy. And if you need to go to the other side of the city, OK, for a Dutch person, just you need to bike like 20 to 30 minutes. And other people who say a tremble time of 20 to 30 minutes is nothing.

But for Dutch standards, that's long, especially when it's windy and you need to go. The Netherlands is flat, but in Amsterdam you have little bridges and everything you need to cross. So you need to go up and down a tiny bit with a lot of wind and then it's cold. So then it's kind of nice to have an electric bike to not arrive at your destination completely sweaty.

But actually we sold the electric bike not too long ago because yeah, so my partner got actually tired of it because it was like, I'm just not really moving so much anymore. Because on this bike, honestly, it's not that you really need to put in an effort. You just go quite fast without. You just need to keep the pedals going. Like it's not that you really put in an effort and you couldn't really turn it off. So he decided to sell it and just get a normal bike and actually move.

So I think they're overrated because they're also just dangerous, like you go way too fast on a bike lane where also normal people are biking. So a lot of accidents are happening with those things as well. Still, I enjoyed the time study headed so that I could sometimes borrow it. Yeah, Full disclosure, I have an actually literally bicycle. I know. I I am still on the underrated camp in some ways, but I feel like I would say the same reasons that you mentioned, but

like more extreme. Like in Sweden it's even colder, the distance is even longer in Stockholm it's like more sprawling city than I. Get that? There you go. OK, well, good to hear your expert opinion on both people science and on cycling and, and, and having a proper bicycle. So this is so much fun.

It was. I really appreciate you coming on. And yeah, it was really a blast to talk about all these interesting topics together. So thanks for all of the great work you're doing and sharing with us. And yeah, thanks for stopping by. Yeah. Thanks for having me. Quick word on Nuance Behavior where we help organizations build impactful digital products

using behavioral design. We only take on a few clients at a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our website, nuancebehavior.com. Happens to Murgatroid. The. I know that obviously you're from the Netherlands. You're a proud Dutch person, actually. How do you say Dutch? Dutch. How do you say Swede?

How do you say? Swedish person, Dutch, British. Oh, I don't know. Dutchess. Dutch Dutchess. Dutch Dutch men, but then I'm a Dutch woman. That's also weird. No, just say Dutch person.