AI and Behavioral Science – What You Need to Know

Hello and welcome to the Behavioral Design Podcast. This season we're diving into the intersection of behavioral science and AI. We want to make the sense of the state of AI, from understanding how humans interact with intelligent systems to using AI to do behavioral design itself. I'm Aline Holsworth, a health tech advisor specializing in AI and product design. Over the past 15 years, I've been crafting human centered products with behavioral science

at the core. At Apple, I LED Behavioral Science for Health AI, designing and launching AI powered features to help users reach their health goals. And I'm Samuel Sultzer, your second Co host. I'm a behavioral strategist specializing in hybrid formation and designing products that drive long term baby change. I work with leading tech organizations integrating AI to scale behavioral design for good.

And I'm also the founder of Baby Bites, a dedicated community on behavioral science and AI. Quick word on Nuance Behavior where we help organizations build impactful digital products using behavioral design. We only take on a few clients at a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our

website, nuancebehavior.com. It's good to be back, Haley. Hi, it's very good to be back. Yeah, I can't believe it's been like 10 months. What happened? I'm not who's counting What can happen in 10 months? Yeah, what can happen? What can be born out of that period? Yes, yes, about 40 weeks or so. Yeah, it's, it's, it's really good to be back. I'm honestly super happy to to get back on track with this.

And I'm, yeah, so excited about today as it marks kind of our return of the podcast, the first episode of Season 4. And there's so much to get to. We're going to do a little bit of a season preview. We're going to cover some of the biggest myths in kind of intersection of AI and

behavioral science. We're going to cover some top AI terms that any behavioral scientist should know and some other fun updates and things, but I think we should just jump straight into it and cover some breaking news. Yes, it is indeed breaking and and local to you, Sam, Did you know that very recently, right around the corner from you the the Nobel Prize in Physics was just awarded? No, this is actually Full disclosure. You told me just before the episode that this has happened.

So. So. Yeah, and we're pretending that I'm telling you now. I know. And it's funny because yeah, this is definitely news that maybe I would be expected to know before because it happened in Stockholm. But no. Yeah, big news. Yeah, it's very exciting. And the this year's winners of the Nobel Prize in Physics are notable for many reasons. One, because how did they not win the Nobel Prize before now, but also because it's it's perfectly aligned with our focus

of this season of the podcast. So the winners were John Hopfield and Jeffrey Hinton, who surely are recognizable names to all of you, no? Well, maybe, maybe for some. I think, yeah, for him to, definitely. Be more. Yeah, he's he's been more on the pages with many of these things. Is that not always the work that leads into people being well known, but more of the

controversy. So the fact that he recently stepped down from his work at Google and talked about stepping down in order to be more outspoken about the the risks and ideas of of AII think maybe made him more well known recently. But he's been known, yeah. I feel like the the media machine jumped on that, I think. I think the reality is probably a lot less exciting than the headlines would lead you to believe.

Yeah. But I mean, just to hear from you like, why do you think like this is exciting or why do you think this is kind of important to highlight their work and why are they deserving of being recognized here in Stockholm? Yeah, Stockholm specifically, yeah. So I think that the award for sure was all about their work in machine learning and, but specifically in creating or and developing and making progress in artificial neural networks

and. And this is important because most of the, the machine learning and artificial intelligence is really based on neural networks and, and this structure for processing information and deep learning and so on. Yeah, you could almost call it the foundation of machine learning the the work that they've done over the years. Yeah.

And I think it is very fascinating where we're so quick to talk about the things that AI can do and and advancement and so on, but it's easier to forget kind of what it builds upon. We'll get into talking more maybe about LLMS and and the the recent hyper on some of the innovative AI where it feels like they are weird humans because they're basically been trained on human knowledge and human human things, but they're not humans. They also they're like weird

humans in a way. The the work from John and and Joffrey, it goes even deeper in terms of looking at the brain stuff mimicking the human brain. Yeah, yeah. And, and it's so interesting, I sometimes I talk to computer scientists who who are, are very familiar with artificial neural networks and not at all familiar with the biological neural

networks. So how our brains are structured and how our neurons talk to each other and how they fire and send information with their chemical and electrical signals. And so I, I've often found myself more often than you might think, in a position where I have to explain that no, artificial neural networks are while while they mimic the brain and have many similarities with the structure of the brain, they are not the same as biological neural networks.

In fact, there are some really, really important differences between the two. So I thought maybe it could be interesting to talk a little bit about those differences and what actually does an artificial neural network look like versus a biological neural network. Yeah, I I would love for you to explain that in some ways I would trust you more than me. How would you? In this hypothetical scenario? They had to explain it to someone sitting across here.

How would you distinguish between the artificial and the biological? Yeah, well, for the sake of our behavioral scientist listeners, I'll do a little bit of the explainer on the the machine learning part as well. So it, I think it, it sort of helps to to visualize what this looks like. And of course, it's much, much, much more complicated than you could even begin to describe.

But if if you think about layers and layers and layers, think of it in three sections, OK, you have your inputs, you have your hidden layers, and then you have your output layers. So if you're like reading this as you would on a screen from left to right, you have your inputs on the left. And then you have many, many, many layers and connections in, in the hidden portion, which is often many more layers than you

could even imagine. And so that's where the complexity of artificial neural networks comes from. And then you have your output layers all the way to the right, and then data flows from left to right as you would be reading. Now, learning, especially in the case of of deep learning, happens through what machine learning scientists call back

propagation. So as data are flowing in from left to right, there's each of these nodes in the layers has a little weight where they're they're making calculations and math is happening. I think like the the most high level way to think about it is like the data flows through, but in order to learn we have to send information back through. So going back through from the right to the left and this is

back propagation. This is the really like what the the learning part of machine learning is. And when that's happening, the weights in the middle are being adjusted. So if something say say there was a calculation at some point where it was said multiply by .3 and then you get to the end in the in the output and you learned, Oh well there was some error there.

Our predicted value of 1 * .3 instead of being the prediction of .3 it was actually .5 OK, well now you have a .2 error and you send that back through the system and then in the future iteration you have a corrected weight. Is that too complicated? No, I think that's good. And like this work on artificial neural networks is what they've been kind of recognize for. And I guess again, it will be interesting then to contrast it briefly with the biological

neural network. How are they kind of similar, different, Yeah. So if all of that sounded very complex, the first thing to know about human brains is human brains are way, way, way, way more complex. Oh my gosh. So in our brains, we have billions of neurons. They're connected through these through our synapses, and our synapses make trillions of connections number. Big. Big, big numbers more, more, more than a, a machine learning

model. And I'm actually not gonna go into the complexity of neurons because that there is, there's so much to unpack there, right? But I think that an important take away is that whereas artificial neural networks can, as a result of their structure, be very good, often better than biological neural networks at at very specific tasks with lots and lots and lots of data. The strength of our biological neural networks is that they're highly dynamic.

They we, we have synaptic plasticity where we can change the strength of our connections changes very quickly and often with very little data. So in, in the case of AI, you need lots and lots and lots of data and you have very little flexibility. And with brains, you, you might have one or two observations and you learn very, very quickly

with with very little data. And this, I'm biased as a behavioral scientist, but makes me believe more and more just how unbelievably incredible our brains are. You cannot replicate a brain with math, not yet. And we're not close at all. The fundamental structure of artificial neural networks is such that they do not have this

ability to to be flexible. Yeah, I think very fun thing recently has been seeing and experiencing kind of that there's been actually a lot of people have noticed that have increased appreciation for the complexity and the beauty of the brain and how complexity is. I've actually had several people that are coming more from some interest in wanting to better understand machine learning or AI that I have started to read more about cognitive processes and the brain.

There's so much work that just looks at like just in all of the things that the brain can do, tries to mimic it tries to in some ways learn from how we work. And, and I think that's quite, Yeah, beautiful. The brain is beautiful and really, if you think machine learning models can be a black box, tell me how consciousness arises from our meat machines. It's just magic. No, but I do love that quote from from Marvin Minsky. He's one of the original AI thinkers.

You might call him that. He says the brain is merely a meat machine. Yeah, that is an amazing quote. I I want to put that on a yeah AT shirt or some form of a hoodie or a cap or something. But talk more about that quote. Yeah. I love it. I, I think, well, I think what's so interesting about it is to think about our brains in terms of machines, right?

And, and not only how we use the brain to model artificial intelligence systems, right, but we then use AI sort of the, the converse and, and technology more generally to then think about the mind. So going both ways, we both mold our computers after ourselves like like our neural networks, but then we then we can't help being influenced by our technology in return. That actually that's something that Sherry Turkle has been studying for something like 40

years, right? So basically the the advent of the computer age. Are you familiar with Sherry Turkle? No, no, tell me where. Yeah, let me, let me tell you just very briefly. So Sherry is a sociologist at MIT and she is an incredible person. I, I actually just Full disclosure, I just finished her memoir, The Empathy Diaries.

So I'm now an expert on Sherry. So I, I won't shame you for not knowing, but what was really cool about Sherry and her work was that she captured this shift really well. Computers were just coming about really like the personal computer in, in around the 1970s. Like this is like it was starting large scale, small scale, then large scale adoption where people were moving from thinking about humans as rational animals.

That was sort of what defined the human was you're an animal, but with rationality to to thinking of them as emotional machines. So in reference to the machine now, not in reference to animals. And really since then, I mean, my entire life for sure, we've had this mantra that humans are unique, where these special, unique creatures because we can express emotions and empathy. That's what's like really special about humans.

Well, consciousness for sure. But, but, but especially when we think about emotion and now we see our AI systems maybe not feeling emotion, but certainly expressing emotion, expressing empathy, making, tricking, you could say people into feeling like they can rely on this thing or, or whatever you want to call it. What does that mean? But that's a big question. But I, I know we have to have to get to our actual episode so we can save that big, big topic for

another day. But I do think that Sherry's work exploring this interplay between humans and technology really sort of captures the essence of what we're trying to do with this season of the podcast. Yeah, so you're predicting that she's going to be the next one for a Nobel Prize? I hope so. I hope so that would that would be due.

Yeah, no, but it's so true. And I I like that kind of frame of of it and AI just briefly on what you're saying, I think it is interesting that pair the original definition of Turing test in some ways like the current AI models, they are already succeeding. They are able to trick anyone to believe that they're real human beings if you're chatting with them. Like there's no way to to really know. But at the same time, as you say, it's also two things can be true.

That can be true in terms of Turing tests. The traditional one at least has been kind of accomplished, but we still probably separate a pretty big part of that to when a large langu of mole saying, if you ask them, is that OK? Can you feel feelings? And do you feel sad? And like, if it would say like, yes, I feel sad and feel lonely and so on, we would picked out with a bit of a pinch of salt in terms of what what actually that means.

And, and we won't kind of I I guess there was some of these news a couple years ago with some people being like, Oh my God, I asked to shut sorry, I asked AI this and they said yes. And I'm, I'm afraid. Well, I'm pretty I I think we have a little more nuanced approach these days. Yeah. And I think you and I know quite a bit about this intersection of behavioral science and AI, but

we're learning too. And, and honestly, so much is changing every every week, every day, that just keeping up on what the literature and what's published at this intersection is could be a full time job for us. Maybe someday it will be. But for now, for now, we'll we'll keep our side jobs. And I, I guess that brings us to the season. Do you want to tell the audience a little bit of what the plan is for for this season?

Yeah, we're, we're really trying to cover everything from understanding how humans interact with intelligent systems to using AI to to do behavioral design itself. So how do we use it in our work? Yeah.

And I today I feel very related to I felt about 6:00 or so years ago now when I initially started have a weekly where I felt with behavioral design, which is really, really hard to stay on top of what's going on. And as a petitioner to kind of, yeah, just feel like you are in the loop of like, what are other people doing? How are other people thinking?

I think right now it's a really triggered time for anyone as a be able scientist to really make sense of maybe both from identity point of view, like who am I today? But also just feeling kind of more on solid ground when it comes to like AI, Like what are useful things for us as petitioners to know? What are things that are already people are doing and what can we learn from each other and, and really kind of do that openly and and collectively?

I think that's kind of where we're coming from, I guess. Yeah, I think, I think we can do the work of sifting through everything that's put out there, not everything, but many things and share what's relevant and interesting. That's that's really the goal. So we can be as useful as

possible. Yeah. And I think from my end, if I'm completely honest, putting all the cards on the table, I personally would love through decision to have a better personal argument for used the business case as well around payable sense and AI, 'cause I have a lot of opinions around the importance of understanding that AI is not a form of panacea or silver bullet or magic wand to solve a lot of things that it's being used for.

And we've already had some, we should teach this, but like some really good recordings of, of guests episodes where we've already explored the importance of thoughtfully kind of marrying behavioral science and AI at the right places and, and the really big benefits that gives. But I'm really excited to to better formulate that argument and that thesis of sort so that that doesn't get lost. Yeah.

I think it's easy to find yourself on either of those extremes where you're jumping on the bandwagon with everyone, everyone and you're all AI everywhere. Or you maybe you find yourself on the other end of the spectrum where you say everyone is doing AI. Like I just being totally dismissive and I will be honest, I, I have been that person on that end of the spectrum at at points in my life. And I don't think that's the

right place to be either. I think that finding the right balance and understanding where is AI useful? Where is it actually the best tool? But yeah, how do we actually use this most effectively and just understand its impact on our science and on society? I think that is just absolutely essential. Hey everyone, this is Maddie, your AI producer of this episode. I just thought this was a good opportunity to jump in real quick.

We really value listener involvement this season and we would love to hear your thoughts, ideas, and reflections as you listen along. So if you've got anything on your mind, topics you want us to cover, guests you'd love to hear from, or anything about the intersection of behavioral science and AI, feel free to reach out. You can e-mail us at podcast@habitweekly.com. We read all your messages and and really want to make this season as relevant and valuable as possible for you.

Again, e-mail us at podcast@habitweekly.com. Now let's get back to Sam and Elaine. Wow. So yeah, I'm really excited for this season and maybe also to share, is it useful to maybe talk about where we're coming from, like both you and I and all of this? Yeah, yeah, yeah. So we did mention that we have some expertise in AI or this intersection between behavioral science and AI.

So I think it's useful to sort of lay down our cards and say here's why you should trust us. So I, my life's work has really been in helping people reach the goals that they have often their health goals, not always their health goals, but you know, how do we, how do we help people exercise when they want to exercise and so on. And so I've been doing that, the digital health space for a long, long time.

And then it may be in the past five years or so, really got interested in AI, specifically in terms of reaching these same goals. How do we, how do, how do we leverage new technologies to do so? So then I joined Apple. I thought this would be a, a good way of achieving this goal. I LED behavioral science for its health AI team. So that was a, a really fun experience working on high impact features, working with fitness, plus really, really working at this intersection of

AI and behavioral science. And one thing that I should mention is that before I entered Apple, I really had very, very little trading and machine learning and, and the artificial intelligence. So I, I sort of took it upon myself to dive head first into this whole world and Oh my gosh, like what a world it is. It's so, so wide and deep. So I, I read books, went to conferences, took courses, like, I even learned how to code a bit myself.

I mean, don't ask me to actually produce something, but let me just say that I, I feel that now that I've gone through that experience, I am, I am a much more humble person when it comes to realizing how much I don't know. And, and I also, I feel the a little smirk every time that I see people on the Internet talking about AI without really knowing the first thing about it. And so I feel like I was that person and I can see what they're doing, but I can also see right through what they're

doing. So I think that, yeah, our our goal of this season is definitely going to be to expose you and ourselves to the people who know quite a lot about artificial intelligence and behavioral science. So. But with that, Sam, tell me about yourself and what's what's your cred, your street cred in this world of? Artificial intelligence, it is hard to compete with being the behavioral science lead at health AI at Apple. I think it's hard to.

Yeah, it's not a competition. No, but I think it's the same, but different in some ways. And I think that's why it's always a little bit interesting because I had quite of a different entry when it came to engaging with AI, where this was when the GPT 3A model from Open AI came out. So that was I think a year or so before. Yeah, somewhat old school, but it was really the model that where you start to engage with it and you're like, wow, this is

pretty cool. Like you can actually do quite a lot of things with it. And, and so it's I think a year or so maybe a little bit more before chat TPT came out and very randomly I was working with some client projects where that started to be used and and, and started to be considered and, and I was part of really leading some of that and, and working with that. And so I, I was tempted to become this kind of people that do the red teaming and and so

forth, testing these models. I didn't end up going that far in, in kind of working directly with developing the models, but I, I feel like I quite quickly got a very intimate experience with, with working with them. And so that's a lot of the work that I've been doing. So maybe less on, I would say you definitely come from having much more of AI would say rich machine learning, understanding of, of AI.

Where a lot of my work has come in on more of the generative AI side where it's been trying to use these models to, to make sense in some form of the kind of multimodal aspect of either kind of making sense of kind of images through using AI or creating text or understanding text or creating personalized interventions and and so on with these models and using prompt engineering and fine tuning and dragging and some of these things that is coming out to kind of improve the consistency

and reliability of them. So. Well, and just to give you a little compliment here, I would say you have mastered the art of prompt engineering, even in just our our work doing like background research and that sort of thing. I will throw something into ChatGPT and and share my results with you. And then you go and hack away at it for a couple minutes and you come back with the like shiny emerald version of what I had.

It's just I've been awestruck by what you can do just just by your experience tinkering around and really exploring these these getting your hands dirty with with these technologies. Yeah, yeah. And then I, I will definitely be honest with that. I have done both to that degree, but also I think recently the last year or so, I've been really excited about the agentic side of what is happening in terms of what we're, we're going to be exploring that a bit.

And when we mean when we say agents, what we mean is basically that you have some form of AI set up to be a a multi purpose tool of sort where it can do multiple things for you depending on the task you're kind of asking it for you. Essentially automating your workflow. Is that what you're talking? About yeah, for example, yeah, it's a lot of automation comes into kind of like using some

form of agentic things. And, and so yeah, I've really become yeah, very deep into that space of like using AI for

automation purposes. And, and really, I think I find it so interesting to understand just when it comes to automation, how again, the human aspect were it's easier something done to automate things because even though you kindly could automate it, there's always this explicit things that needs to be done, but also implicitly things we don't think about what we're doing and, and how how that

plays into things. So, so yeah, that's a long, long with a way of saying that I, yeah, I, I've had my fair share of exposure to, to AI, but as you, I feel also very humble. I feel so much so much that I feel still that I, I want to know more about and understand better. And, and I think that's, that's also where I've really enjoyed our private discussions. And I feel like I've learned so much from from you. We're kind of privately talking

about some of these things. So, so yeah, I'm really excited to have this season up and running again and having this recorded conversations too, and. Yeah, I, I feel like I have to mention how our, our current professional lives intersect with this as well. Because in, in the in the meantime, since I guess since we've since we've last released an episode, we have created a company together with some friends nuanced behavior And at that company we are working on

advising on AI driven tech. So this is exactly what we do, sort of injecting behavioral design and behavioral into products that use AI. They don't have to use AI, but many of them do. Yeah. And I think that's what's interesting. We, we ended up calling this nuanced behavior and it's easy to kind of find yourself using the word nuanced quite often. But it is where we're coming from where we have a lot of experience when it comes to applied rural science.

And it's been really interesting to to meet a lot of clients that maybe have a really strong capability when it comes to some of these aspects, potentially even in AI in some cases, but also helping them get the most value from the behavioral science side. Firstly, when understanding the human, like how do we really support humans in leading better

lives? And whether it's sustainability or in health or in in more financial related context, how can we can we both with the technological understanding and capability, but still not forgetting that in the end, we have to understand the human side of it and we have to make the most of that. I think that's been really, really fun to to complement with an income really from that side as well. Yeah, absolutely.

So given that this is our intro episode to the season, we thought that it might be useful to share some overarching themes about AI and behavioral science, share some terminology that's going to come up throughout the season and really kind of get into what are the myths out there? We can, we can sort of debunk those those myths and set the record straight. And I thought that the, the version of this that we could do is AI1O1 for behavioral

scientists. And, and I want to do this in the spirit of a, a course that that existed back when I was an undergrad at Reed College. It was called physics for philosophers. And you can just tell by the by the name of the course, this is not your hardcore physics. So this is our version of AI for behavioral scientists, just to give kind of a general what's what.

And I think there are some really basic basics to know about AI for, for the very beginner and, and in terms of just how we think about AI, of course, we've already spoken a little bit already about about neural networks. But if you think of I, I think at their very core AI systems are and in and in particular machine learning systems.

I'm not even going to get into teasing those two apart, but if you if you think of them as super powered prediction machines, right, you are taking data and from that data finding patterns in the data, often massive amounts of data, and then categorizing that data, classifying it and making a

prediction. Yeah. I think that's a really helpful way to simplify it and really think in my asset super powered prediction machine also reminds me of this conversation we had at some point with Lisa Feldman Barrett about emotion and so on where she talked about us and how we engage and understand our emotion, this kind of prediction machines. And then it ties into this whole thing that that episode started with and and that of course we

are also prediction machine. But I when we're talking about artificial system, that's, yeah, narrow, narrow pitch machines. Narrow pitch machines? Not so. Good at common sense or information transfer or or really generalizing at a conceptual level that that's a human thing. For now, SIM, what do you think are the term the absolutely essential terms that our listeners have to know in order to understand any conversation about AI? Where should we start?

Yeah. So let's start sharing maybe three key terms that I think is helpful to have heard of and have some understanding around. So I think let's start with large language models or LLMS. So when people talk about degenerative AI or Gen. AI, in some ways, they're often referring to large language models, since these models are what typically drive any type of generative AI content

generation. And so when it comes to LLMS, the more popular ones or the, I would say the ones that maybe even some of our listeners have been using or testing, obviously you have open AIS GPT 4. And important to understand is that these models, you should have two components. They have one maybe shot version, which is like shot GPT and so on. But they also have AAPI component, meaning that they can be used directly within others

softwares or in other products. And so when it comes to opening the eye, a lot of people think about shot GPT, but they underestimate the fact that their underlying GPT 4 O, for example, API is really what they're often times interacting with in many, many places outside of directly shut GPT. And so then we also have Meta who have their Llama model and that's quite popular for more kind of open source use cases and Entropics Cloud or their various cloud models.

And they are probably like the biggest direct competitor right now to open AI in terms of exactly how they're trying to structure their their models. And so they're all similar in that they all generate content and they've been trained on insane amount of data. Like they've been trained on so much data from everywhere from Wikipedia to Reddit.

And it's, yeah, it's basically a large degree of whatever we have available to us from whatever has been recorded within human knowledge online is being used and has been used to train many of these models. And so when we talk about them being kind of weird humans in some ways, that's why. It's because they are trained on things that we have written on Wikipedia or Reddit or in books and and so on.

And so overall they are designed to again create text, but they've also become multi modal in recent years, meaning that they can often times handle images, audio and even video in some cases. And so that's why they're also becoming kind of increasingly powerful because, you know, you can use the same model to basically look at what you see in front of you. Like you can take a photo of what you see and you can ask a model what it, what do I see?

And then it can give you a response in text or an audio form. And so that can be extremely powerful in many contexts. And we see now with the versatility that these models are actually able to perform about 50% of tasks done by knowledge workers. It's kind of estimated. Obviously, this is when the LLMS are set up correctly and the tasks are like explicitly defined in a very, very clear and simple way.

So, so yeah, they're capable of everything from summarizing to generating code to even creating personalized interventions and and so on. And what kind of limits their power is often times based on the input they're given. And that brings us to prompt engineering, which is the process of guiding these models to generate useful and consistent results. So for example, if you simply ask the model, like, hey, write me an e-mail, you will probably get something very generic in return.

But if you provide the in the prompt some clear context, like write a formal e-mail to a new client introducing our services and so on. And then when it comes to prompt engineering, often times it goes much, much, much deeper. It's often times looks at the idea of providing multiple examples of the output that you want. So multi kind of shot prompt or non non zero shot prompts.

So it can also include some form of chain of thought prompting basically where you're kind of trying to get the model to a review and reason and and think about what is trying to provide before actually gives it as an output. And so hopefully when done properly, you'll get something much more relevant this way. And I feel like a lot of people, I guess, get disappointed because they don't realize that even though these models are very powerful, they still need

clear instructions. And we often talk about the idea of kind of having 1000 AI interns because that's what often feels like is that they can do a lot of things. And they are like, you know, with scale, you can do so much at same time. You have to keep things simple and be very clear about the context, and vague prompts honestly lead to vague results or generic results and so on. And that's oftentimes the first step to work consistency.

And there's always more advanced kind of steps to take. We won't get into this too much today, but you can use things like RAG retrieval, augmented generation, or RLAHF, meaning reinforcement learning with human feedbacks and so on. There's a lot of things that could come into play to fine tune or to improve or to get better kind of content that it retrieves for its output. But yeah, we'll, we'll, we'll

want to get into those. For now, we'll probably go cover them more in this season, but yeah, finally the last term so I covered. Large language models the role of prompt engineering and then AI agents. So AI agents are autonomous systems assigned to perform tasks without needing constant supervision. So one example is having some form of customer service chat bot that handles some form of routine queries. Another one is setting up an AI agent to automatically draft

emails for you. And so the difference here with an agent is that you can set it up so that it knows how to kind of perform certain types of writing styles. So it knows when to adjust the tone and style based on the recipient. So that when you draft an e-mail to a colleague, that is different to the e-mail that's drafted for a client or your partner and so on.

So that's increasingly what my work has been around, you know, working with these kind of AI agents combined with, you know, prompt in earring and so on. And they can really create some powerful, often times automated systems or when when you talk about automating workflows, they become extremely powerful. And yeah, the these are the three key concepts that I guess

I wanted to then start with. So we've covered large language models, a little bit of intro of what they are and some of the different models out there, the importance of prompt engineering and how it's just not the same as just writing something into the model. It's about kind of ensuring consistency and then how you can scale some of this work when you're bringing AI agents into to play.

So together they enable us to kind of optimize and scale and, and automate the tasks in a way that I think, you know, just a few years ago was completely unthinkable and, and something we never imagined would be possible today. So yeah, but there's obviously much more to talk about here. And so maybe Eileen, you can share something more maybe on recommender systems or I briefly mention reinforcement learning. And yeah, what would you want to share with with people?

Yeah, I would love to. I think there are really three major categories. Recommender systems is 1, Reinforcement learning is another. And then I feel like it's easy to forget because we we don't see as much of this in the at the intersection of AI and

behavioral science. But there's also just our sort of classical machine learning where you have a bunch of data and here you say you train your model on a million cat pictures and then you have some new data and you want to learn is, is this photo of a cat or a not cat? You can with fairly fairly good accuracy, understand with many of these classic machine learning models, whether a photo

is a cat or not cat. And I'm, I'm borrowing from Cassie Kozikov's famous cat, not cat example because I really admire her work. But but that's, that's sort of like the most basic version of a of a classic ML model. The the other categories that you mentioned, recommender systems, we talk a lot about recommender systems in our episode with Carrie Morwich. And so, so you'll have a, a recommender systems one O 1 in there.

So you'll wait for that launch. And then of course, we have reinforcement learning, which we really dive into with both Amy Bucher and Susan Murphy, and we really look at the potential for using reinforcement learning within behavioral science as a as a method for doing behavioral science. So yeah, that covers some useful terminology and we'll attach this in the show notes as well. We can refer to if, if in doubt, hopefully that's going to be a little helpful.

But then we also wanted to cover a bit of myths because I think we both feel like there's quite a lot of those going around when it comes to talking about AI and especially like this intersection of behavioral science and AI. And I think 1 myths or slash like oversimplification that I feel like I hear a lot is just really equating intelligence in

the same way. Like you have a large language model that is taking an IQ test and then it's very easy to say, oh, it's as intelligent as a human being. So that means that it's as human level intelligence. And that is true. But I think a lot of people also don't extend that to say that it's human like. And I think again, it's really, really, really, really important to not let us may be seduced or confused or made us to think that they are one of the same because they are again, really,

really different. I think as I'm seeing the data is coming out, how impressive some of the models are now at performing certain tasks, at providing certain types of expertise. It certainly is leveling or exceeding human level intelligence in some areas in. Some. Areas in some areas, in some areas and it's still a very, very, very big difference to what is AGI like what, what does it look like, but it we're still so. Far say what AGI means for our.

Yeah. So artificial general intelligence, you actually referenced the study of narrow intelligence because you got things, important things to really hit home that we're still in a very narrow set of intelligence where, OK, these large language models can operate and do a lot of different tasks and a lot of different things, but they're still very narrow.

That's why we often talk about having 1000 AI interns and this idea of generalized intelligence where they can highly and reliably perform in a lot of general contexts and really understand what we know as human beings have like human like intelligence. It's still far away. And you're right, it is extremely seductive when you see that oh ChatGPT passed the bar exam or ChatGPT outperforms physicians in XYZ. But you know, then the next second fails. A very basic cognitive

reflection task. Or are you familiar with the Nick Bostrom's paper clip? Example, yeah, coming from effective ultrasound, that's like one of the classic. Yeah. So this thought experiment is, say you give an AIA task and it is maximize the production of paper clips, then maybe the AI with access to all the resources that it needs can maximize paper clips. And if that's your goal, maybe that's a good thing. But of course, there are always unintended consequences.

And when, while maximizing paper clips, you, you might find that there that there are actually other goals that you want to have in mind as well, such as not killing humans while you're at it. And and so then you might tell the AI, OK, but in, in order to maximize paper clips, please also don't kill humans. OK? So like it's, it's given that direct order, it'll do that.

But then you think about all of the other indirect ways that you might kill humans by destroying the environment by by by excavating all of the resources that you need in order to maximize paper clips. And you can sort of go on and on and on with all of these edge cases. And you can tell an AI system to directly do all of these things. But the the contrast is for the human, you don't you, you it's, it's so, it's common sense, right?

It's so obvious that you don't want to deforest the entire planet in order to maximize paper clips that that that's not even something you would have to consider saying. However, if your only goal as an AI system is to maximize paper clips, then you might you know, you don't have that consideration. You don't have that human element of of common sense to guide your decision making. You only have the inputs that you're given.

Yeah, and I I mentioned this and I think that's speaks to a lot of people's experience where I think it's a little bit sad in some ways as well. But like when they for the first time they open shat GBT and ask it something and they just get like a really shitty response because these assets something very generic and simple and we get some kind of generic and simple back. That's me if you're describing me. OK, OK.

But in that case it is easy to feel like, OK, this is shit, but it again is because these models and and so they're currently quite far away from real understanding the context that we're in. And we we're doing our best to kind of kind of communicate and, and make the context come across in an efficient way, but it's still still very narrow in in nature. So yeah. OK. So what, on your end, do you feel similarly about as a myth? All right, so we have human

level is does not equal human. Like that's our myth #1 and you just mentioned Gen. AI, so I wanna throw in a myth that I think exists about Gen. AI, which is that it boosts creativity. And this is this is a really fun one for me because some research recently came out that explored exactly this looking at OK, it's have have some users ask them to create a story and either give them access to ChatGPT or not.

And then just objectively assess the, the, the products that they come up with, how creative is the story that they write? And it's really interesting that giving access to ChatGPT does increase creativity on average. So across the board, if you, if you rate the creativity of these stories, it seems like, yeah,

OK, that that's a that's a plus. But then if you actually drill down into the the stories, if you look at people who were more creative at at the start, the sort of your high skill workers, they really didn't benefit from ChatGPT at all. This lift was really can be contributed to helping the those who were not very creative or or or not very skilled in writing in the 1st place. So you actually find that that the ChatGPT is much more helpful for for less skilled workers.

This is this is kind of a another myth in itself. But another side effect of this actually is that even though the, the, the stories were more creative overall for the, for the individual users, if you look at the content in a collective sense, the diversity of stories is actually reduced. So everyone is kind of getting the same ideas from Chachi BT and they're all converging on similar stories.

And that to me is not a good thing that that that is really just making the the content in our world much more homogeneous and much less interesting. Yeah, and I can speak a little bit just based on just having a partner who's a motion designer and and artist coming from an art background.

It's really like I, I see the frustration when she's trying to use some of these tools because like she either has the option to do it herself in a full control and rely on her own unique creativity and so on. Or it's just kind of like almost tie an hour behind her back and be kind of reliant on a shitty version of that through some of the quote UN quote creative tools that can't really do things as well as she can. And where she doesn't have full control.

And she's kind of have to be like, no, I didn't mean in this way or like, I mean in this way. And like, no, like this is what I mean. Like instead of actually just being able to, to do it and just having the control. And so while someone who has never created some form of a digital artifact, we can now do that. And that's really cool. It is also, as you said, like it's really important to understand like what, what are we really risking if we throw away the the people who do it

really well? And yeah, I think we risk a lot of lot of that collective collective gain or collective good. OK, so myth #2 Jenny, I doesn't boost creativity overall. Yeah, I would say that let me share one pro tip, a personal Aleen pro tip and I think this so this is related to using ChatGPT for for creative tasks. Some advice that I hear out there is, oh, use it to brainstorm and get ideas and like, like basically use it for your first draft of something. And I think that is a terrible,

terrible, terrible idea. I think you absolutely have to use for all people use your brain as the first draft. If you need help structuring it, clarifying your ideas, seeing what you've missed, fine. But to, to use ChatGPT as your first draft, then, then what do you have? You now have a new status quo, which is this fully fleshed out report. And one thing we know as behavioral scientists about humans is we we are attached to

the things that we own. So it's like this is the the endowment effect in in play right here. We don't want to lose by editing, getting rid of the suggestions that ChatGPT made. We don't want to lose that content. Yeah, you're, it's a great,

great, great, great point. I think you as you're like alluding to like you basically sell yourself short by just molding yourself by the most generic responses given to you by Shajibati. It's really both important not to let these models and so on think for us too much like it's really useful to get feedback and, and and expand our thinking and test our thinking about not to do anything.

And, and sadly, because it is actually what I've noticed myself, because I really agree with your advice, but at some time also noticing that it is tempting to start also being like, hey, give me 4 ideas, give me 4 things. Give me, give me. But instead of jumping that straight away, taking a little bit of a moment to reflect of again, what do I think, what are my thoughts, what I believe, what are my ideas? And then starting from there. And then you can always like build on that.

I've never seen that not being the best way for like, I've never seen the way where I've started what he is like asking JPT to just do something for me and not in the end I feel I've wasted 30 minutes where it just giving me some generic stuff where I have to. In the end you start over and, you know, give it what I actually need. All right, so that was myth #2 Gen. AI boost creativity. No, it doesn't. Myth busted. OK, next one.

So I think there's this this tendency to talk about AI as being this huge risk in terms of user privacy. And I think that this misses some really important nuance. And that is just that not all AI is equally unsafe. It's not all equally non private. And, and, and really you can create models depending you being a company, a product manager, whoever's in control of the AI product, you can, you can create it differently depending

on your privacy goals. And some companies have really made a real effort towards towards using and developing privacy preserving AI. And this is a has become a whole subfield in itself. And you can look at what does that mean for maintaining privacy in your training data, in your input data and your output data. And one pretty popular way of doing this is called Federated learning. And this is basically just a version of on device machine learning.

So the data never leaves the user's device. It means there are some implications of this, some downsides. You might say the the AI models have to be smaller. You can't fit ChatGPT on your phone. ChatGPT is extremely energy inefficient, so you can't do that. But your, your data never leaves your phone. Like that's a beautiful, important thing. And, and I feel like that is not talked about maybe as as much as it should be. So your, your personal data is totally secure. It is.

It belongs to you and it isn't given to anyone else. Yeah, I think that also looking at comparing the large language models that are open source that you can kind of self host and and store locally compared to the ones that you can't like in the case of opening eyes or and tropics. The the difference is becoming I think less in many ways.

And yeah, it's also both for the bigger models, if you wanted to use them with your company data and so on, they can certainly be self hosted and privately stored and nothing leaves the company. But as you said, like also what what is exciting is this kind of like increase kind of promise of these maybe tiny models that can exist even on the device. When I think that's kind of where, yeah, you, you obviously you can't maybe do exactly all

of the things in the same way. But as you say, like it's you can do a lot of things, yeah. A lot of things, in fact. That leads me to our next myth, which is that more complex models are better because they often are not. There are quite a few downsides to having more complex models, including their lack of transparency. So when we talk about the black box, the bigger and more complex your model is, the less explainable it is.

So you don't have what we call interpretable or explainable AI where you can say here's what the model is doing. So if, if you have a really big complex model, you can no longer do that. And that's that there's a very clear downside to that. If you don't understand it, you, you, you not only cannot tell your users what's going on, but you yourself don't know what's going on behind the scenes.

And so that there's some obvious, some very real and obvious dangers there, but also the, the plastic downside of complex models is that you have a risk of overfitting. So this, this is what happens when you say you, you train some, a model on some piece of data, but you train it so closely to that data that, that it, it can only predict that data. When you then send it out into the real world, it does a, a pretty poor job. And this is overfitting to the training data.

And then of course, more, bigger, more complex models are slower, they take more energy. So if you don't have to do that, if you can use a some a linear regression model, then do that by all means. Yeah, yeah, 100%. I think that's so true. And yeah, we covered quite a bit of myths there. Any any final myth or a little thing that you wanted to mention before we get to our final

showdown of this episode? I think I would only say one final thing, and this is going to have to be an episode on its own because it's such a huge, huge topic. And this is one that Sendhil Mulanathan has, has really tackled. And this is the idea of algorithmic bias and human bias and, and sort of comparing the

two. So the myth is that algorithmic bias is worse than human bias because in in the public conversation, we talk all about how terrible biases in our algorithms and how AI is so biased and so on. And Sendal makes the very good point that humans are very, very hard to change. Algorithms may be hard too, but surely not as hard as changing humans. And so that that that has been something that I think about a

lot. And obviously work has to go into making algorithms non biased or biased in the in the correct way, right? Maybe correcting for bias out in the real world, but in in terms of the lift compared to changing humans, as behavioral scientists we know just how difficult it is to change people. So if we can just change some math. Not to understate the difficulty, but surely relatively much easier.

Yeah. And yeah, Sandhu, if you're listening, let us know and and we'll book you in for episode. Awesome. So finally, we have a new quick fire round for this season. It's called to AI or not to AI. Basically, we're really interested to hear from my guest in terms of their thoughts about really using AI or not in the day-to-day or their work. And so we're kind of providing with some tasks and asking them, is this something that you think AI should do that it's well suited to or not?

And so for this kind of first episode, we thought it'd be fun to use. Yeah. Ask each other a few of those to warm, warm ourselves up. And also just, yeah, we've kept this secret so we don't know what the other person can ask. But I have a few ready for you. Eileen, I'm. Ready. OK so my first one for you to AI or not to AI let your Tesla drive your son to daycare on its own using autonomous mode. Absolutely not. Never, never.

Or like another. Really hard to imagine a time when I would be, Yeah. I mean, he certainly won't be in daycare, but he probably won't even be in school by the time that that autonomous vehicles are ready for that. Even if you knew that you were probably on average a worse driver than most autonomous vehicles, yeah. Would you still? Would you still wanna do it yourself? So I probably am a worse driver. That's the reality, especially in the Tesla's too big for me.

But yeah, I don't trust it. I've had too many incidents where it does a crazy thing and you have to you have to take over. And it doesn't feel like the technology is moving fast enough in order for me to trust it. I want so badly for that day to come. It just doesn't feel close at all. So maybe my answer is eventually, but not before.

My son is an old man himself. OK Sam, for a podcast on behavioral design, generate guest BIOS and episode summaries and then combine that into a script for hosts to read hypothetically. Hypothetically, some of it, yes, some of it, no. Maybe. I do think there's amazing things you can do today in terms of if you have a very strong and clear understanding what the output you want to be like, let's say like a bio or a transcript or various things that you want to include in your

podcast. Definitely that can be done. And maybe it's being done for for this podcast. But in terms of like providing the extension of a longer script or like actually having some form of artificial holds that would do some form of thing, No, not not that, but but yeah, introducing the guests, yeah, that kind of stuff, yeah. Interesting. OK, OK, my turn.

OK, Eileen, to AI or not to AI? Create a virtual version of yourself that speaks Swiss German really well to provide care and companionship for your older relative. Aw, that's would be gosh, I have such mixed feelings about that because. They can chat with someone that can maybe. Well, yeah, it's it's so interesting. In the general case, I feel pretty strong anti anti feelings to not AI.

However, in this specific case, Mike Grossi is 90 something years old and it's not like I'm in Switzerland with her. So it's not replacing me. I'm not there. And so, gosh, wow, you, you really made made me see the flip side that I hadn't seen before. OK, I'm going to say I'm willing to try it, but now I can't let any of my relatives listen to this episode. I'd like to oversee it for a few years at least. OK. OK. To AI or not to AIA marriage counselor? Actually, it's funny because

that is my next to you. I have to not to AI have AI relationship coach analyze your marriage and suggest improvements. Yeah, I would say to AI actually, I would be quite open to to getting feedback and yeah, ideas from from an artificial marriage coach I think. I think so too. I think so too because I think it's easy for you as a couple to discredit the AI suggestions if you don't like them. Easier than a person, and maybe it'll bring you together to have a common enemy.

Yeah. And it's it's probably also where it's one of the cases where at least initially short term, it would be easier to probably be able to talk about anything and be more open. As long as you feel like your privacy is protected, then yeah, I wouldn't mind trying it in terms of, and I would say this is a good example where I do think long term, even though it could perform as well in terms of use, like task for task perform as well.

There is I think some inherent value in actually having a human being that you feel accountable to and that you trust and so on. That's kind of there to support your relationship that will make you work a little bit harder, be a little more engaged and so on.

Whereas you might not be as motivated to show up in time or like do whatever thing you need to do and you are starting less seriously than if you actually have to shop for a person and pay for it and so on. So I think there is some part of the traditional setup that I do think it's hard to replace in

that sense. So this one is a little bit longer, but again, to AI or not to AI to use some form of AI that tracks your day through, let's say your smartwatch or smartphone or maybe smart glasses. So it tracks your day. And then at the end of the day, each night, it gives you feedback on whether you lived up to your best self, like whether you like, you can set the standard in some ways, like this is who I want to be. And then every evening getting a little bit of like a mean a

report of like, hey, we'll talk. Eileen, here's how you've been performing today. I think if I would be able to set the seriousness of my commitment to that goal and also not get that report on a daily basis, I think maybe less frequent. I could handle like a weekly check in or even a monthly check in and then at those check insurance, set new goals for the next the next time period. Yeah, I'd be up for that. I mean, if I'm controlling everything, sure. I don't, yeah.

And and it's, I think I just say, I think it, I would definitely be more up to it also like on a monthly basis, but like on a daily basis, like I think eventually, because it's very uncomfortable to be putting that pressure on yourself. And I think that's what we're doing as society today. And that's why I put it like it's more extreme to that. But yeah, I'm sorry. What's your What's your final? My final one it it's related to my your your Swiss question, but

this is in the afterlife. So to AI or not to AI your great grandfather. So for example, in a virtual reality world, you have a fully formed great grandfather who walks like him, talks like him, you can talk to him, you can play catch with him, you can visit him anytime you want. Not really. No. Yeah, yeah, it does Like this so much. Speaks to all of my sci-fi interests in terms of like Westworld and and there's an episode of Black Mirror that's kind of mirrors this concept and.

But no, not for, not for me. I do think it's nice to be able to let go and to, yeah, to. Maybe not nice, but important. Important, Yeah. It's important to to be able to let go. And it's important to in some way, I feel like it also diminish what actually was there before because you can like holding on to some form of fragment of who they were or who they are, but it's actually not. And it's not actually them. Yeah. Yeah, this is this. This reminds me of since I'm on

a Sherry Turkle kick. She tells a story about meeting a virtual Steve Jobs and she's done. She's done some more recent work on artificial intimacy and looking at people, for example, who have created a version of someone that they knew who is who is passed on and they want to continue interacting with them. So she met a virtual Steve Jobs as part of this research and she actually knew Steve Jobs when he was alive.

And her experience with the virtual Steve Jobs was, oh, like maybe they uploaded his mannerisms and everything that he's ever said. And they have this, the wealth of everything on Steve Jobs. And he's a public figure. So there's a whole lot of information about Steve Jobs out there. But her experience was like, he would never say that this is the, like, there are some things that maybe feel like a likeness, but he would hate the virtual Steve Jobs. Like, he would not approve.

Yeah. Fidelity, it's hard to, yeah, establish. And yeah, I do think it's interesting that we're both kind of shows a lot of these questions that centered around significant relationships in our life, like our connection to our kids or elder relatives or partners, and I guess ourselves as well.

And I do think there's, I'm excited for this season to talk about how AI can be in terms of either a tool or convenience when it comes to recommending certain things regarding, I guess classic stuff like recommended mention components or recommending TV shows and so on. I think that is interesting. But I do also think it's, it's super interesting and profound to think about how AI can influence who we are as, as people. And kind of it does confront this notion of who are we as

humans? What are we? What is it to be us? What is it not to be? What is it be artificial? And, and how do we grapple with that? And I think in some ways I think that's, that's really real, especially as we starting to use AI to be redefined as we have replica replicating partners, we have a replicating, you know, customer service agents, these things that is trying to imitate and as you say like creative

jobs or whatever. So it does gets to some very deep and human questions and I I think if we can become a little bit better understanding of how we can relate to those and how we can kind of, yeah, feel comfortable there, I think that's a big win as well in this season if we can approach on this thing. So yeah, just a final reflection on my end, but I think that's super interesting.

Yeah, I mean, all of these questions are just so perfectly suited for behavioral science and just to toot our own horn like that is why we have to do this. I remember early on we were, we were sort of teeter tottering about whether we really wanted to do AI because of the hype. We felt the backlash and we decided that it would be irresponsible of us not to do this. So that's that's why we're here. Yeah.

And we're excited to share this with you and, and we in terms of listeners and we have a e-mail address which is podcast at habaweekly.com. And we're very, very keen to get your thoughts, questions it asks people you want to hear from, questions you want to get answered. Send us all of that and anything or on that e-mail. So podcast at habaweekly.com. And yeah, to wrap up this one, get excited. We have the first interview

coming out next week. We have some fantastic, honestly, I can say because we're recording them already, some fantastic episodes that are already lined up for you. So yeah, hopefully this was a little bit of a primer and yeah, time to get excited for some some really interesting conversations on paper science on AI and and beyond. So so yeah, I'm excited, Eileen. I'm excited.

Yeah, and that's a wrap. You've been listening to the Behavioral Design Podcast, brought to you by Habit Weekly and Nuanced Behavior. Sam and Elaine tell me This season is packed with incredible insights about behavioral design and AI, so be sure to subscribe and share the podcast with your friends. Though you might want to keep it away from your enemies. In case you haven't noticed, I'm an AI voice. Yep, pretty crazy. Quite the improvement since last season's AI outro, don't you

think? If you'd like to collaborate with us at Nuance Behavior, where we use behavioral design to craft digital products with Nuance, e-mail us at hello@nuancebehavior.com or book a call directly on our website, nuancebehavior.com. A special thanks to the amazing Dave Pizarro for our show music and to Mei Chen Yap and April English for their help in producing and publishing this episode. Thanks again for tuning in.

We'll be back soon with another exciting conversation where behavioral design and AI intersect. Happens. To. Mugatroid. The. The brain is beautiful and really, if you think machine learning models can be a black box, tell me how consciousness arises from our meat machines.