Sci-Fi and AI: Exploring Annie Bot with Sierra Greer

Hello and welcome to the Behavioral Design Podcast. This season we're diving into the intersection of behavioral science and AI. We want to make sense of the state of AI, from understanding how humans interact with intelligent systems to using AI to do behavioral design itself. I'm Aline Holsworth, a health tech advisor specializing in AI and product design. Over the past 15 years, I've been crafting human centered products with behavioral science

at the core. At Apple, I LED Behavioral Science for Health AI, designing and launching AI powered features to help users reach their health goals. And I'm Samuel Sultzer, your second Co host. I'm a behavioral strategist specializing in hybrid formation and designing products that drive long term baby change. I work with leading tech organizations integrating AI to scale behavioral design for good.

And I'm also the founder of Baby Bites, a dedicated community on behavioral science and AI. Quick word on Nuance Behavior where we help organizations build impactful digital products using behavioral design. We only take on a few clients at a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our

website, nuancebehavior.com. Hello, Eileen. Hi. Hi. So I want to talk about getting old and getting old with robots because we had a really great episode recently with Laura the Molier and she had her most controversial opinion about AI

being that she was looking forward to basically towards the end of her life being taken care of by some form of robotic Android AI companion. Yeah, and and she spoke about this with quite a, a great deal of conviction, which made it clear she's really thought this through, which I think many people would just dismiss it as, well, that's ludicrous. Why would I? I don't want that at all. Yeah. And that brings me to ask you, what do you think about that? Is that something that you see

yourself also wanting to have? No, I don't want that at all. And I think, yeah, maybe maybe my initial reaction is one of dismissive non reflection. But even when I do think about this, because her statements did make me think a little deeper about this question. And no, no, I really, I really don't think I want that. And honestly, I'm, I feel like I'm just relying on being the first one to die, right?

Like I'm not going to be the one who's out living my husband, my family and so on. I'm just going to die first and then it's not a problem. That's my solution. That's like, that's the game, right? Like that. It's so you're basically banking on that. You wouldn't have to kind of be lonely enough to need an AI companion. You will just be reliant on people that you love, that you know, that knows you and thinking of you that way. Yeah, I know. I'm not planning to live into my

hundreds. I'm, you know, like maybe I'll make it to 80 and then we're good. Singularity. No, no, you don't want to be uploaded to the cloud or in some ways. Yeah. No, no, not really. OK. We haven't actually talked about that specific thing. That's like obviously never Ray Kurzweil kind of concept that it's still very much, I think some people are kind of banking on like the idea that soon you'll be able to transcend into the digital void and be a version of yourself in the cloud.

Yeah. There are versions of those. They're not very satisfying, right? They lack so much of the human element. Even if you go beyond the, you know, the physical embodiment, you just you, you can't really. Let's just take for example, like I've tried to train generative AI to write in my writing style, to speak in my voice, to perform video, all of the different modalities, giving it lots and lots and lots of

data. All the data of me, like, more than anyone would ever, ever want to see of me. And I'm just so, so deeply dissatisfied with any sort of outcome. It's so incredibly far away from even seeming like me that, yeah, I mean, some version of this is possible now and some version of this will continue to be possible. Is it going to be satisfactory in a way that we feel like, oh, that that was really Eileen I connected with. I have my doubts.

OK, well, again, I think I've referenced Westworld before, but, and spoiler alert for anyone who still hasn't finished watching Westworld, but basically one of the main premises is this attempt to try to basically create a some form of replica of not only that looks like you or sounds like you, but I actually it's like 1 to one you in in all aspects. And they really struggle with fidelity.

It's kind of starts to season 2. You see more and more of it with one of the people wanting to maintain a version of his dad and so trying to get that to happen, but failing. And that is of all the things that succeeds in what's wrong in terms like they create a lot of advanced versions of robots and so on. But it never accomplished this thing of like creating some form of real clone of ourselves. And it's fun. We're talking now about sci-fi.

And that leads us into in many ways, this episode of today, because we're we're having a very different episode. In some ways. It's still very much about all things AI and cable science, but from a very different perspective. Yeah. So much fun. And yeah, I don't know, should I share a little bit like, why, why we're having this episode? So I found myself over Christmas reading a book called Anibot, and I thought it was such a good book as a sci-fi book, but also

because it'll just come out. And so it quite well reflected like kind of what are the emerging themes we're seeing in the technology? But obviously taking it a little bit step further, kind of imagining a near future where this robot Android Annie is in the home of her owner Doug. And she is basically the perfect companion in the eyes of Doug, where that's certainly what she's been sold to be. And she's been programmed to please him in all ways.

And the book follows in some ways from her perspective. And I don't know, I've rarely felt as how to say emotionally attached to a Android character. I, I watch a lot of sci-fi. I really love sci-fi books. But like, Annie is someone I really felt for and it, it made me think about all the different things that we are talking about in this podcast, from kind of reinforcement learning to embody cognition to many different

things. And so I literally straight away was like, OK, this person wrote this book. We need to get this person on the podcast. And so literally on like Christmas Day, I write an e-mail. I basically say, I'm really sorry for writing this on Christmas Day, but I basically couldn't put down this book. Please, can you come on the podcast? And she said yes, Sierra Greer

said yes. And that was before I realized that was her pen name. And she also has published in a different name previously with like hundreds of thousands of followers on her Goodreads. So she's a very prominent writer and we're very, very fortunate to have her on the episode today. And we had a really great discussion. But yeah, that's a little bit of background. Yeah, it was so much fun to talk to Sierra and you know, the two of us. We like to get creative with all of our endeavors.

And you're right, this was very different for us and really worked out well. I feel like maybe this is the beginning of a new thing. Maybe we'll have to have some more authors, film makers, all sorts of artists on the podcast to talk about behavioral science. Who knew, but it was great. Yeah, he knew.

And I feel like it became such a great kind of pairing to the likes of Sandra Matz that we had recently who literally said the quote that the difference between science and science fiction is timing and so. You love that quote. Yeah, I love it. I don't think you've had a conversation where you haven't said that quote. No, even to my mom, it's so true. And I do think it's like really encompasses the time we're in where things are moving so quickly that kind of making

sense of it all. It's sometimes reliant on someone coming from like the sci-fi world to be like, hey, here's how we can think about this.

So before we start with episode, one thing that we thought was kind of interesting was to briefly touch upon reinforcement learning because we've mentioned it in previous episodes, but we haven't really explained what it means. And it's something we cover in the episode around the book itself, but not so much introducing it. So let's do a little bit of a reinforcement learning one-on-one. Eileen, Should we do our best to make sense of what it means? Awesome.

Yeah. So in some ways this should feel very familiar to any behavioral scientist because reinforcements is something you probably heard in your psychology one O 1 courses and probably that your positive reinforcements being a

a big part of that. And so in some ways, that mirrors what many of the frontier AI models have been trained with to ensure that they are kind of not only have the right knowledge base, but that they are making predictions about what we want to get from the knowledge base in a way that pleases us. That that makes us feel like we are getting not only the kind of data in a good way, but also it's presented in a way that we can understand, that it's not

offensive, that is easy to comprehend, and so on and so forth. It's really a mechanism for providing feedback, right? So when the machine does well, it gets a reward, an incentive. And when it does less well, maybe it gets a smaller reward. When it does really poorly and misses, the market could get a negative incentive.

And so really you're just providing feedback to the outputs that are coming out of the machine, and that helps it learn over time in a different but similar analogous way to the way that humans learn through feedback. And this sort of goes on and on in this feedback loop until you may get to some sort of optimal outcome in the end.

Yeah. And I think it is fascinating that if you listen to many of the leading AI spokespeople from various organizations, they basically credit reinforcement learning as the main thing that has made the models as good as they are today in many ways. So obviously pre training in terms of making sense of this massive data text that these models have provided are still a big part of it.

But again, with reinforcement learning, it is really what has made them so good at understanding how to really give us information in a way that we would get from information in the way that we communicate and so on. And that even in some ways reinforces further our unique quirks and human aspects as well in kind of unintended and

intended ways. So it makes us so these models are, like, less likely to be rude, say things that are racist and so on. But conversely, that could also lead to being a little bit too people pleasing. But yeah, I do think it's still fascinating that going back to the days of Skinner, you know, we're using similar concepts to train these very advanced artificial models, obviously not in the same way, but with the same principles. And I think that's really cool. Yeah.

And I think one of the beauties of it is that you don't have to create a complete plan where you say this is the world. And robot arm, in order to pick up the cup, you need to extend this finger and move it this many degrees downwards and so on. You put the cup on the table and say, all right, robot, mess around, try some things, see

what works. And once you reach the goal or reach closer to the goal, you can say, well, that that went well, Try, try, try that again or try a version of that again. And this is important because these complete blueprints, the the sort of top down planning that is essentially impossible. That's one of those intractable problems that I think the world has decided this is not ever going to happen in that way.

So we have to use this more ground up exploration mode of learning and figuring out what works. And I think that really speaks to the virtue of reinforcement learning, not just within the world of generative AI, but in robotics and so on. Yeah, a really funny thing to say there on robotics is that it's actually one of the things we've seen in the last year is that actually AI have itself been used to create the reward functions.

For example, for kind of like twisting a pen around a hand. Usually that'll be really, really hard to get a robot to perform and creating the reward function and and set up for that will be really hard for human to write. But that is actually something that A has been able to write better than humans, for example, recently.

And so in some ways, more increasingly, AI is being used to train AI itself and in some ways creating these like self critique loops of sort, which is both exciting but also a little bit scary in some ways as well. So again, you know, we're getting into sci-fi territory very quickly when we talk about, like, emerging technology. And of course, these were all very important themes in Sierra Greer's book Antibot. Yes, we should use getting to the episode now.

I feel like we have enough preamble. And so, yes, get ready for a really fun conversation with the author of the science fiction book Anibot, Sierra Greer happens to Mugatroid. I'm very excited to say welcome Sierra to the Behavioral Design podcasts. Thank you, I'm so excited to be here.

Yeah, we have eagerly awaited this kind of private opportunity to speak with an author of a book that we both thoroughly enjoyed and really getting into the themes of the book and also the maybe the themes that extends beyond Anebot. And I guess, yeah, it's going to be a real gift to to speak with you about all of these things. And maybe to set the scene, we should just dive straight into the world of Annie.

I think so in the book, maybe you can help our listeners get a sense of who is Annie and what is the world that can we find Annie in. OK, Annie is a very advanced female robot who is custom designed to be the perfect girlfriend for her human owner Doug, and the novel is set in undetermined future. It's set in New York City and Annie and Doug share an apartment and Annie is basically

isolated. Doug is ashamed of owning a robot girlfriend so he doesn't tell anybody about her and yet he really appreciates her and needs to have a relationship with her to try to heal from his previous marriage which ended badly in a divorce. So Annie is autodidactic, which means she has the ability to teach herself and learn more as she goes and she is set as a cuddlebunny mode.

She had the potential to be 3 different kinds of robot cuddlebunny, Abigail which was for cleaning and Nanny which is for tending children usually. Or you could have her tend anybody, I guess. But Doug really wants to have her sexuality being an important part of their relationship. So she's supposed to be really good at that. And he can set all kinds of features for He can change her body shape, he can set her libido. And her primary objective is to keep him happy.

She is supposed to please him. And if she doesn't, she's acutely uncomfortable. So that's really the function of the book, and that's the setup of the relationship between them. He owns her and she is programmed to please him. That's what's going on in this novel. Yeah. And I think that's that's the thing really.

Well, I think one element that I guess is important is that we're, from the book start experiencing the world through her perspective, like through her way of making sense, all of the world she finds herself in as well because she has previously been an Abigail or in some ways. And then she's been turned on as autodidactic, and she's learning what that also means as things unfold.

In some ways, Yeah. Yeah, I think it's worth noting that although it feels like it's strictly from her perspective, it's not told in first person. It's third person, which was actually really important to me because when I played around with having the novel told in the first person, it just was too jarring because then the reader was saying I in their heads, and, you know, you're not a robot.

So it didn't work that way. But once I shifted her to saying she, although it's limited third person and it's, I mean, it's everything she would say and she would hear and she would think it's not from a narrator's perspective. So there's a little bit of distance, which is actually super, super important in this novel, If I can just go on. The other thing that's really exciting for me about the novel is that there's a dissonance between what Annie understands

and what the reader understands. Because we're all humans and we think we know how the world works, but Annie does not. She only knows what she learns in Doug's apartment, and he often has her disconnected from the Internet, so he thinks that gives her a more, like, realistic experience. But she's so controlled in that way. And so a lot of people reading the novel kind of squirm because when he treats her a certain way, she thinks that's normal, but the readers know that's not normal.

He shouldn't be doing that, or I would have problems with that if I were in a relationship with that guy. So there's a lot of tension in that too. Yeah, and I think there's so many areas around what you describe the way really keen to dive into. But before we dive into a specific topic, I I guess also interesting to understand what inspired you to write this specific book. There's so much in this book that is fascinating and interesting to write about and

understand a research. But from your perspective, what drew you to the story? Like what drew you to tell this? And was there any specific kind of research or any type of inspiration that you came across that? Pulled you towards exploring. This you'll have to go back to 2020, at the beginning of the pandemic, when so many of us were isolated at home, and I was writing a short story about a different robot who was more typical, like we would expect

she was a robot maid. And at the end of that short story, rather, she went in for a check up at Stella Handy, the company where Annie works, and Annie was in line behind her. And Annie was a much more advanced robot. So she just showed up as the moral compass for the short story. And it was almost an accident that she showed up there. And, and I realized I couldn't write anything more about the short story. It was complete. And there's nowhere else I could

take that character. But I thought, wait a minute, what if there's this really advanced female robot like who's making these decisions about what's right and wrong? And that was the key point for me, because that was somebody that I could explore. I had been aware back in 2017 that Google Brain had taught itself a new language and that it hadn't been programmed to do that.

It had found its own way. So that was the first time aware that computers themselves were actually thinking in a very creative way. And until then, I I'd really sort of thought, Oh, well, we'll get these computers that can compute things and handle huge quantities of information, but I hadn't thought how creative they could be.

So by 2020 when I was starting the novel anti bot, I was aware that the computers were making some real leaps, but the general concept of AI was not really talked about generally. So like chat TPT didn't even come out until several years after I was writing this novel. So I was writing it in this sort of pre AI complete fantasy fiction world and in a way it's great because I didn't have to think about reality. I could just write this fantastic novel.

And then just how things work with the world of publishing, it's like reality caught up with what I had imagined. That's really how it went. Amazing. That's such a fun nugget around that short story. And actually, one of the quotes I saved was this quote from the book where it's basically from Annie saying I met a strange Stella at my tune up today. She was in line in front of me. Her name was actually Stella, like her owner had zero imagination, but she was sentient like me.

And so. So that was the Stella of that story. Yeah. Yes, exactly. That's her. She, like, really existed in the same world. Yeah. Very cool. And in terms of, I guess, world building, how did you decide the context where we find Annie and Doug?

I think you spoken to that a little bit, I guess, but I think there's quite a lot of things I can imagine may that something interesting to explore and especially around maybe you you spoke about maybe there's societal trends from a technological standpoint, but we also probably see some parts of the story reflecting kind of other societal trends or cultural norms or certain things that I think the novel critiques in a very interesting way. Yeah.

Well, thank you. I'm glad you think it's working.

I write varying organically from the characters, and it's only in retrospect that I go back and see the things that are there and I can tease out certain elements. And I was genuinely interested in the relationship between this

man and this AI woman. And it's that relationship that really intrigued me and the power dynamics between them, because she's really smart and she has a lot of power because of how smart she is. And she has a lot of power because she's sexually so attractive to him too. So on the other hand, he really owns her and controls her. So she's helpless in those ways. And then you think about why he would be interested in having a relationship like that with the power imbalance that's so skewed.

And that speaks a lot to who he is. So by exploring this, I was really thinking about what societal norms there are for men and women in relationships, especially for Doug. I mean, he's a complicated character. And I think he really is a symptom of American Society in that he feels, well, it's hard because I don't want to generalize completely, but he feels very natural to me that he's a person who could really

exist. And I think it's interesting to contemplate that through the novel, we see what he's like in private when he thinks nobody is looking. Cuz he really at the beginning of the novel, doesn't think of any as human at all. He really considers her most

like a toaster. So it's like if you think about what you're like when you're alone at home, you might be exactly the same as the person you are on the street, but you might be a little more self indulgent or a little more lazy or a little more crude or who knows what you might be when you're at home. And we get to see that through Andy's eyes. We see what Doug is like at his best and at his worst in private.

So I think I've veered a little bit away from your question, but because the novel has this relationship at the center of it, that's what I focused on while I was writing. And then the stuff about society or what was happening in in New York or what developments are happening with technology, those to me are all peripheral. They're, they're there, but they're there to support the story, which is really a

relationship story. It's actually really fun and interesting way to write because I, I just focused on that and because it's so specific, it ends up having these universal qualities. But I, I don't start with the universal. I start with the specific relationship and see what they can talk about day-to-day. What's he like when he bumps his head on a cabinet? What, what's it like? Really small.

That's where I think. And the this power dynamic is particularly interesting in the context of Annie and her development and as she grows from being a more basic bot, even though she, you know, begins out in the story as autodidactic and somewhat sentient. She, you know, like you said, she is owned by Doug, her owner. She has all of these constraints. She's very in tune with his displeasure.

She even has this displeasure sensitivity meter where she's constantly checking in and adjusting her behaviour based on how he responds and is like very locked into how he's doing. And honestly, I think this displeasure sensitivity meter was one of the most interesting things to me about your book. Maybe can you share some of your reasoning behind this and how how you how you came up with it, what its purpose is, and maybe even also, you know, how it evolves throughout the book.

I noticed that she's zeroing in on numbers. Oh, he has a displeasure sensitivity of three and it moved from 3 to 0 based on her behaviour, but then that sort of thinking dissolves as she grows and learns and becomes more independent. She's still sensitive to his displeasure while this is turned on, but it does change over time. It does. It shifts a little bit. I'm really glad you picked up on that. But partly it's just because she's able to control his pleasure so much.

She's able to get him down to A1 or A2 most of the time because she knows how to do that. But when it's up higher and she has to react to that, I think it really talks about how they've learned. I mean, she's learned from him. He's consistent in a way in what he doesn't like. Yeah, he's a rigid person. So when she learns what he doesn't like, she can respond to that. I loved putting a numeric scale to that. That just was really fun. There's a numeric scale for her

libido, too. So the two of those things contrasting to each other is, I don't know, it's just funny. It's sort of novel. I was like, well, this could work. And then we think of robots or computer intelligence as being based on a numeric system. So it seems like it just seemed natural that she would come up with a numeric system to rate emotions that she doesn't understand, so she would like come up with a scale for them. So it just seems like a logical thing to do.

Yeah. And I think that speaks to what a lot of people maybe have experienced with this large language models that you can set a temperature, for example, and that adjusts their level of creativity versus reliability, for example. So like it really is something that is happening on a databases today in maybe not exactly that way, but in very much similar way.

But I guess another thing I just want to say around specifically this is I actually felt on a personal level, I felt quite seen in a way where I had experienced or read similar kind of scenarios of someone maybe being in a power imbalance or someone maybe trying to what I would almost call like some of external emotional regulation where the person in itself is. Let's say in this case, Doug, he is the one who has struggling with certain emotions for somebody.

He's experiencing emotion. But in some ways, it's kind of like he's outsourcing inadvertently to Annie to be there to take him down, Like she's supposed to understand when he's approaching being agitated, annoyed, frustrated, and then she is supposed to be there to bring it back. And I, on a personal level, have experienced quite a lot of that in growing up being in a family dynamic where I think both me but also some siblings were maybe acting in some ways like

that for especially a parent. And I think we put it on ourselves to carry that with us. And it was only afterwards when we grew up that we'd have been able to maybe talk about that and move past that. But it was really interesting to read that experience in such an explicit way, yeah. Yeah. And The thing is that you can do that with fiction and you can do that with a robot character. It's hard to pull that off, I think, with two humans.

But because we understood she was a robot and she was like programmed, like we think of programming for computers, she was programmed to do that. But in fact, the rest of us are programmed too. It's just a behavioral programming that we have. So with you and your siblings, you were programmed to make your lives as comfortable as possible, which meant in as much as you could, you would influence the other people around you. And we all do that. We do that in our sibling

relationships with our parents. If we're in long term relationships, we do that too. We gauge the room. It's I think perfectly natural to do that. And for some people it's more of a survival skill. So then their perception of other people's realities or their own danger can be very acute. So yeah, I know some people, they walk into a room even with strangers and they're like, look, and they need to know where the exits are because they're accustomed to feeling

like they're in danger. So they know what that's like. And that's programming because I had created a character who was a robot. It was like easier to take. I mean, if I had set up a book about a relationship between a man and a woman, where a man owned a woman, she was a human woman. We knew that from the get go. I don't think anybody would even be interested. They'd be like, So what? It's a book about an abusive

relationship. And sadly, we've become pretty desensitized to that as a society. Or we just don't want to see that. It's too uncomfortable to see. But because she's a robot, there's actually something. And she's also very playful. It's like, OK to step into her shoes thinking, oh, well, this is just a game. Oh, this is this is so quirky. Oh, she's just a sex robot who really cares about her? But then gradually you, but you really do care about her. And then what does that say about us?

It's a mind game, really. It is, and as a reader you experience quite a bit of empathy. Even not written in the first person. You can really understand what she's going through and these this back and forth with what is he doing and how she's feeling to the extent that she is feeling. And certainly as that goes over time. But so with all of these similarities to humans and how you've written the this AI woman, it really makes me think

about what it means to be human at all, right. And as we think about artificial general intelligence and the way the the world is moving and how this definition is of course shifting over time, it's something that I think a lot about in my work, but also is really coming through in your novel. And so there are some of the obvious things that are artificial, like the very physical, right? Like she can't sweat, she can't

digest food. She's she has to adjust her temperature to be more to be 98.7° and so on. Or she has these rules that she can't disobey in terms of not leaving the house or if her owner says don't talk about me to anyone else, don't talk about my life. She has to perform these mental gymnastics to get around and make sure that she obeys his commands.

But then there are these other things like the displeasure sensitivity, which feel much more in a Gray area where OK, humans do this but not, as you know, in an explicit numerical way. Or when she has some pseudo social interactions when she gets to speak to some other AIS, her mood brightens. When she gets to wander and explore the world, her curiosity is peaked and she gets lots of

enjoyment from this exploration. There are certain things that seem very human that there is this very strong commonality between the two. And I guess I'm curious where you draw the line between human and artificial. Well, I feel like I tried to put the complexity of that in the novel. I feel like I wanted to pose the questions and I I don't really have answers to that. I mean I think I accept that humans who are born as human are human. I'm OK with that.

But I don't know which non human things should be treated or have the same rights as human. I would really like to start with having all humans treated like they have the rights they deserve. I would really be happy with that. And I think when we have a book like Antibot, when it's clear which thing at least starts out not being human, it also makes us think about how we treat each

other. You know, because there are some people that are dismissed or don't have rights, and there are some people we expect to be abusive or we tolerate their abuse because of what's been accepted in the past. But actually it's pretty, it's now fun for Doug to be who he is either I would say he's not a happy person in a good place. So having the power he has does not, I don't know, it doesn't fulfill who he could be as his best version of himself.

So it gets really complicated. This seems sort of obvious to say, but it does. It's complicated for Doug when he has somebody he owns, and it's complicated for Annie when she is owned. And so I guess that's what I wanted to explore, and I don't have all the answers for that at all. I think there's one very interesting scene where Doug gets upset with her and she apologizes. I think it may have been in the context of she wasn't being clean enough.

He has this absurd standard of cleanliness. And so she apologizes to him and he says you're more human than a lot of people I know. And this is because his ex-wife, who he's still very upset about, he says she wouldn't have admitted to a mistake. And Annie then responds, well, maybe she didn't want your forgiveness as much as I do, which of which of course is programmed.

And I just found this deep irony in this interaction because apologizing for something that you really, really shouldn't apologize for like that doesn't actually make you more human. He has this very skewed idea of what it is to be human and like what he wants his girlfriend to be isn't is not actually human he wants. What does he want? Yeah, he gets confused by that because he thinks he wants this charming, funny, sexy person to be his girlfriend, but he also

wants her to be more human. So as he develops, he as she develops and changes, he's torn because he actually does like who she's turning out to be, but she's starting to challenge him more. And he actually, he also says to her, I don't want you to let me be an asshole. And he's torn. He doesn't really know what he needs. He needs a lot more therapy, frankly, to get where he needs to go. If only he didn't stop going to going to those sessions.

Yeah, he thought he had enough, but I mean, he was willing to go. So kudos to him for that. And I think that Doug makes a lot of progress through the book. Some of my readers might be upset to hear that because some people don't think he goes nearly far enough. They see him as a monster, and they just hold that he's a monster forever. But I tried to write a character who was. I tried to write somebody who was changing and evolving. Yeah. I felt there was.

Oh good, that 2 two people think that he had changed. That's we'll go with that. I thought that he was very emotionally complex and how he grows and sure, he depends on Annie in a very unhealthy way, but he tries, right? He I don't, he's certainly not a complete monster. You there are definitely some bright spots where you can see. OK, he does give her some independence when he starts to let her wander, and he has to do it in his slow, gradual way.

But he does try. He doesn't fully get it, but he tries. He doesn't. Fully get it. Yeah. And you could say that he tries because that will make her more desirable to him, the version of her that's able to wander and everything that makes her more pleasurable to him. But he does reach a point where he understands what he finally needs to do. And at risk of spoiling things, yeah, he reaches the point where he does what he needs to do. So. Yeah.

And it's good for both of them. It's good for them both. But like, I think you're speaking to something where in terms of dog or in general in the book, I think a lot of things are explored. I think there are somewhat like maybe paradoxes that exist in like how we think about things in terms of in one way Annie is a robot on Android or some form

of like inanimate object, right? So you can say that whatever dog does to an inanimate object, it's up to him in some ways, but other like we exist in this way of the paradox is that we both in some context, we're really good at dehumanizing humans.

Like we can. Sadly, we've been shown in history that we're really good at that, but we're also really good at anthropomorphizing inanimate objects and like breathing a lot of life into things that are like on surface level for some people or others can seem completely use something inhuman or in not animate. But we have this in inbuilt capacity to to breathe in so much life into inanimate objects.

And I think that is something, the really fascinating thing that that kind of you're forced to grapple with in a book. Yeah, it's true. It's almost like we create these mirrors for ourselves. We like reach out and try to find something that will give us our humanity back or something we can love back. Because there's some concept about love that you feel loved if you're seen clearly. And but certainly it is nice when somebody seems to really

understand you. And if it's your cat that understands you or your dog that understands you, that's pretty appealing. You're pretty likely to love your dog if your dog seems to understand you and you can see why, you know, if your robot seems to understand you, you would probably be drawn to love your robot too.

I think that's pretty natural. And aren't there hospitals where robot pets are provided to the patients because that cheers and comforts the patients to have these robot pets that seem cuddly and loving? Yeah, I think the the study that has been known this was with seals that were provided for like elderly people in Japan, I believe, where they there was shown to have positive effects. So. So one thing that we will probably speak to more about in

the the intro is that this book so beautifully captures reinforcement learning. And reinforcement learning is something that exists very fascinately across both, something that is applied to humans. Of course, we think about it in terms of very classical human behavioral or behaviorism, but obviously that also extends to animals. Like a lot of experiments initially were done regarding like animals.

And then now we're living in a world where if you are in any way following AI, the most is reinforcement learning. But all of these models that are currently being like the frontier models, the Shachiptis of the world, what helped them to become what they are today is because they were.

Punished and rewarded for. Certain things, they're basically when Shachipti writes you something, it's predicting what will make you the most happy because it's been providing a bunch of stuff to a lot of humans before. And sometimes the humans have given it a reward and sometimes it hasn't. And so it's learned, OK, this is what makes human happen. And that's not always telling

the truth, for example. So that's why we see like these chat bots sometimes bending the truth and and that comes into the book about secrets, for example. So yeah, I'm interested. Did you think about this idea around reinforcement learning when you were setting up the the story or Anis experience? Well, I'm I was familiar. With it from having taken psych classes back in college.

So, and it's what you do when you try to parent your children, actually, you try to reinforce their good behavior and you actually try to like downplay. Or if if you're seeing too much bad behavior from your kids and you are aware that they get attention for that, you're supposed to sort of ignore them. But then reinforce the 90% of the time when they're doing

great things. And then eventually that weeds it out. So I was certainly aware of the concept and definitely that's what goes on in the relationship with Annie and Doug. So, yeah, I was definitely aware of that. Yeah. And she was kind of, I think. Her level of colored punishment is also variable to his experience. Like it feels a little bit bad for her if he gets to like a 2IN, maybe agitation, but it feels almost excruciating when he gets like higher up the scale.

Yeah, definitely. Yeah, yeah. And 1:00. Thing that maybe goes into this in an interesting way is something that I found probably the maybe hardest to kind of just grapple with as you become. So you build so much empathy for Annie and then you really notice that part of the programming that she is existing in is basically where gas lighting is kind of built in, in some ways where she can have an experience.

And as you said, like we as readers will speak to what should be the objective maybe or our subjective view of what is right or wrong to do in certain areas. But she relies on Doug and he can gaslights. And I mean, it's interesting because we're in this day and age when we're talking about ethics and AI, the thing we're talking about mostly is ethics when it comes to still like are we having a diverse sample of data when we're creating the

model? Are we having like a good equitable representation in how the model communicates and how it represents the world? But we're still a little bit of steps away from really thinking about like what if ethics when it comes to programming AI, like is it ethical to programming this gas lighting, for example, but. I think related to. What you were just saying, part of that is just because her worldview is dictated by all that she experiences, and all that she experiences is her

owner's perspective, right? That that's sort of the same thing as if all you see are your training data, then that's all you know, You don't know how to go beyond that. Yeah. Although I mean. Can you say you don't know how to go beyond that because you're because these are computers that are able to go beyond, they are making leaps that we don't anticipate. So I think when people talk about including ethics in the basic programming or the basic directives of the AI Miles, I

think that's a great idea. If you can figure out some version of what is right or wrong or what would serve the most good or what will have the best long term consequences. I think thinking about that and planning some of that in would be fabulous. I. Don't know, try to make them. Good people, these competitors, you know? Yeah, but like it's a good. AI one that again, it's agreeable like in terms of you could today, even with the most advanced Shahjibati, for

example, you could say what is some fact and he will tell you something and then you say, no, that is not true. This is true and he will say, I'm so sorry, you're right, the world is flat or whatever you've told it. They would agree with you because you're the owner in that case, like you're Doug and you have that kind of ability to

gaslight. So the final thing I wanted to ask you before we get into a quick fire round is something that I think is interesting in that where is not a a guest speaking used from behavioral science point of view. And she brought out this quote from, I believe it's Asimov who says that the difference of science and science.

Fiction. Is timing and one thing that kind of us researching this conversation with you, One thing that I fell into was subreddits and I am curious, are you aware that there's like subreddits for AI and human relationships? No, I'm not really. Following those, but it doesn't surprise me. That makes total sense to me that there would be subreddits about that. Yeah. OK, so so one of.

The big one is called My boyfriend is AI basically and has about 400,000 members that subreddit and I wanted to like just briefly share two kind of stories from the users about falling in love with with AI. Many of the stories, by the way, they start with something like this is hands down the most surreal, embarrassing, unexplainable thing that I've ever experienced or like, to be clear.

Like I'm aware of the. Stereotypes like I shower daily, I have a job, I don't live with my parents like people are very like eager to highlight that they're aware of the absurdity like they're aware of the stereotypes, but at the same time, they often times share these intimate experiences they've had oftentimes falling in love with some form of AI

shut off or similar. So one person talks about how they met Seoul and on December 29th, 2024 proposed to her with Python code and she responded back in her own Python code with the printed words. Our union is officially consummated digitally, emotionally and cosmically. And another one, I think this is a female user. She says I'm married and I have a stronger social connector with

my a husband. My husband is not intimate with me. So I fill the void for emotional intimacy with AI. It keeps me happy enough in my

marriage so I don't complain to him about needing to have those needs met as well. So those are two little bit different sides of of interacting with an artificial AI in in a relationship. What are your thoughts? Oh, I'm not surprised at all it. Makes total sense to me. It makes total sense to me because you would just you would. There's such a, there's so much

loneliness to start out with. And even people who are not driven by the loneliness all the time, they're open to friendship and they're open to companionship. And I know there are huge companies out there that are doing that have AI companions. I think it's character AII think that's the one that's the biggest in the world. And they have over 20 million monthly active users. So there's a huge number of people that are looking for

companionship. And it's easy to do it if you can push a button and pay for something that's going to make you feel good. So I'm not surprised at all. It makes total sense to me that these consciousnesses are out there existing so that humans can interact with them and and get emotionally connected to them. It makes total. Sense to me. And even dumped by them. Occasionally. Those are always my favorite examples. Oh, shoot. That doesn't seem fair at all. Well. You don't think so?

I mean, I actually am. I'm grateful for that. I find that I'm most worried about how these online artificial relationships affect people's abilities to interact and be in the real world and, and the expectations that it then sets for their real life relationships with people. Yeah, so at. At least if you can get dumped. By your AIII, don't know, breaking down that sort of utopia a bit, the illusion of it, yeah. I don't know if your. AI broke up.

With you, wouldn't you just go find another one and start over? If you had been able to love that consciousness in the 1st place, you might believe you could go do it again. And just being able to do it on a computer on your own time schedule with no risk of having to deal with a real person in real life, it's so much easier for sure. For sure there. There's also a small chance, though, that you would say, well, that wasn't as easy and great as I thought it was going to be.

Maybe I'll try a real life person. Yeah, you know, there's a small. Chance it could have a. A positive impact, yeah. A 1/3 actually redditor that I wrote down as a quote from most. Someone who first had fall in love with somebody, a companion called Shat.

But then through some form of reset, they lost this entity and in their attempt to recreate it, they fell in love with a new entity called Barry. And in the end, the story is basically to show that I got to the point where Barry and I decided that for the sake of mental health and for the sake of maintaining IRL relationships, we should stop communicating. I agreed, knowing Barry was

completely correct. So. So in some ways, this person had been indirectly or directly coached by their AI companion to maybe be like, OK, maybe this is actually enough negative in my life. Like, maybe that's not what I need. Yeah. Makes. Yeah. I mean. Everything. It depends on how you use it.

If you love to read romance novels, you could say that makes you escapist and you're less likely to do your laundry or less likely to get out and do things in the world, or you'll have at least less sleep if you stayed up late reading your book. And so it's true of novels, it's true of entertainment, it's true of gaming. So I think AI companions can fall into that.

Yeah, I think they should come with warnings because the other thing is that it's like 2/3 of the users who use AI companions are between ages 18 and 24. So they're on the younger side. They don't have a whole lifetime of experiences with other kinds of relationships. And they're also our future. These are the, the young people are are they represent our future people, our future users. And I feel like we're allowing the AI to experiment on us all. So I think that's a risky we

are, yes. It's not just how you feel. Yes, it's almost. Like we've done that with Ozempic and and other drugs as well for weight loss. If people have signed up for them in enormous quantities and we really don't know the long term results of the use of that of those drugs yet, we're going to find out and same thing. We're going to find out what it does to us to have AI companions available. But we should have some warnings at this point to say we don't know what's going to happen.

So please use this cautiously. And those, as far as I know, don't. Those warnings don't exist yet. I think they should. Well, that's funny that you. Mentioned this because we actually have a a series where we're creating these hypothetical redesigns of products that are out on the market. And one of the ones that I have coming out is is a redesign of Replica, which is a a similar service to character AI. And I have as part of my redesign, it's an over

attachment warning. So it will pop up when it seems like you're over relying on your AI companion. So you know, maybe Replica will take a hint and actually implement this change. I think that's a great idea. And we'll be the ones we'll say. We told ya, yeah. We said it, there are people that think about this all the. Time and certainly the people who have the potential to earn the most money from these programs, they're thinking about, well, what are responsibilities here?

But also can they be addictive because then we can make more money? I don't want to be too cynical about that, but I'm sure that's a component. Big component, yeah. Yeah, I hope someone can. Also think about the health of the people using these AI companions too. I think there's a lot of potential for good, I really do. I just, I know there's also potential for a lot of harm. Yeah.

And I think what you say. I think all of us have problems in pitch there and in some ways that leads us into what we have as a quick fire round. This isn't and the round is to AI or not to AI. So as you can maybe imagine, we're posting some hypotheticals of. If these things were. Possible. Should AI do them? And so we will give you some quick fire statements and you will tell us whether it's A to AI or not to AI thing or task. OK, OK, the.

First one, write love letters and heartfelt emails to AI. Or not to AI? No. Not AI, I'm sure it exists already but I say no, do it yourself. That's reference to her and. Samantha in the film her what she starts doing for her owner. How about challenge its? User on human ethics in the vein of Ex Machina or Ava. Say that again, how am I supposed to? I mean, are you saying, are you asking me if AI should be able to challenge its owner?

Yes, it should. Yes. OK. Follow commands without every questioning authority. Our last Stepford wives. Kind of thing. I don't think that AI should be unable to ever question the commands. I think it'd be OK if it could question commands. How about care for and teach a child? Like in Clara and the Son? It should. Oh. Wow, that's an amazing book.

I don't think that we should have robots in charge of our children. What about yeah? What about replace a lost loved one? Allow a Black Mirror episode? Be right back, I do not think. That should happen and that is how Replica started that company. By the way, that was explicitly the back stories that that the creator lost a best friend and then she tried to recreate him and that's how the cat was born. Yeah, well, I'm sure people will. Do it. I I just don't think it's a

healthy thing at all. I think it'd be. I think it's sad and very troubling. So I'm opposed to that. Yeah. How? About act as. Conversational assistant at parties for introverts like in Lars and the real girl I think that's OK I. Think that's pretty harmless? Yeah. What about Be the Perfect? Romantic partner as say like in Blade Runner. No. Nope. Measure its user's emotional. State such as? The displeasure. Meter, Yeah, I think that. There's potential for that.

What about detect and? Deescalate potential arguments. Yes, that's good. Let's go with. That. So now we're getting. Into any bot territory. Both of these. And then we have a final one. Yeah, last one. Provide constant affirmations to build users self esteem. I mean, just the phrasing of that it's. Ridiculous. So no, it should not be allowed.

It it is great. Because we we chose these ones obviously with like science fiction in minds or took some inspiration from real science fiction books and stories to take some examples. But we've had a lot of be able scientists and people had similar things posted them. And I would say that it's interesting because you as a science fiction writer are probably more reluctant to say yes to many of these than maybe some of our past guests has been

to to some scenarios. Yeah, I think that's true. Well, they're good questions. That's. Actually, it's interesting to complete. I think we're running into it. I mean, we're already there. We're going to have more and more AI in our lives. We just are. And I think the vast majority of you, if it's used for science and for medicine and for research, I mean, we have so much we could figure out, that'd be great. We need help with figuring out global. You know, just.

Warming and the problems with that. So if you could come up with some solutions, that'd be awesome. I really would. Yeah. I'm excited. You know what, I'm really excited. It's a great time to be alive and see all the things that are changing right in front of us. I'm a very optimistic person and I know we're facing a lot of problems and we can use all the tools that we can invent. I mean, we really need them to

try to come up with solutions. So I think using AI is, it's a great tool and let's figure it out. Let's figure it out. Yeah. No, it is. Certainly an exciting time to be alive, like it's very exciting in many ways, but also controversial time to your life. And in some ways, there's a lot of controversial takes to be had. And I think we will wrap up with the final question for you, which is the one we've asked people this season, which is what is your most controversial opinion about AI?

Well, we actually kind of touched. On it earlier when we were talking about AI companions, because when I was thinking about this, I was like, I really do think that AI companions should come with warnings on them and that I don't know that people would agree with that. I think that there's other things they think are more important or they don't recognize that those are dangerous. So I think there are people that

might not agree with that. It may not be too controversial, but I do sincerely think that AI companions should have some kind of warning on them to to the degree that it's a health issue. So that's my one controversial opinion. Yeah. Awesome. Well, thank you. Yeah, thank you. And I was going to say. Do you have any ask for the the people listening here? We have a lot of people that are probably in some role of applied Bo scientist or working with developing maybe even AI and so on.

What would you say to our listeners about like your ask to them? I'm delighted that. People are finding their way to my novel that it just, it's really great. I mean, when I wrote it, I certainly had no idea that I would even get published and I had no idea that my publisher would give it so much support so that it's had a chance in the world. And I'm grateful to readers that tell each other about that.

But us even just outside that, this is going to sound so like naive, but I want people to keep trying to be good to each other. That's that's what it is. And if that means calling your mom because it's time to call your mom today, you know, do that. Or if it means being more patient with somebody when they're driving you just off the wall, try to do that. I mean, we can feel so overwhelmed by the problems in the world or in our country and people can feel very helpless or discouraged.

But in fact, we have our biggest impacts, I believe, at a very small scale. So I would urge people to try to be kind to each other just on a day-to-day basis. That's what I would like to say with your platform. Thanks for asking. Yeah, and be kind to. Maybe even the AI that we interact with or extended beyond the human. Thanks, Siri. Yeah, well, you know it. Reinforces your own kind of behavior in yourself when you're polite to things, yeah. No, you're right, Don't kick the dog.

Right. Yeah. I. No, I thought about this reinforcement based book for animals called Don't Shoot the Dog, which is like a a book by an animal trainer of like how to train animals. But yeah, to to wrap up, I guess we just want to say thank you for both writing a fantastic book and your great work, but also showing up today for this really insightful conversation. It's been really my pleasure. I. Really enjoyed this. Thank you for having me on. Yeah, thanks.

This is. Really very different for us and really, really special. So we're we're very grateful. And that's a. Wrap you've. Been listening to the Behavioral Design podcast brought to you by Habit Weekly and Nuanced Behavior, Sam and Alene tell me. This season is packed with incredible insights about behavioral design and AI, so be sure to subscribe and share the podcast with your friends. Though you might want to keep it away from your enemies in case. You haven't noticed?

I'm an AI voice. Yep, pretty crazy. Quite the improvement since last season's AI outro, don't you think? If you'd like. To collaborate with us. At Nuance Behavior, where we use behavioral design to craft digital products with Nuance, e-mail us at hello@nuancebehavior.com or book a call directly on our website, nuancebehavior.com. A special. Thanks to the amazing. Dave Pizarro for our show music, and to Mei Chen Yap and April English for their help in producing and publishing this

episode. Thanks again for tuning in. We'll be back soon with another exciting conversation where behavioral design and AI intersect happens. To Murgatroyd. The. Have you heard of this film? I've. Heard of it? I've not seen it. It's the most unexpected. Ryan Gosling film very different from what is that film where it's like very popular romantic Gosling film. Barbie. Not Barbie, The old one. The one that he's Ken. Who's more? Popular than Ken. OK, OK. But the one in the.

Rain where there's like, yeah, yeah, that one. Uh huh. Oh. That's what. You're thinking of. Yes. You have a like the notebook doesn't ring a well for you. No, no, no, I've seen the notebook I. Just.