Hey, Sam. Hey, Lane. So I've been thinking about something and I want to get your take on it. And I think we have an interesting situation that's emerging in this landscape of AI and mental health, which of course, is one of the things that we think about a lot. On the one hand, we have therapy and companionship, like really rising to the top as the very
popular use cases. These are common use cases for people using generative AI. There's evidence that this can sometimes be helpful, especially in cases where that chatbot is clinically validated, say like this Thera bot from Dartmouth that we'll talk about in this episode. But on the other hand, we see from research like this study that pretty recently came out from Open AI and MIT that spending more time with LLMS actually has a detrimental
effect on well-being. So this study showed that people who spent more time really like heavy users of Chad GBT had an increase in loneliness and sort of substituted for these real world interactions that of course are very important for, you know, being alive and feeling like a human and so on. So my question for you is, how do we reconcile these findings that seem to be at odds with one another?
I think it's a very interesting state of things where there's clear evidence, again, that using AI can be extremely useful for many things, but again, that it can also, through overuse or through extreme use, have some form of black fire effects. And as a Swedish person, I can't help but think about a very historically significant thing that happened in Sweden. Can I give you a quick history lesson?
Yes, please do. In the mid 20th century, an idea was introduced in Sweden. Basically the idea was that how can you truly love someone let's say a partner if you're depending on them for their income. So if you're a woman in in 1950s and most women at the time were depending on their counterparts salary to survive, like is that love? Is that true love? It's really just utilitarian. It's one way love.
Exactly. Well, the Swedish theory of love was that basically like what the government should be, It should be what's called folk Hemet, basically the home of the people that you should only depend on the government, so you don't have to depend on a partner. Even as a child, you shouldn't even have to depend on your parents, because if they're bad parents, you should be rescued from them. So that's the idea. The government should serve as
this home for the people. The kind of backfire effects of this is that, of course, Sweden has had a really good Social Security net and like provide a lot of support for people to live relatively good lives if compared to most countries. But the flip side of this is that, well, if you don't need to depend on any one of them, you might find yourself lonely in your apartment because you don't need to depend on your
neighbors. You don't need to depend on your partner, you don't need to depend on anyone. And there's been a lot of this case in Sweden, kind of always character of Swedish society where, you know, people are very cold towards each other. They kind of like isolated and there's a lot of loneliness and a lot of people that are basically in the worst of cases dying alone and no one noticing for months because they don't have to depend on anyone.
This one I'm trying to take back to with AI is that like AI is amazing and so incredible in what is able to provide, but it's also risking to some very similar things like that you could become so easily quite dependent on AI for companionship, for productivity purposes, for all these things. And you're less incentivized to do things that are harder, like getting out of your comfort zone or going out of your apartment to hang out with a friend. Like all of these things take
more effort. I think it's a really interesting, I did not think of it that way, but certainly you can have too much of a good thing, right? The risk of taking that argument too far perhaps is you get into the American conservative talking points of like, we shouldn't have a social safety net at all because people will take advantage of it and so on. And like, that's not the answer
either. And then if you take that, if you sort of, you know, copy and paste that argument to AI, we're not saying that we shouldn't have AI because there are some potential detrimental consequences of overuse. Maybe we just need to design systems to prevent people from overusing it. And I think one thing that I've been thinking about so much recently is this one design feature of ChatGPT where every single time you get a response, instead of just giving you the response.
And, you know, if you have another question, you can ask another question. And instead of just leaving it at that, every single time ChatGPT offers to do two or three more things for you. Would you like me to create a visual for you? Would you like me to, you know, turn this into a table? And it's just so irresistible to continue this chain of work forever because, you know, sometimes I'm like, yeah, sure, make me a table.
Like, that sounds fun. But then you find yourself just spending so, so much time going down this trail of things when you only wanted to do, you know, one thing. And you end up just lured into this constant cycle of creation where ChatGPT is just making more and more work for itself. But we don't have to design systems in that way. Yeah, No, exactly. Yeah. I think it's not only in terms of work, but also again, we'll
come back to a companionship. It's the same way there where it's like you can have the most boring conversation in terms of like saying the most boring things and it's just going to be like, that's so interesting, what about this? It's just going to want to continue talking forever basically. And I think it's the same thing there. It's just the sign currently is used for you to engage as much as possible.
And this whole conversation reminds me so much, actually, of some of the more nuanced research on social media. And I think often we're like, we take this blanket approach of social media is bad. It's like ruining the children. Everything is terrible about social media. But actually, there's quite a lot of research showing that there are, you know, many people who benefit from social media because it gets them to actually augment their real life
relationships. So these people are not spending too much time on social media and they're just like using it to connect with people who they like actually see in real life, not as a substitute, but as a sort of supplement.
And I think that we can kind of think of LLMS in a similar way where if you're using them to not entirely substitute your real life, you know, therapist or your friends and so on, but perhaps you are using it in smaller ways, then there's like a healthy way to approach it. So I think it's interesting that we're currently finding ourself in this, call it sycophantic era of AI where, you know, AI is super keen to talk forever,
always encouraging, always like if not prompt, otherwise blindly positive. And I think there will be kind of an interesting transition at some point where if you truly want AI to add the value of a human, we don't want it to be that way. We want it to be a little different. Like if we had AAI therapist for example, we would probably need it to tell us some hard truths and to not, you know, just bake everything into some form of empty platitudes and sharing. Correcting negative behavior.
If you say I want to do something that is, you know, socially unacceptable or threatening to other people's lives or your own life, then yeah, your AI therapist should tell you that's not a good idea. Yeah, right. Shouldn't say, yeah, I understand. Go for it. Yeah, and just be some form of counterbalance. We want AI to help people form a better understanding of the world and how they can live a good life. And currently it's not really
doing that very well. Whatever people bring to the AI tends to often times to make worse somewhere if someone comes with AI and have some really wild conspiracy theories at this point unfortunately, with the exception of using Gordon Pennycook. 'S the debunk pot. Yeah, except for the debunk pot, they will find themselves like probably deeper down that conspiratorial route. But yeah, I think this is always leads us quite interestingly into our episode of Today with Allison Cereso.
So Allison is a clinical researcher, psychologist, professor, and senior vice president of research at Empathic, overseeing AI tools that supports therapists so that it can deliver more accurate empathetic care to their patients. So this really strikes this interesting balance between understanding when and how to
use AI athletics is today. And what Empathic aims to do is provide AI tools that gives feedback and suggested text to therapists to help them be more empathetic to their patients. So basically maybe again as we talked about like not looking at the extremes of whether or not using AI at all or using AI completely, but Morris is a form of middle ground of seeing AI
maybe as a Co therapist. And really interesting given our episode with Mickey Insleck where we talk about how a generative AI actually does such an incredible job. At least, you know, simulating empathy. Then actually you can use it as a tool to then coach or teach
therapists, be more empathetic. And in our episode with Allison, we talk all about this, how empathic works as this sort of Co therapist and the role of AI versus humans in therapy and where they can really kind of help each other, balance each other out, where it's important to have a human in the loop. And finally, how we can use these tools to even mitigate
bias in therapy. So really exciting discussion and we hope you love it. So I'm very happy to say welcome Allison to the Bible Design Podcast. Thank you so much for having me. It's really great to be here
with you both. Yeah. And we're excited to get into so much of the interesting stuff we hope to cover with you because you are, I don't say really in this kind of fascinating space of trying to see what can be done on this intersection of thoughtful science based design, combining that with AI and how to leverage AI in good ways. So tell us about Imperfect and what do you do?
Sure. So Empathic is a conversation intelligence company, but we think of ourselves as expanding human understanding to foster clinical precision and to also foster precision medicine. Our tool is used in a number of places, but the majority at this time is in the clinical trial space. So we're often times in the background helping to make sure that clinicians are sticking to fidelity of a particular protocol of how they're supposed to run a trial. But we also are working in
clinical outcome assessments. And so again, providing AI 100% quality support to ensure that clinicians are able to do their work in a standardized way and to also have more precision in the way that they carry out their job. So I think often times you can think of us as like a Co supervisor or sometimes people use the word copilot. But for us, it's the way that we can really like embed or integrate AI into clinical practice in areas where humans
do need support. I'll just give the example of like, you know, in a clinical trial space it is, it can be humanly impossible to review 100% of what happens in every single session because humans get exhausted. But also it's just very difficult, you know, time and resources, whereas an AI can do it. And so it's a really great place where an AI can provide more of that coverage and not just do the coverage, but also allow clinicians or the humans to really focus on different kinds
of parts of the job. So to really maybe focus when an AI might be able to detect clinical risk than to go in and uncover what's going on there. You've referred to it as a Co therapist at times as well. Say that you're a clinician who's engaging with the tool. How do you sort of interface with it? What is your experience as a user? Yeah. You know, for us, often times a user like it, that can look
different, right? But it could be that it's somebody who's overseeing a particular site for a clinical trial or somebody who is overseeing that the assessments that people are doing in a clinical trial have been done in a standardized way. So often times, like I think, our tool is used by the supervisor who can then be flagged when more support is needed. The best example I can give is, so I am a former professor. I worked at UC Santa Barbara for many years.
I had my lab there, but also founded a trauma clinic. And for a period of time, it was just me and, you know, several doctoral students who were learning to be clinicians. And it was impossible for me to do 100% coverage of what was happening. It was a trauma clinic that was founded in 2020 during the Black Lives Matter movement, when it was really getting national recognition.
And so we wanted to be able to be a safe place for Black residents in the Central Coast, to be able to connect with therapist and just be able to talk about trauma, but to also really think of trauma in a more comprehensive way where it isn't just, you know, exposure to maybe a traumatic event like at war or something like that or a car accident.
But it was also just the ways that people had to navigate, you know, living in a world where there was a lot of police violence at that time and sort of just beating community, right? And especially for folks who were being triggered on a regular basis because so much media exposure to trauma or having to re see, right, like
violent events happen. And so at that time, you know, I was training this group of doctoral students to do better assessments around understanding trauma, understanding PTSD, and at the same time making sure that they also had the basics down of like, how do you connect with a client right in your first session? How do you do a comprehensive assessment intake and just provide support over time?
And so it was impossible to like, you know, sort of do my job, also get funds for the organization or the clinic, hire new folks. I do all of that. And so I was pretty like resource restricted. I mean, this is an environment where having kind of Technical Support would have been incredibly helpful, right? Because at that time, then I would have been able to use a tool like empathic to be in the background of therapy sessions.
And we do that. We're in the background of therapy sessions at times, but to be in the background to be able to flag to me right as the supervising and clinician moments where we're growing therapists, right? And junior therapists, like where they need more support maybe in detecting suicide, responding to suicide, but also just need more general support in all the different kinds of
ways. And so this is where I feel like AI can really help clinics like determine like what areas they can give more support to. I want to unpack. That a little bit more so when you think about like areas where you really need a clinician, where that expertise is really critical versus the types of tasks that you can sort of offload onto something like an AI, perhaps a clinically validated AI. How do you sort of think about this distinction? Yeah, we don't want to replace
humans. That's not the goal at all. It's more so fostering precision medicine when we're able to. And so I think like with your question that there's different ways that you can engage with AI to do some of the more like rudimentary things. Maybe, you know, getting the gist of like a clinical note together so that if you only have a few minutes between clients, the note is generated for you and you can go in and add, you know, like nuances that
maybe weren't captured. But I also think that there are moments like with the work that we do where it can help bolster the work that clinicians are doing, so help them be maybe more acutely attuned to conversational elements that would trigger that somebody might be an imminent risk of suicide or something else. So I think both, right?
Like, I think that you can do the more simple things and I think in today's world, I think that clinicians are often times maybe a little bit more comfortable with doing the things that, you know, that they do need help with and that like nobody wants to write notes all day like that. That is hard. Like as a part, you know, somebody who's like doing clinical work all the time, Like that was not the fun part of the job.
But at the same time, I think there's also other areas where I do maybe like, I want to engage with an LLM to understand, like, could I do this differently? Could I approach the way that I'm asking about maybe like the grieving process for this client? Like could I do that differently? Am I missing anything? Almost like a training tool. Yeah. I mean, I think there's so many use cases for it, right, in psychotherapy and other clinical spaces where psychologists are,
right? Because I think often times, like I'm very engaged with like the American Psycho Association and we talk about different ways that people are using AI. And I feel like often times we just completely forget that there's so many psychologists engaged in clinical trials as well or in drug development. And so I'm a clinician that's really driven to do good, robust science. And I make sure that the science also informs the way that I do
my clinical work. Yeah. So I think even with that, like there's so many different ways that I would use AI to do my job as a clinician, but to also do my research and. Here's what you describe as picking up on some of those things that maybe it's missed by a clinician during a conversation. Is that based on the words that are said or is it more even looking at how they are said in terms of the cadence or the tone? I think it's both, right?
So I think that sometimes we can pick up on literal words or like therapist utterances or provider utterances, but I think also you can pick up on things like synchrony or on therapeutic alliance, right? Or sentiment. And, you know, one of the best use cases that I can think of as well is that often times like our tool is in the background of like medical visits. And so we make sure that we give a report to the provider within a few minutes after a visit.
And the provider might see 1020, who knows, right, patients a day. And so they may engage in certain communications that might be off putting to a patient or maybe they just didn't pick up on a certain description of a symptom. And so Rai told, like, you know, from 7:00 AM to 7:00 PM is going to be just as effective.
But humans get tired over time and so it's our ability to give you the report on every visit and to make sure that you really do have a documentation or details that are going to be hard for you to remember. I. Think it's interesting in terms of how, as you say, clinicians are human. We have various kind of things
that we might struggle with. And so when I think one thing that I know from my good friends as a leading psychologist here in Sweden highlighted research around that, the best clinical psychologists are usually the ones that are in their first five to 10 years of their practice because they're
following the best practices. And actually one of the tricky things sometimes is when people get a little bit too secure in their own patterns of things where they kind of lost sight of what's actually the best practice, the best way of doing things. And they've used develop some good habits, but also maybe some bad habits over the years. And I'm just curious how you think about that as a challenge
and relating to this? Yeah, I feel like it makes me think of statistics where you would talk about like regressing to your own being, right? I think you're right. Like I think in this example, right, like AI can help you because, yeah, like I used to teach for a second year therapist and you need to pay attention to synchrony with like verbal and non verbal cues. And so especially when you're like a junior therapist, you're paying attention to every single
detail, right? And so, but I think over time, we become comfortable, we become confident in some of the ways that we engage in therapy are beautiful. But then sometimes we might not catch that we're actually doing things that might be off putting to a client. And so you're correct in that, you know, getting the AI feedback, The AI doesn't care if you're like a first or in your 20th year as a therapist or provider, but doesn't care if you're the attending or the fellow, right?
Like, it's going to give you that feedback in a particular way, in a way that isn't necessarily biased to who you are in terms of like, your training or maybe your title. But of course, like, AI bias is a real thing that we always have to build, you know, with that in mind. But I do think that, yeah, I can definitely see what your friend is talking about. And I think in this way, like, in the same way that you would do a tune up, right?
Like your car needs a tune up. You know, we need tune ups. I always talk about that. I think it's wonderful for everybody, but ADB in therapy, not just when you feel like you really need it, because it is this emotional and reflective tune up that a therapeutic space can provide you. So I think similarly, yeah, like an AI tool can really pick up on
things. And I would say like with our own platform, what I do love is that we give you a report according to, you know, certain detections, right, like ensemble models of collaboration trust, but you get to choose. We have 200 behaviors that we have models for. But in addition to that, it does give you the transcript and it'll show you like when the detections happen in the
conversation that you had. And then based on how you are connected to our platform, we can either show you that as soon as you click on in the transcript, it'll show you the audio or the video feed. So it's a wonderful training tool, right? And it's really hard to ignore that you might have minimized something with an older patient if the transcript and the video or audio feed shows you exactly what you said, right? So in that way, it can really
shift how people practice. When I was a supervisor, I would do my best to make sure, especially that junior therapist for recording those sessions and really paying attention to what they said, not necessarily the patient, but like how they were responding or how they were, you know, just engaged in conversations or if they were affirming or minimizing at times. And I think similarly, we can do
that with an AI tool. But here you can do it 100% of the time, right, versus only 1020% of the sessions that you're doing? I'm also curious, just thinking about individual differences in terms of the clinician's response to that feedback. So if I think about, you know, you just told me I've minimized the patient like, well, no, I didn't. What is the full range of responses that you see? I can definitely imagine someone being maybe more or less open to that sort of quote UN quote
feedback. Yeah, I think it depends on the person. It could be the case, right? That like we pick up on something that's minimizing and you say like, no, I don't think that it was. And so we have the ability for you to give a thumbs down. So on our end, we might see that as like a technology failure, right? That like, oh, we need to go back and like really make sure that we're capturing the breadth of nuance and like what it means to minimize.
So you absolutely can engage with the platform to help us build stronger models over time. But I think with anything, right, like if you were to create like a psychometric instrument of social support and you did that in like 1970, it's going to look dramatically different what social support is like in 1990, 2010 and whatnot. So I think that's one thing that folks always should keep in mind is that whenever you're working
with AI, it's not done right. It's an evolving process and that product should evolve with time as well. Yeah. When it comes to the feedback that they receive, is it only afterwards or do they also get some form of live feedback during the session? Our platform is able to do live, but for the most part, I don't think therapists are necessarily ready for live feedback. We have an NIH grant that we secured and where we are building out LLMS that are really focused on like synchrony
and precision medicine. And the hope is to get some user research where we can understand how our therapist engaging with it live if we if they were to find that to be effective or distracting. I think currently though, because we're not really training folks in this way, right? And I say this is somebody who was a professor last year. So like, I know we really aren't
training people this way yet. I think with time, yeah, probably that there will be more openness and receptiveness to being able to have like, live coaching. But I think currently that's just not how therapists are trained. So I'm not sure that we're quite ready to, like, make that match happen. I think the technology is there, but I don't know that as humans were there yet. All right.
I'm going to touch on a touchy subject, which I think is probably something that you've thought about before. One thing that you mentioned was like, we don't want to take therapist jobs, we don't want to replace therapist, right? I think every therapist would agree. I think the APA would agree. We probably all agree.
But of course, when we look at the landscape of AI that's very swiftly evolving, we see, you know, new RCT's coming out, new tools being tested, some clinically validate, many not clinically validated. We see people engaging with LLMS like ChatGPT as if they were therapists. We see these like, you know, AI health coaches, this huge, incredible proliferation of tools, some better than others. In many ways, it feels to me like the reality is that it's a
little bit out of our hands. Whether therapists are at least some being replaced by artificial intelligence tools, How do you grapple with this changing environment? And how are you thinking about the future? When we talk about working with an AI chat bot, I think that there are different ways that people are developing them. And so you could look at Wobot as an example where you have a deterministic AI, right? And so there are scripts on the back end. It's not generative.
And so there's like 100% control of how the AI model would respond to you if you're talking about anxiety or loneliness or whatnot. And then I do think that there's other ways that people are able to add some kind of like I'm thinking of like when you bowl right and you're a kid and you have these like kind of, you know, the. Bumpers. Yeah, there you go.
Right. Like, so I love the metaphor, the visual bumpers of how you would work with, Yeah, with generative AI. In terms of like working with therapy chat bots. I do think that if you can have a chat bot that is able to flag clinical risk and then make sure that a therapist is connected immediately, right, or flagged, then I think that's safe. But, you know, it's tough because I don't think that we have such clear guidelines or policies of like how these are supposed to roll out.
And I think that, you know that there are organizations like American Psych Association or American Psychiatric Association that are really actively working on it. The challenge is that like, you know, LLM development is so fast that sometimes the policies are not coming out fast enough to really map on to the pace of innovation. But I don't necessarily think that things are good or bad.
I think that if you are somebody who needs therapeutic support at 2:00 AM, there's not a therapist that's going to answer the phone call, right? Like we have really clear boundaries for reasons that are important to have, but people are in crisis at different times. And sometimes to have a live human is cost prohibitive. Or maybe you live in a rural area and it is impossible to get to a live person more immediately.
So I do think that there are times where like virtual care or even therapy chat bots, like there may be made space for it. But I think that we have to make sure that the people building them are clinicians, that they do have expertise. And if you're in alignment with APA ethics, then yeah, you don't practice outside of your confidence, right?
It would be inappropriate for me to maybe take a client with the severe eating disorder today because that isn't an area that I've like really focused on in the last couple of years. So I would imagine that there's elements of an assessment that I'm missing, right? Or like interventions that are
just not actually helpful. So I think similarly when we're developing these kinds of chat bots, we have to be really specific that it's, and I saw them the paper, I think we're talking about the same paper where they talked about working with major depressive disorder, generalized ID disorder, and I believe eating disorder, but a very specific type.
And so it's clear that they were really focused on like we are doing an intake that is comprehensive and we're really understanding what the symptomatology is. And I imagine that they also did a comprehensive clinical risk detection, right? So I think that there are times where you can do it, but it has to be as careful as humanly possible in the same way that you would engage with that level of care in a clinic. I'll add this Therabot paper.
They also assembled custom data sets, so they didn't just Hoover the Internet as many of our LLMS do. So they used this data set which was based on evidence based practices like CBT, cognitive behavioral therapy. So we have a little more confidence that the data that's going in is training the model in a more reliable way than some of the other more unpredictable models out there.
Yeah. And I would say, you know, for us, like we recently built out a feature where we are doing like oversight of clinical outcome assessments. So if we built out this new tool for clinical outcome assessments, it's really important for us that we make sure that our AI tool is able to detect that somebody is talking about reported versus maybe apparent sadness. And then when they are doing a sort of oversight of the scoring accuracy, we benchmark it in with licensed clinicians.
So we always want to make sure that there's a really high level of precision. And I will say that like we have a much higher level of precision because it is an LLM, right? Like it's AI then you might expect between humans. But I think it is that like you have to really be building in a way that puts patient safety at the heart of the work.
And that's why whenever I'm, I'm talking with clinicians about like different AI tools, I'm always really focused on whether there are like clinicians and especially licensed clinicians on the team that are helping to develop the tool. Because we're LED with ethics, right? And we could actually get our license taken away for not focused on ethics. And so you have to practice and you have to really build with care.
Yeah. Given that currently what you've kind of illustrated in terms of the support given to clinicians in terms of having that kind of sense of someone that gives you feedback in terms of showing you if you missed something or maybe helping you detect signs of various kind of serious or maybe also some more subtle things. That is really useful.
And I feel like in some ways that same guidance could be given also to not a human, but an AI shotbot, basically giving fair about the same support as like some form of almost double AI team where you have a like a clinician AI and then you have this kind of support guiding expert AI like, and it helps make sure that the clinician is doing a good job and so on. Have that been something you've been looking at as a additional path to provide value towards an AI instead of a human?
In this area, we're talking about like clinical assessments that we ask our LMM to explain, like why did you choose that by like, why did you detect this? Like tell us behaviorally what you picked up on. And for us, it's like an area of transparency because you have your training data set. And when we're building, we make sure that our clinicians also memo, right? Like why they're choosing particular things. And then we compare that to what
the AI is producing as well. And so you can ask your AI agent to give you their logic of like why they chose a particular thing or why they are maybe scoring somebody as having moderate versus severe depression or something of that sort. So in that way you can have checks and balances. It's sort of how you think of adversarial your Gans, right? Where you have your two models that are sort of playing off of each other, doing their own
automatic checks and balances. Yeah, but you always have to have a clinician in the loop. Like, that's so critical all the time. But I think also because it's like you can't really ask folks to audit themselves. Like, we're so biased, right? And so I feel like, you know, the LLM would likely be less biased, but at the same time, it's like, no, this is where clinicians really matter.
So I think in two ways, right? One is that ask the AI to explain its logic and then map that onto what clinicians are doing. And then at the same time make sure that clinicians are reviewing portions of the work or especially areas where you have to handle it with more care you. Used to be a devil's advocate here. I guess I'll be interested to hear your answer with like, why does clinicians matter?
Or like in what way? Because we talked about this day of like, I think serving at the top of your license as an example, Like it's really important that we can help people to serve a top of their license.
And I think it would be really interesting to have a better shared understanding of what is the things that we do as humans in various contexts, like in a clinical context or in other contexts where we can do the most good and add the most value and so on. So I guess I'm just interested to hear from your end, like, why does clinicians matter? Like what is the most important aspects that they add to
treatment? Yeah, I do think clinicians matter because when we talk about clinical risk or when you talk about somebody maybe for example, or having suicidal ideation, the thing that you really have to pay attention to is whether it's imminent, right? So you have to very explicitly ask somebody, like, can you go home and be safe, right? Or you need to not be vague. You want to really ask somebody the exact question of like, are they going to kill themselves,
right? Like you have to do these kinds of things. And so we are trained to do that. We're trained to understand the nuance. We're trained to, like, remove vagueness. We're also trained to ensure that we really do understand, like what imminent risk means. And I think sometimes conversational output might miss that to some degree. And so that's where you want a clinician in the room. I think there's also things I do think multimodal technologies are being built now, right?
So, you know, AI is going to be like a year from now. Our conversation could be so incredibly different, but I do think that they're also things like affect, right? So you're trained as a clinician to be able to see depression, to be able to hear depression. It's not just what somebody says, but it's also how they present. So I think in those moments, that's where you do want a
clinician in the room. And then the final thing I'll say is that like clinicians are humans too, and we have biases and we're going to pick up on things that are salient to us, not just with our lived experience, but how we are
trained theoretically. And so if you're somebody who, right, has been part of the team that has like created a clinically validated training data set, but you are trained as a psychodynamic therapist, then you might be training an LLM to be picking up on family dynamics more than maybe behaviors or being out of alignment with values in the way that you would if you were doing. So our theories and our
trainings matter a lot. And so that's where I think you should have different kinds of clinicians in the loop because it also helps to ensure that your AI is not missing things, right, because it's been trained to only pick up on certain
characteristics. So one thing from my end when I get this question, and I think the same is true for things like Shachi PT will be the death of Duolingo or like whenever there's like this kind of very sensationalist takes around, he has developed really high capabilities. I would say that there's a difference between capabilities and in this case also a part of treatment is actually having the social accountability, support and understanding of another
human being. And like knowing that it's not a human being that cares for you and that's going to follow up with you and that you will follow up with. Like that is not something you can also use to replace just because the capability might exist within an AI. Even if it feels human, you know that it's not. Right. But it is interesting because I feel like Robot has put out some research, right?
They've had a lot of scientific papers where people sometimes feel like they can say more to an AI agent because they know they're not being judged in the same way. And so I do think that there's that. But then I also think that, like, you know, our company is named empathic, right? So like our core models are based on the common factors, which are like the key ingredients across all different kinds of theoretical orientations that we know really matter for behavior change.
So things like collaboration and trust and unconditional positive regard, right? And so that, that isn't the core of like how we built our models. So we wanted to make sure that we could pay attention to the nuance in popular media or just like in the ways that we talk about empathy, like in a more common way, it's this idea that like you're a nice person,
right? Or that, you know, that you care, but empathy really is like about, if you really pay attention to the behaviors, it's about accurate understanding, right? So ensuring that you really do accurately understand what somebody is trying to convey to
you about a particular instance. And so I do think that there are moments where sometimes the way that AI models are built, especially, you know, because that's why training data is so critical, that you can train your AI to be extremely empathetic and more so maybe than a lot of humans. So it's not just the training of like, hey, you missed this, you know, this symptom. But it's also like, hey, the way that you said this was out of alliance, right?
Or it's out of synchrony, it was not empathetic. And so here's coaching. Here's a report for how you could do that better. Yeah, I love that study from way back when with the Reddit doctor responses versus the generative AI. People found the Gen. AI response to be more empathetic than the real doctors. And of course, you know, many studies have been done since then really replicating this
finding. So I find that extremely interesting, knowing that the machine is not experiencing empathy, but it is conveying empathy. It is mimicking empathy to I guess like more convincing degree than the humans. Yeah. And but I would say right, like that's where like AI is not a black box. Like you have training data, right? And you train AI like built on
that data. So that's why it matters that the team building it is diverse because it needs to capture the breath of how people experience the world and what they respond to. There's a study that I loved they did use machine learning to track like Reddit responses around mental health and they found that.
You know, according to like the typical like, DSM criteria for depression, it was missing depression for black women because black women express different kinds of ways of like, whether that be guilt, shame, somatic complaints that aren't typically captured in the way that we think about depression. But because you would imagine that like the training data that went into the DSM, like the same idea wasn't built around having
enough diversity. But I think that there's really beautiful ways that you can use machine learning. And I think you asked this earlier, Sam, but it's not just like what it was said, but it's how it was said. And I think that's the thing that I care a lot about, right? Cuz I've had a long history of being a clinical researcher and I care a great deal about like how do you work with different
kinds of data? And I've always been somebody who goes back to like qualitative, quantitative mixed methodologies. Because I feel like you can with quantitative, you can tell that a relationship exists, but you need qualitative to go underneath. And like, even if you find something is mediating, you need to understand like what exactly is happening here. And so I think similarly, like if you use natural language processing like we use, then it's like paying attention to
conversational cues. And it's not just like a yes or no that something happened, but it's more so how are people communicating guilt or how are people communicating empathy and being able to get at that kind of nuance in communications in a way that is really hard to do as a human 100% of the time.
Yeah. And especially as you say, you know, being human that engages with like 20 patients per day with a limited time, like all these things contribute, I think to what we talk about empathy is maybe on the best of days, in the best of moments, you know, humans will be most AI. But on a regular day and it's all of the other things going on, it's hard to maintain the same level of empathy. You know, like they do talk about, like, decisional exhaustion. So like humans, we do bias all
the time. I do think that there's ways that AI can really help humans to, yeah, just to do work that is more precise, but also just more comprehensive that we've
ever been able to do before. So talking about bias, interestingly, we talked a lot about algorithms going to be biased, but it's often times, I think the interesting thing is especially for us as a paper scientist to see both side of that, like where this human bias, but there's also algorithm bias and kind of the best of worlds.
You want to kind of mitigate both of them, like you said, have good training data that supports good algorithms that are better trained on a wide range of diverse data, but also understanding how we can support humans in being less biased. What do you think about the most effective ways to mitigate both of those sites? Like mitigate bias in the LLMS but also mitigate bias in the humans?
Yeah. But first I'll say that I think that they're experts out there, right, who know a lot more about this than I do. And I still have a lot of learning to do. But if you're a researcher, I think that there's ways that like we know how to develop psychometric instruments, right? Like we understand that. And I think sometimes using the same logic where it is things like have you been comprehensive in understanding the content? And so that's kind of similar to like training data.
Do you have diverse people like that comprise your training data? But do you also have diverse people annotating your data to make sure that when you say trust that is, is it just trust for a middle-aged white man, right. But it's trust for the breadth of what human experience can be. And sometimes that can be impossible to do, right? Cuz it's like you can't get that for every single person. And we know that intra variability is oftentimes larger than intergroup.
But I think also you can test the performance of your AI precision across groups, right? So if you're benchmarking it against clinicians, you can see, is our AI performing with bias? Is it more precise for men than women? Is it more precise for white versus Asian communities or whatnot? So those are like 2 of the ways that I think about it more immediately is I always am paying a great deal of attention to the training data.
And then addition to that, once we have working models, right, then we do test the precision against different kinds of groups to see whether they are more or less precise. Like are they demonstrating bias? But then I think that there's other ways that you can do it with user research as well. And I do think that there's other elements as well. You know, the NIH grant that we got really was built around synchrony, but also like elements of cultural responsiveness too.
And so I think that's something that I'm so excited to be able to focus on in the next few months and to kind of understand like, what does it mean, right, to really pay attention to like maybe miss cultural opportunities or to not evoke how culture might impact how somebody is experiencing their mental health condition or responding to intervention. But I love this idea of doing this because I feel like so far right, like AI amongst clinicians is really focused on
like note taking. And now we're maybe seeing things, right, of like, you know, deterministic ways of like how we might respond to depression, which is great. Like that's a beautiful evolution and I love seeing it. But I also think that we can use AI in ways that that we've never
done before, right? Like understanding patterns of maybe how rural youth or youth in like certain communities, like maybe trans youth, like talk about their mental health that we can understand like communication patterns in a way
that we've never done before. And so I think that it's about doing precision medicine, but precision medicine means that you really are paying attention to like individual differences and doing your best to really develop interventions that like are going to be most effective for the patient in front of you, not like a general kind of intervention that's supposed to be effective for everybody. I do think that AI can allow us to do that in a way that we haven't done before.
All right, it is time to move on to our quick fire round, which we call to AI or not to AI. Are you ready? I don't know, yes. You must be, you know, it's fun. You're going to like it. We're just going to give you a bunch of tasks and you're going to tell us whether you think it's well suited to AI or not. OK. All right, first one a digital twin of yourself to ask for advice. No, I think I already have my journal for that, so no. OK.
But also because I feel like that's a pretty biased advice, right? Maybe it's what you want to hear. Exactly. I mean. Just give me the permission, please, OK? A playbook of cultural expressions of distress. I like that with a caveat though. I was like who built it? But yes, I like the theoretically. I like the idea of it. OK, what about this VR exposure
therapy that is AI powered? Basically providing some form of personalized exposure scenarios for patients based on their various phobias, PTSDS and social anxieties. Yes, and that's already around. And I think it's funny that you mentioned your journal because the next one is a fact checking journal. Fact checking I would be really worried about the training data there. So I don't know who's doing the fact checking but I do love I
have a journal I use. I think it's reflection and I love it cuz it also generates AI like gets me deeper. You can have AI prompts that make you expand on certain things. But you say, I had a great time at the library today and it says actually your mood was like 4 out of 10. It's more so like what made it good, like what would have improved it for the next time, you know? Yeah, and it brings us maybe to
this one. So predictive nostalgia machine, basically an AI that projects what you'll be nostalgic about in the future based on your kind of previous experiences. So like it knows that you know you've had some experience in the past. So like it tries to kind of let you know, like, hey, you should really appreciate this moment because this is something you'll be nostalgic about in a few years time. Yeah, the predictive analytics, Sure, why not? Sounds like it's AI for
mindfulness and I'm about that. I like it. OK, last one, this is the emotional soundtrack generator. So this is an AI, listens to your daily conversations, basically, you know, takes a peek at your life, generates a soundtrack that reflects your emotional state. Yes, but I think that I would want it to also generate soundtracks that Get Me Out of a funk. They can tell that I'm in one. Yes, good.
There's a lot of soundtracks there, like on Spotify for example, where it's like you're sad or happy, but there's not really one from like from sad to excited. From sad to happiest. Yeah, I love that. Yeah. And my niece had just the funniest. I remember when she was young, at one time, I was like, why are you sad? She's like, well, the music is sad for that reason. I feel sad and I want to keep listening to it. And I was like, OK, yeah. So chicken or egg? I'm not sure. Right.
And I think you can get us into a mood or sometimes get us out of 1. So. Yeah. Yeah, love that. OK. Well, you made it to the final question now, which is what is your most controversial opinion about AII? Would say that I think that clinicians can be better with AI than without. I think that's what I would say. And I think that I have been converted to that now, being able to see that AI can do 100%
oversight. But I think also, yeah, having founded a clinic, being the slow psychologist for a minute there, I now see that AI can really help with supervision and yeah, and quality care in a way that humans, just because life happens, not just that we get tired, but sometimes we have tough moments in our lives that can compromise, like our
precision. And so I think that AI can be a really wonderful tool, but I think it's a tool that that we need more than we think we do. So you're really let's embrace AI as collaborator. Let's embrace ethical, robust AI as collaborator. I love it. Well thank you, this was so much fun. Wonderful. All right. Well, thank you all so much. And that's a wrap. You've been listening to the Behavioral Design podcast brought to you by Habit Weekly and Nuanced Behavior.
Sam and Alene tell me. This season is packed with incredible insights about behavioral design and AI, so be sure to subscribe and share the podcast with your friends, though you might want to keep it away from your enemies. In case you haven't noticed, I'm an AI voice. Yep, pretty crazy. Quite the improvement since last season's AI outro, don't you think?
If you'd like to collaborate with us at Nuance Behavior, where we use behavioral design to craft digital products with Nuance, e-mail us at hello@nuancebehavior.com or book a call directly on our website, nuancebehavior.com. A special thanks to the amazing Dave Pizarro for our show music and to Mei Chen Yap and April English for their help in producing and publishing this episode.
Thanks again for tuning in. We'll be back soon with another exciting conversation where behavioral design and AI Intersect happens to. Mugatroid.