47. Designing AI Experiences: What to Consider (feat. Caleb Sponheim PhD, NN/g)

This is the Nielsen Norman Group UX podcast. I'm Therese Fessenden.

I'll be honest, sometimes it feels like you can't throw a rock in any direction without hitting another AI headline. But more and more companies, tech or otherwise, are integrating AI features into their products. And that means designers who may have never worked with AI before suddenly have to navigate this new terrain, whether they signed up for it or not.

So to make sense of it all, I sat down with Caleb Sponheim. Caleb is a UX specialist at NNG who has been closely researching AI trends, tracking the latest advancements, and analyzing how designers can rise to the occasion. In this episode, we talk about the current state of AI tools, so what's working and what isn't, how designers can better position themselves to work with AI, and the critical role of safety in AI development. There's a lot to unpack, so let's dive in. Here's Caleb. Caleb

Thank you for joining us today on the show again. It's so fun to have you back. I know you were on a previous episode with us featuring Savina Hawkins, but it's nice now getting to really spend more time with you and have Sh just dive into your experiences. So to kick us off. Could you share with us a bit how did you get into researching AI? Because naturally, this is kind of a whole

I don't wanna say new because it's not the newest thing, but there's just so much buzz now that it feels like research into this space is particularly new. But how did you get here? W what what brought you to this path? Um well I was born in Minneapolis, Minnesota. No. I think um first of all, thank you for having me again. It's super I'm super happy to be back, super excited to be here.

As as you may know, my my background is in uh science. I was a scientist for a decade and studied uh neuroscience, psychology, my PhD is in computational neuroscience. And One of the Sorry, can you share what is computational neuroscience? Uh yes, I'm happy to happy to define that. Um computational neuroscience is the practice of Bringing I would say mathematical approaches to understanding how. the brain works. Most generally.

And computational neuroscience is particularly helpful when trying to understand how all of the hundreds of thousands and millions of uh neurons, brain cells in our heads, Work together to do all the things we do every single day, whether that's understanding what we're looking at, or planning what we're gonna get at the grocery store, or uh doing this little motion uh for listeners I'm moving my hands around. All of that involves uh coordination.

Across many, many, many, many, many different small little uh cells in our brains. Little electrical impulses running around. And fun fact. Many of the foundational discoveries and innovations in artificial intelligence. Come from neuroscience. I'm not just tuting like our own horn here. Like legitimately, many of the advancements that br have brought us to where we are today, ChatGPT, all these different things.

are due to us learning how our own brains work and emulating that and taking inspiration from those things to create artificial neural networks. You may have heard that term. Neural networks. That literally refers to emulating aspects of what's going on in our own heads. So there's a direct line between Uh trying to understand how the brain works. And the type of work that well.

Personally I do now, but the entire world is obsessed with at the moment. Now the two fields have kind of diverged and are still interrelated and inform one another, but it's not like Chat GPT is a perfect copy of your cousin's brain in there. That's not really how that works and there's lots of fun tricks that computers can do that brains can't and vice versa.

So I've been exposed to these types of methods and algorithms and approaches for years and years and years uh in relationship to neuroscience. And so it was a fun and fresh opportunity when I was given the option to speak about it and start um instructing folks on how to apply artificial intelligence. to their work and how it might inform how we can make products and services and features more easy to use. So

That's kind of how I transitioned into researching AI. And as new applications came out, it was a natural interest of mine to say, let's not only figure out how these work, but how they interact with. and integrate into user experiences.

Got it, got it. Yeah, and you've done research with a lot of other organizations as well, like across your you know, re I when I look at your your bio, there's loads of experiences with the NIH, with NASA, like and again, try I if you're not gonna toot your horn, I will because I think there's so many great experiences here that I think really make you well equipped to be someone to talk to today because

In many ways you've seen lots of different ways that there's different networks form and like how to process different kinds of information. Um and so

With that in mind, the research you've been doing now, could you tell us a little bit more about it? Right. I I know it's naturally around AI tools, but could you share a little bit about what you've seen uh over the last couple years? Yeah, yeah. So the work that I've been doing at N and G Um was originally around a lot of quantitative research, um, doing quantitative user experience. That was kind of my wheelhouse, shifting into AI research.

There is so little known about how to construct and design a quality user experience when it involves artificial intelligence. There's positives and negatives to that. So the the the the downsides are that there are lots of bad examples out there, and there are many people clamoring for any sort of Heuristic, any sort of objective ground truth of here's how we can bring value to our customers, to our users. with AI. And there aren't many examples out there.

The upside is there's a lot of low hanging fruit. Yes, we have to construct like a well structured research question on what we want to find out about AI. But we're going to learn something important. about how users maybe understand artificial intelligence, maybe the limits of current AI products, what users expect and want out of a quality product that uses AI and maybe even advertises it.

We're in the midst of a study right now, looking at AI chatbots in particular, looking at a user's expectations, mental models, their levels of trust. Around when they know that they're interacting with an artificial intelligence algorithm via a chat interface. Now that's obviously not the only place where AI shows up in our products, services, and features.

They're all over the place. And the potential for AI to improve the user user experience doesn't just happen on the front end, it can happen behind the scenes as well. So Not just the study that's happening right now, but we're bringing existing knowledge, we're bringing uh our own original research to this new class that we're debuting soon. about designing AI experiences that ideally will help folks not just create a good AI experience, but also decide whether to use AI at all.

Right. I am not the I am under no aspersions that um AI should be used everywhere. In fact, I think it's probably being shoved into too many things at the moment. Um, not that there's not potential. But many folks are being driven to introduce AI into products with the wrong incentives. Not to actually improve user experiences, but because there's some other priority at play.

As user experience professionals, we have to prioritize business uh priorities. Absolutely. We need to keep that in mind. If we didn't, we'd be bad user experience researchers, designers, um practitioners. But it is my opinion that If there's some higher up priority on integrating AI into products right now. There may not actually be a good business case for it. And at the very least, a not a very good profit.

Case for it. So we talk about all that in the course. Like so obviously all the research involved will help us. Um, inform how to design good experiences, how to be involved in the development of these algorithms and models and frameworks, but also how do you deal with people? That are yelling at you to integrate AI when you know in the back of your head, maybe it's not the best idea.

Yeah. Oh my gosh. So two major things really stood out to me in what you've just said. So the first or l I should say the latter, since this is the kind of the second thing you mentioned, which is that not everyone needs AI. I think that's really kind of

Maybe there's something like in the corner of your brain, like as a practitioner, you might know and you might just feel it, but you don't really have the right words to articulate. So it's exciting to hear that the course is gonna tackle, you know, really tangible specific ways to address some of these clear

misalignments for lack of a better term, right? Or discrepancies between what people are ultimately being asked to do, which is perhaps, you know, a appeal to investors, appeal to um customers who might be trying to like you know, compare between two different products and services, and maybe even the term AI just sounds interesting enough that.

it it seems like, oh, if we can just hook this customer because we're promising AI, then that might be worth it alone. But then the downside, which is what you're saying, is there are lots of AI tools out there that are terrible, which is kind of the first thing you said. And uh and I wanna really pause on that and let that sink in for listeners. Not because I'm trying to be like a a Debbie Downer here and say, uh.

you know, AI is bad. I do think, like you're saying, there are lots and lots of opportunities for AI to really revolutionize not just the way people work, but the way people live day in, day out. And I know plenty of people who use it daily for various different tasks to think through complex problems. But I also think there's this

And sometimes I nickname it the Amazon effect. It's and I'm not trying to pick for Amazon specifically, but it can be like the Fang effect, right? Where it's just like if it's a Fang company and oh, another company like Meta does this. Um Amazon does this, therefore it must be a good design. Right? There's often this sort of Stereotype or assumption that

These massive organizations have huge research budgets. So naturally, they must have researched this. And therefore, that's an automatic pattern I should adopt because I don't have a research budget. I'm just gonna defer to the experts. But yeah, one huge challenge is that not Well, a couple challenges, right? On the one hand, you might not have the same user base. Um, you also might be operating from an entirely different context.

And, you know, those two things alone might make something completely inappropriate, even though this might be a standard established by like a huge

tech company, that doesn't necessarily mean it's right to do it. And the reason I'm bringing this up is I think especially in the context of huge you know, company like AI companies specifically, like a and not to name any particular ones, but of course, you know, OpenAI anthropic, there are many, many other great companies that are all operating with a very similar

structure, right, which is a chatbot. But sometimes chatbot isn't appropriate. So I would love for you to kind of tease a little bit. Um, you know, what might be Something that does work well versus something that wouldn't, right? Because obviously there are lots of bad patterns that we've seen in our research and that I'm kind of privy to, but maybe your audience isn't quite sure about because maybe they've had only good experiences with

Chatbots, and maybe there are some that are not so great. Why are there certain bad patterns with AI? So I would I have I have two examples in mind. And I appreciate I appreciate you calling all of that out. Uh I bet if you're listening to this, there you have had experiences with AI that didn't meet your expectations. But perhaps you maybe interpreted that quite generously. I certainly have in the past, right? Um maybe I am interpreting some tool's utility.

Through the filter of being excited and passionate and optimistic about AI in general or the future of AI. I might see a feature or a product that's being presented maybe in a beta stage or a research preview or something like that and say, wow, this is a great sign. of things to come. Right.

Maybe you uh maybe, and I haven't had the option to do this because I'm not going to pay$200 a month to be given the privilege to do so. Uh, maybe you've experienced OpenAI's recent operator, their agentic framework. And By all metrics that I can find, it's actually a horrible agent. It actually doesn't do what you want it to do. It asks for um Side note, can you share a little bit about what an agentic experience?

Okay, yeah. So, um Agents are what the next are are what the large AI invested Companies Are betting on to deliver large amounts of value to customers ahead of A larger breakthrough. It's their attempt to provide a service which can automate tasks that previously weren't. uh feasible to be automated. The promise of agents are essentially autonomous. actors which once given a task will interact with

other products, services, websites by interpreting your task and maintaining some independence to follow through and deliver you some result. That might mean Ordering it from a um a consumer standpoint, it might be ordering you groceries based on what's in your fridge. It might be The classic example, which I actually think is a horrible example, is uh ordering uh arranging travel.

It's funny that you mention it's a horrible example because when I think about it, like that's probably one of the few tasks I hate having other people do for me. Like even when even like family members try to do it, I'm usually Can I can I just see what you're planning for me before I commit to number two?

preferences that we have about how we move through the physical world, especially if we're getting on a plane and we're staying somewhere where maybe we haven't been before, the number of preferences we have are so many. And the chances that an agent, an autonomous system, can correctly Uh, either anticipate or ask the right questions to acquire all of that information is so low.

And so I don't understand why AI companies are using this as an example of how it might work because it it there's there's such a low probability that it will actually work even once these tools get better. And let me be clear. You can run metrics on these tools. You can ask it to perform the same task.

six times and it will fail five out of or six out of six times. These these tools aren't reliable. The promise of agents is certainly laudable, but the reality of it is that they're not they're they're not up to snuff. So anyway.

All I'm saying is, um I would encourage you, if you're listening, to um actually cast around and actually evaluate your own experiences with AI, and you'll you might realize that you actually have a a wider spectrum of experiences from maybe spectacular and revolutionary to A bit of a letdown, perhaps. Now the two examples I'm thinking of

are uh one uh one category of AI applications that you might not realize is AI because it's not shouting it from the rooftops. And these are Recommendation algorithms. These have I mean these have been around for a long time and they've been labeled uh ML, machine learning, neural networks. Right now they're labeled AI.

These are all algorithms which learn a pattern based on data that are given and then reproduce that pattern, generate a new pattern. In a recommendation algorithms context, it is: let's say, let's uh an e-commerce platform. It learns about your browsing history, your purchase history, where you um lingered on a listing page. And brings that into some sort of feature to recommend other products related to that activity. Those tend to be quite successful.

And there's no s uh AI sparkle icon next to that section. There's no check out our AI feature on recommended products. It's just something that clearly delivers value without having to try and work extra hard to convince you that it's something nice and flashy. So that's one example of like a good uh kind of AI implementation that you might not think of as

Yeah, I think that's a really important point. Um, because I do think right now when people think AI, they think of it synonymously with chat. But like you're saying, there are certain implementations that are just so effective. That they are invisible. And what's really fascinating too, like when I talk about prioritizing user needs.

Most people that I speak with, especially if they're UX adjacent, like they get it. They understand that it's important. Sure. When I talk about business needs, and sometimes there are some folks who are really like zeroed in on conversions, right? And and the main Yeah. As as the main means of making money, which I understand. That is a way that you generate revenues to convert people into customers, right? That's how.

What is always really um, you know, in hindsight is always 2020, the best implementations of these different designs are ones that Do provide value, right? The recommended products. Does that do a great job converting people? Probably. If they're like, yeah, now I have exactly what I want.

Did I have to shout from the rooftops like, Wow, what an amazing AI tool this was? No, right? But but someone simply being able to find exactly what they need, that does a heck of a lot for conversion, right? So In in many ways it might not be like an immediate, you know, one step like this one change.

um can directly lead to conversions? No, but maybe we think of it in a more long-term sense, or maybe we think of it in a long-term loyalty sense, or even revenue per visitor. Are people buying more things because they're able to find more things as opposed to converting more quickly? So even just thinking about how we define success, you know, I think also changes the way you might think about AI itself and how you implement it. I mean, you're speaking to the perennial challenge.

of UX, which is how do you prove UX's value and worth in an organization, especially when The value, the immense value that usability and attending to user needs brings is. stretched out across a long tail of time and and Uh kind of obfuscated across departments. Our our our impacts are diffuse because user experience touches everything.

A big new thing About UX professionals involved being involved in AI development is actually how we can be involved more broadly in the process. So what I mean by this is you there there have always been opportunities to affect Product development throughout the pipeline, throughout the process, right? Oftentimes in organizations we're thrown in at the end to validate.

Some sort of you know feature that's already been developed, right? We don't love that. In an ideal world, we're involved from the very beginning. With AI. So much of the end user experience depends on decisions that happen at the very beginning about these types of algorithms. A great example of this is training data. The data that the patterns that are given to these algorithms to understand, to learn, uh drastically affects the outputs that they have.

User experience professionals have a real opportunity to help curate um gather and provide training data that reflects the needs of the end users. Not only will that actually improve the product, but it will also make our jobs easier when it comes to actually developing and designing what that experience looks and feels like at the very other end of the process. If we are handed a bog standard large language model and said, hey, make this easy to use, our hands are kind of tied.

Right? An example that I'm thinking of is a pretty well-designed if generic chatbot on uh some car dealership websites. That car that some car dealerships essentially just said, hey, let's just plug ChatGPT 3.5 into our customer service system and um let's see what happens. What turns out is you can get users that end up getting offers for a dollar on a Chevy Tahoe from 2023 or something like that, right? It's not great.

And and and user experience professionals can only do so much if the underlying technology can't support. A good user experience. Now we've always had to go back and forth with other stakeholders, right? We've always had to um negotiate with developers. We've always had to negotiate with pe product managers, product people, and so et cetera, et cetera. The stakes of those types of interactions and uh kind of get heightened, I would say, when it comes to AI.

Because the the impact on the user experience becomes Even more heightened because these models are probabilistic. They don't always act the way you expect. They're a little bit harder to test. They're a little bit harder to predict how they're going to work. And so UX professionals, we need to be part of those development conversations much more than maybe we have been in the past with traditional products, features, or um services.

Yeah, yeah. And actually, now that you mention this element of unintended uses and um being involved in development and

Even the concept of how these language models are created and trained, what comes to mind for me is this term about safety and AI safety. And there are lots of companies. naturally talking about this because that is a huge concern, both for investors and even users themselves, who are

concerned about safety. And so I feel like that term safety is very big. And maybe this question is therefore very big as well. But what is safety, when you think of AI safety, like what does that mean to you? And are there genuine ways that Designers can incorporate safety into their work. I'm gonna answer this question by actually dividing the word safety into two different buckets. Safety means different things to different people.

When you mention safety in Silicon Valley, if you walk into open AI and talk about safety, what they respond with is How likely is it for an AI model to take over the world? How likely is it that an AI model has secretly achieved sentience and is hiding it from us and is going to turn the entire world into Grey Goo? These are apocalyptic scenarios that drive investor behavior to want to be first, to try and reach.

What they call artificial general intelligence, AGI, or ASI, artificial superintelligence, to reach those before other people to try and avoid catastrophe. The things that investors and founders of AI companies aren't thinking about are things like privacy, things like mental health, things like intellectual property. There are Teenagers who have ended their lives because of relationships with AI chatbots. This is true.

There are people that are sharing large parts of their personal lives with AI chatbots that have poor privacy standards and are using that data to train future models. These types that

fabric of safety is incredibly important and I believe is underinvested in when it comes to these kind of frontier AI companies. Now, if you're working at an organization that maybe developing your own AI models or developing your own features, you have an opportunity to think about the costs associated with introducing AI. Thinking about the worst possible outcomes that could occur after shipping such a product.

Now it's not the best to kind of be in that horrible headspace forever, but it is useful to do it early in the product development process, to be like, all right, if we shipped this product. And it failed spectacularly. What happened? What could have happened to cause this thing? And it's possible that a safety issue could be part of that.

So there's safety issues when it comes to end users and real costs and negative impacts that occur to them. There are also safety issues that can impact a company. If you're using something like a third party model, so let's say you don't have the in-house talent to hire, you know, f five uh AI researchers at five million dollars a year. Who does, right? And you, you know, use OpenAI's API to, you know, use one of their models in your product.

If you don't have the right safeguards in place, you can end up at the end of a month with a bill that uh eclipses the revenue your company makes in a quarter if you're not careful. Um those bills for these types of services can go through the roof. Still, even with the advent of recent models like Deep Seeks R1 and other more efficient uh open models, AI is expensive.

And you have to make sure that it is worth the cost, and you're not accidentally running into a situation where you can't possibly have a return on your investment. Yeah. I think that's really helpful. And in a way, like the those definitions, right? There's the safety to the immediate user, there's safety to the company for like financial sustainability and security.

And both of those are incredibly important. And one thing I find particularly invigorating, at least for me, I'm like a risk management nerd. Yeah. Reading about risk management. I like thinking about, you know, contextual factors. I read like global risk reports and like that's stuff that I love to read about. And we can't predict the future, but

keeping that sense of, you know, a multifaceted definition of safety, long term, short term, even medium term definitions of that, I think are really valuable. Yeah. And so a couple more questions actually. I I know you mentioned a bit about how there are fundamentals of design and, you know, keeping things human-centered that have not changed. And these fundamentals still hold up even when we're looking at some of these emerging technologies. And just because we have AI as like a new

tool or a new platform through which we're delivering value, we're still trying to deliver value. And so with all of that said, do you have Any advice for designers? Right. And there there might be designers who have already been doing this for a long time, but are looking to break into AI design and feel like a little bit of a barrier, right? Even though those fundamentals might be important.

There might be some particular skills that could be useful, you know, if they are going to bring value to AI firms or otherwise do AI-driven work in their current role. Do you have any tips?

understand the technology. The you already have the skills necessary to design a great interface. You already have the skills to conduct great studies to learn new knowledge about user behavior. What's important when it comes to AI is to understand AI technology itself. How do diffusion models work? How do large language models work? Why is training data so important? And not just from an academic point of view. You want to learn about this technology.

So that you can understand what can change about the technology, what potential it has to improve a user's experience with a product. There are things you can do with large language models to change the way they behave, to be more or less transparent, to engender more or less trust, to be more or less human-like. esque to le j to be more or less visible. To best understand how to work in design for AI.

You have to understand what can change about tech technology to be able to design things best. You also need to be in the room, and that's tough. Getting in the room where those decisions are made. is hard because it involves learning a lot of engineering stuff, learning a lot of development stuff, and building and strengthening working relationships uh over time. That the work of of stakeholder management and uh cross-functional communication is is not new, to be clear.

But being being able to curate the right set of Facts and knowledge and context about AI technology is essential for you to just do your job when it comes to AI, to be honest. Yeah, absolutely. Thank you so much for talking through all of this and in a way bringing a level of reassurance. I know there's a lot of fear, there's a lot of excitement, um bringing kind of this level headedness that I think is much needed. But I guess

To close out, do you have like if if you were to like put a billboard in front of every designer who's working on AI products and like they had to drive past it every morning? What would that one billboard message be that you would want people to really sit with as they leave today? Sure. Maybe not. Ha ha ha! Sure. You don't need to put AI in everything. AI is not a silver bullet to solving almost any problem. There are better ways to go about addressing your user needs.

That is y you can have that truth in your heart. And still go to work and have to work on that AI chatbot because your boss told you too. Like that that cognitive dissonance can exist within you, but just know that. Y y it it's not maybe not. Or yes, I I would say that's a really I love that, just maybe not. And I love it'cause it's just so memorable. But but even to like

to take it a step further to maybe not chat, right? Or maybe not exactly as it's prescribed to you in a requirements list. And maybe there are ways we can explore delivering value in maybe smaller, more You know, more more niche, more targeted ways and you know, maybe small is good too. Small incremental improvements can still be meaningful. Yes. And then the little like air freshener hanging off your your rear view meter says, it doesn't have to be like this. Ha ha.

I love it. I love it. Well, Caleb, thanks so much for those who want to learn more about building AI experiences, designing AI experiences. Where can people sign up for this course and where can people follow your Work. Oh my gosh, go to nagoop.com, sign up for designing AI experiences. Um, I will be teaching it essentially every month until the

Heat Death of the Universe, if you keep showing up. It's constantly improving. It's um we we change it every single time we teach it. Uh highly recommend. I'll be I'd be happy to have you there. Otherwise, find me on LinkedIn. I'm the only Caleb Smonheim in the universe, for better or for worse, so you can find me anywhere. Um

But uh also, uh, if you're if you are not on a product team that is developing AI experiences, but you are just interested in using AI more, we have practical AI for UX professionals. That is a mature course. That's going really great that we also update all the time. I just updated it with all the deep seek stuff.

And so that helps you integrate AI tools and use uh to get those stakeholders off your back if they're pressuring you to use Chat GPT to write your emails or whatever. Um, but thank you, Teresa, for having me. I really appreciate it. I'll see you all around. Awesome. Thanks so much, Caleb. That was Caleb's phoneme.

He's presenting his new course, Designing AI Experiences, this March. And you can find information about that upcoming event on our website, along with thousands of free articles and videos. On UX design. strategy, careers, and of course of UX in the design of AI experiences. So to learn more about any of that, W dot nn group dot com that's N G R O U P dot com And finally if you like this show.

please leave a rating and don't forget to subscribe or follow on the podcast platform that you love the most. This show is hosted and executive produced by me, Therese Vessenden. All production and editing and audio post-production is by the incredible Chrissy Richardson. That's it for today's show. Until next time, remember, keep it simple.