Predicting 2025 and Beyond with Jared Peterson

Hello and welcome to the Behavioral Design Podcast. I'm Simon Schultzer, and today we have a very special episode where we're joined by Jared Peterson, a real scientist and colleague at Nuance Behavior. I have done all that I can to convince him to join because I think Jared is such a perfect person for the episode we have today. Jared is not only an expert in decision science, but he's recently also been involved in AI alignment work, which is

perfect for today's discussion. He has also become a thought leader in the field with prominent writing including The Science of Context, and most recently, a fantastic review and ranking of all of the top payable science frameworks and models. So look that up. It's in the show notes. But All in all, Jared is the

perfect person to help us make sense of the state of behavioral science, the state of AI, and everything in between. As we're today, we'll be trying to look back at some of the key breakthroughs in 2024 and peering into the future, trying to make sense of the state of our field and what we can expect in 2025. You can expect from us some hot takes, some predictions for the state of AI and payroll science, and maybe even some nuanced thinking and perspectives.

We'll see. OK, let's dive in. Welcome, Jared. How are you? Thanks. I'm doing excellent. Our, I believe our takes should better be nuanced. That's kind of our label. Yeah, we we have to live up to that. We'll see. How are you feeling so far in 2025? What's your temperature? 2025, It'll be an interesting year, I am sure. I'm having a child this year, so that's the most prominent thing on my mind, but there's certainly some bigger things happening in the world as well.

But it'll be an interesting year. Have you had that thought of like, what world am I bringing this child into? Quite a bit. I'm very curious how education because you know, for me, my childhood is pretty much defined by like the schools like that's what childhood was, is like you just go to school all the time and like those are my like most prominent memories. What is school going to look like in the age of AII? Have no idea. It'll be very interesting.

Yeah, yeah, hopefully it doesn't have to be like Ready Player One or something like hopefully it actually it's going to be more like some form of utopian version. But I guess that's what we're going to dive into. But actually, before we look into this year, I kind of wanted to do something because obviously we're recording this in mid January and 2024 is already quickly becoming a blur.

But I thought it would be useful to kind of first walk through what I guess I'll be sharing some things that I saw as key moments to reflect on from the past year. And I'll kind of throw them at you and see what you think. And I hope that kind of looking back will help us better looking forward and kind of forecasting as well what's going to happen this year. So Are you ready for my top 4 happenings of 2024? Let's do it. OK, So for me, personal level, a big thing that happened was

really experiencing and seeing the shifts from AI being something cool that you could chat with to AI becoming something multimodal in in a seamless way. And so this kind of multimodal evolution or revolution, whatever you call it, I feel like that was one of the big things that happened last year where most people started to notice that whatever interface they were using, suddenly they could generate a image from a prompt or a voice from a prompt

or chat with a voice. It became such a kind of cohesive experience and accessible for most people to have that feeling of wow, I can talk to AII, can generate images with AII, can generate songs with AI. Whatever you you wanted to do, you could suddenly do it much more easily. So like quickly some things that popped out for me as like related to this open AI and the Scarlett Johansson lawsuit and eventually them launching advanced voice mode. I think that was a big thing.

I look forward to it a long time being able to chat with an AI that it was like sounded like human and something that was a big, big moment. And it was delayed because they they realized they couldn't make it sound fully like Scarlett Johansson in What's the Film Again? Her. Her, yeah. So we didn't get a chance to have that full experience, but that was probably a good thing. I think that gave us a spice of like the arrogance that exists in a lot of this AI labs and

that was interesting. Then we saw a mid journey releasing kind of what they now have a 6.1 model and a web interface for their model. Before you had to use the discord for mid journey, but like you can now generate like this amazing images. Like it's just insane what you can do and it's all now through an easy web interface. 11 labs made it possible to create voice clones. I cloned my voice, which was super surreal and weird.

Hey, Jen and Synthesia allowed for video clones, which I also did, which was even more surreal. And then on top of that, there was all of these tools. They can send out kind of clones of you to call for you and make voice clone like voice clone calls for you, which was like super strange. And maybe the last thing that happened around this was notebook LM was was one of the tools that came out towards the end of the year.

And they launched this kind of podcast mode where you can basically feed your own documents and then have two AI podcast hosts discussing what you gave them as like they were kind of having a shot and it sounded super realistic. So yeah, that's the multimodal evolution revolution. What's your thoughts on this here? I feel like I've barely touched it. There's so much going on that it's so hard to keep up with everything. I've.

I've done a few things. I remember my first kind of wow moment with the multimodal was I was struggling in Framer, which is the software we use for developing, like my personal website and also the nuanced behavior website. And I couldn't figure this thing out. And I took a screenshot of the page and I was like, all the instructions online are telling me to do one thing, but clearly the interface has changed and that's not working. So here's a screenshot. Tell me where to click.

And it told me specifically where I needed to click in that image. And it was like, oh, wow, that's pretty cool. Because like, that's the thing that, you know, if you have like parents or grandparents that aren't as tech savvy and you have to like guide them to like, OK, like here's like physically where you're going to click. That's what chat CBT was doing for me.

That kind of creeped me out because that's some really advanced image recognition, and then being able to translate what it's seeing in the image to actual text, it's pretty impressive. Yeah, that's a great example. And I can relate to having for many years been kind of the form of tech support for several family members.

So, but yeah, that is wild. I agree with you and I actually picked up tweet or I think it was maybe posted on on Facebook used the other day by do you know Paul Schrader? He he wrote taxi driver and regime bowl. Do you know him, the writer? No, So he's like a very, very significant screenwriter. And he had one of those moments that, like, similar to you, he had to vent online.

And he said, I've just come to realize that AI is smarter than I am, has better ideas, has more efficient ways to execute them. This is an extensional moment aching to what Caspero felt in 1997 when he realized Deep Blue was going to beat him at chess. As I just sent Chad GPTA script I'd written a few years ago and asked for improvements. And in five seconds it responded with notes that was good or better than I've received from a film executive or a fellow writer.

I'm done basically. So that is like someone sharing something and then having that kind of interaction with AI in some form of semi multimodal way. Though to be fair, I think chachibti is best when you already have some kind of base for it to build on. If you say start from scratch, sometimes it's not quite there because you kind of want to embed in whatever your human

insights are. So if you're making a movie and you have some kind of theme that you're trying to get through, if you are the one designing context around that theme, then it can really help you perfect that. But is it really going to do it from scratch? That I'm a little bit more skeptical about and, and most of my use are with AI is more about cleaning up what I already have.

I'll do some kind of stream of consciousness and it'll clean it up rather than here's the like, rather than just saying, hey, write an article or here's the germ of an idea, write the article. No, first I have to do some work and then it's better at cleaning up. Yeah, that's great take. And I will say that probably the thing I've said the most to people as like an advice of sort

is like leverage your expertise. Like if you have expertise that is super, super valuable with AI, because if you have no expertise and you ask for something, you're going to get something very generic for the most part. And you'll have, even if you get something really good, there will be something there that maybe is terrible and you won't be able to really pick up on it. So having expertise and leveraging that is is really powerful and that takes us to

something interesting like this kind of leads us into another thing that happened last year, which was surpassing human benchmarks. And that was kind of these AI models, especially in the frontier models. So the ones by open AI or entropic or even some of the open source one, they started to perform as well or outperforming humans in many tasks and at some tasks outperforming human experts or PhD level performance.

And so for example, it started with ChatGPT or like GPT 4 passing the bar exam, then more advanced version, I believe it was O one or it was 4 O that was able to basically perform as good or better when it came to some creative thinking and divergent thinking tasks. So it was a nature study around this where a were shown to be more creative than humans on divergent thinking tasks. So basically creating multiple unique ideas or solutions to a problem and trying to solve it.

There was one where AI generated poetry was indistinguishable from human written poetry, which is kind of probably where you would expect AI to be really strongly performing. It was kind of more of a less of a study, but more of a experiment of sort where 1278 people who said that they hated AI art were unable to distinguish between AI art and

human art. There were quite a lot of strong benchmarks from the one model from Open AI on various things from math to legal analysis to various things. It was pushing quite close to biology, science related topics, pushing quite close to PhD level.

And then we ended the year with this release of the report saying that Open AI SO3 model that has not been released yet, but has been used to basically pass what's called the ARC ADI benchmark and which is a benchmark to try to kind of approximate for some level of artificial general intelligence. And again, maybe with some arrogance, Open AI came out and basically alluded to that fact that like, yeah, we've kind of

cracked the code in some ways. So what do you make make of these kind of what I would call like the vibe shift in some ways in the talk around AI performance? It's really hard to know how important any given metric or benchmark is because of essentially Goodheart's Law. Goodheart's Law means that once a measure becomes a target, it

ceases to be a good measure. And in these cases, well, it's like there are so many test studies of the bar exam online that of course AI is going to know how to do that. And as soon as the AI companies are like, oh, here's the new benchmark, well, they publish it and then they train the model to

beat that benchmark. And so it's really hard to know is, are these fundamental leaps in performance or is this a very narrow, all right, we've made our AI just a little more general, a little better able to do this one specific task. And that's still impressive. That's still a cool thing, but it's not clear that it's as impressive as it sounds on the surface. Yeah, I think you hit the nail

on the head. It can be really impressive on one level, like doing something again, like trying to pass the bar exam because again, it has a lot of training data on that. But then you give it something relatively easy that any human would be quite easy able to make sense of. Like what's it called? Like this game of tic tac toe or something like some from a relatively easy challenge, common task fails completely. So we're still at a point where

it's in person. I guess what I would ask to kind of change the question here a little bit, what from last year what, what experience impressed you the most with AI? Like was there some form of specific use case or any general thing that AI was able to accomplish last year that you were impressed with? There's quite a few things I was really impressed with Notebook LM, the podcast feature. So basically in Notebook LM, what you do is you upload one

paper or two papers. You know, I, I sometimes upload 3 or 4 academic papers and then I ask for summaries and it'll very closely reference the source material. So we're getting away from the hallucinations, which I think was just like a major roadblock for trusting LMSLMS. And now that I can see, OK, here's what the LLM is saying and here's why, because it'll have like a footnote and I can reference the actual text from the original source materials.

So that was really impressive. The podcast you can't do that. You can't see why it's saying what it's saying. So I'm, I, I trust it less. I'm I'm afraid that it's hallucinating because I can't verify that it's not. But I think that was a major breakthrough. And another major thing that really impressed me was my wife asked me for something in her in the bathroom and I opened this closet and there's 2 billion things of makeup and lotions

and, and whatever else. And I have no idea what any of it is. And the thing she asked for, I have no idea, like what color it is. I don't know the brand. And so I just say, I open up ChatGPT and I say, hey, ChatGPT, you know, look inside this closet and see if you can find this thing for me. And I turn on the camera and scan and it finds it and it's

like, Oh my gosh. Because like it had additional context that I didn't in that situation, which is like, you know, it knew what I was looking for. It knew the likely brands and therefore it knew what colours it was looking for. And I didn't have that context. Now most of the time I have way more context than Chachi PT, which is why I can do things that it can't. You know, tic tac toe, it sucks at it because it doesn't, you know, it can't even remember where the XS and OS are.

And so I have this context, but we're getting to a point where now it has more context than I do, and it makes it more impressive than me at times. And you're staying with that. I think we're probably starting to feel like at some moment we feel a little like Paul Trader in terms of like we feel a little outdone by AI. How are you feeling about that sitting without feeling like, is that something that you're excited about or is it uncomfortable or how do you feel

about that? I, I wish I had an answer. We've talked about this a bit online, but for me it's a bit of a cycle. Sometimes I'm really excited and it's like, wow, this is just so impressive. And other times it's like, man, this is kind of terrifying. And sometimes I'm like, this is, it's so good. And other times I'm like, well, it's not bad good, You know, look at these silly mistakes it's making.

And I'm kind of all over the place and it kind of just depends like minute to minute, you know what, what particular problem I'm working on. As you alluded to in the intro, I'm working on a couple of AI alignment projects. And sometimes the AI is just doing really stupid things. You know, one of the projects is about triage and like the AI is like, well, what if I put a bandage or a tourniquet around this guy's neck? It's like, no, that would choke him out. That would kill him.

Do not put a a tourniquet around this guy's neck. And it's so stupid at times, but at the same time so impressive. And yeah, it's so unclear. Is this something that's going to be able to do our jobs in a few years? It's it's so hard to predict something like that. Yeah, but so we shall very soon get ready. But yeah, I'll just quickly say two things.

So I, I think it is something that we probably should start to sit with a little bit more that feeling of like there's nothing that says that at some point, especially when it comes to specific knowledge or tasks or someone that AI is likely not going to be like more capable of specific things. And so you're going to practice something this year. It's probably acceptance. It's a good thing to practice this year accepting that kind of fact and then take me like

quickly around a notebook. LM I used it as a fun thing over Christmas because I picked up a lot of reading and I ended up like while I was reading, putting in some notes and some voice notes in my own kind of AI note taking app. I'm sorry. And then afterwards I kind of have this function where basically when I tell it to like summarizes things in a format that I like in the end.

And usually I used to have it as that because it like gives me kind of, OK, these were these were my thoughts from this book. But then I also put that into notebook LM and I created like this podcast, as you alluded to, and it was really fun because then it was this weekend. I was like, wait, I read this book. Like, what was my thoughts? Like what was this book about again?

And then I listened to this podcast that I had created back then and it was like perfect because it was like a 1520 minute podcast talking about all of the points that I had to explore, but also some further exploration around those points and summarize some of the stuff that happened in the book. So that'd really help with my kind of retrieval and a memory of the book and helping me kind of better, I guess, absorb it long term and, and thinking more about it in hindsight.

So yeah, definitely recommend everyone to use notebook LM because it's free so far and it's, it's very accessible. So it's useful. Great. And there you go, a free recommendation for a tool. Yeah. Any quick thing? I was going to mention what's interesting about Notebook LM is that it is so tied to the source material that I sometimes don't like using it because sometimes I want to generate new and novel insights and it's so tied to the source materials that it can't.

So I actually, it's still useful to kind of switch back and forth between something that hallucinates a bit and you have that risk because that hallucination still might be useful. And I can still validate that by going back to the source materials. And so I, I think there's sort of a pipeline where you want something maybe in the beginning that's really tied to the source

materials. But at some point when you have a little bit more expertise and you really understand the area, then you want to kind of turn up the temperature, so to speak, and and deal with something that's a little bit more creative. Yeah, no, I, I've noticed that too. And I think like you said, I think they have twist the temperature to be extremely like basically, so it's not hallucinating anything at all.

And usually for a lot of stuff like the sweet spot is around like .7 or something like you want to have a little bit of like hallucination or like allowing it to be creative.

But yeah, that's a great take and helps us now also to I think move to. I was going to talk about these AS22 events, but basically it's one is AI agents and and the second one was going to be like the dead Internet, like the flood of synthetic content. And I think they're kind of one of the same in some ways. So quickly the defining AI

agents here. So what I saw and experienced last year was kind of the thing that shifted was that suddenly not only could you use API directly with the model in some form of shot interface or or similar versions of using it, but there were more and more frameworks and more and more way to basically allow some form of autonomy and give them whatever language model that you want to use, give it basic access to

certain tasks. So initially it was like super simple stuff like you can create some form of your own agents that can send an e-mail for you or that can book a calendar appointment for you or do some of those like quite simple things. And then technically, as I experiment, experiment with this, you can then like add more and more tools or tasks that this can do.

And it can kind of like based on what you give it as a prompt or ask, it will has autonomy around what it chooses to use and and what combination of tools they might use in order to accomplish the task that you want it to do. And so that is I think something that is exciting because it's allows for a lot of potential use cases that it's not possible use with the kind of a one to

one exchange of sorts. And it started to have some form of call it implications already last year where there was a lot of frameworks. Again, as I mentioned that came out both how to use this in automation platforms and also like how to use some form of Python based or other type of code based frameworks for this. And yeah, I'll stop there. Jared, did you notice anything around this or any any thoughts on aged agent stuff? I'm still terrified to use an AI

agent. I just don't quite trust it enough. I think there's probably a few things in my life that it might be able to do. Like, I don't know, maybe there's something around my file system that it could clean it up a little bit more quickly than I could. But even that it's like, well, in some ways, the file system, my computer is, you know, a second brain for me and like, I need to know exactly how everything is organized. So what what can I actually

trust an AI agent to do? I I'm just not sure yet. I haven't found the use case where I'm like, oh, this would be really cool and I would really trust an AI to do this. I'm always just a little skeptical, a little hesitant. That's. Very understandable though, I think a lot of people can feel what you described. And have you noticed there was actually something papers that came out, especially the end of last year using agents in

research? There's one paper at Stanford and Google study, I think it was literally called generative AI simulation of 1000 people. Did you see that paper? I haven't, but I I assume it's similar to what other people have been doing, which is just using the AI as research subjects. Not exciting in this case like they kind of did, but like they basically what they did is that they aim to replicate 1052 participants. So they had two hour interviews with each of these participants

in the study. And then the team trained a generative AI model to mimic their human behavior. And I think the headline of the study was basically that they were able to replicate their personality with about 85% accuracy when it came to tests, surveys and games.

So of course, it's like very confined context that is not really like representing their personality in like the real world, but in a very confined setting, they were able to kind of like basically relatively well mimic the personality of these people with about two hours of interview. So the idea was, OK, we replicate these people. We have a version of them. There's like 8085% reliable.

And then we can then use them as study subjects instead of having the real people because the real people are kind of expensive. Or if they're hard to use or they're like maybe unethical to use for certain studies, but they're AI clones. We can basically put through another Stanford prison experiment and you know, I don't think they would even do go there because Stanford I think they want to keep away from Stanford person experiment 2 point O, but but technically

they they could in this case. So that was kind of the headline. And so it was kind of an interesting study, I think in terms of what is being kind of explored currently for this stuff. That's a super interesting study. It's very similar to the AI alignment project that I'm working on, which is about aligning to specific individuals, not to human values in general, which a lot of AI alignment projects are about general values, but about how can you be like this specific person.

And in some ways, it's kind of the easy problem of AI alignment. You can nudge an AI to be more like someone fairly easily, especially if you have quite a bit of training data or it's a narrow domain. If there's only so many options. Does that generalize? How well does that generalize? I think that's a big question.

Just 'cause I know how you're going to act in a few psychological studies doesn't necessarily mean I know how you'll act in the Stanford Prison experiment, for example, or how you would make triage decisions or any other very complex topic. So yeah, it, it, it is interesting. I'm very curious how well it generalizes. I'm not sure how you know, most of psychology is about trying to figure out how people do things and you know why they do what they do. And we're just not very good at it.

And it'll be interesting to see if AI is able to be better than humans. Yeah. And if it will be like, I think from our perspective as behavioral scientist, you know, the curious case would be whether they replicate or both better angels, but also like our vices. I will say that as a personal experiment, I did actually set up something that I never actually launched, but I basically set up, I mentioned before, like an AI voice clone of myself.

And then I set up like a voice agent that you could like I could send to have calls, like soon calls or stuff like this. And then because I have for have a weekly pro, I've been doing mentor calls for the last three years and I have transcript for all of those calls. I basically summarized those transcript and used all of this transcript as kind of rag database or, or training data for, for this AI agent.

And then I gave it some instructions as well to act as kind of like you're acting as Sam spam, a kind of AI clone of Samuel as kind of doing mentorship calls. And it was not really to ever like send it out and, and do it, but, but it was like, OK, I have a lot of data on this. Could I create something that would, you know, work and then

see how good it would be? And I, I tested it quite extensively and it was like, I was impressed with the advice at some point, like, OK, that's a little better advice than I would give really good advice. So, yeah, if you're interested, anyone you can e-mail me and I can give you a link to try it out. But it was definitely something that was kind of wild to see what it can do. But I wouldn't like ethically, like I could never still

ethically do that. I have teased around some version of like what would be the ethical version of this, but I just feels very odd to to in any way kind of do some version of that where you kind of create an AI version of yourself and kind of send it somewhere. It feels still very dystopian. So why exactly do you feel it's unethical? Is it because of the hallucinations or because it might reference something you said early? But now you've changed your thinking?

What? What exactly is the concern? I think the one version of this that I felt more comfortable with is where I've used it to create a video that basically has a video avatar on me that looks like a Pixar version of me. Because then it's very clear they're like, OK, it sounds like me. It kind of looks at me, but always like an AI avatar because

it's, it's animated. So anyone who looks at this will never get confused with me trying to intend that it's actually me. But however you set things up, you know, you never know to what degree someone is going to understand that they're talking with an AI version of you.

Or like that I feel like will be the scary part if like I somehow set something up and people don't really understand that you're talking to an AI clone or someone who's not a real person and they have a really meaningful conversation like my nightmare. And someone would be, someone would be like emailing me being like, Oh my God, what you said the other week was like, changed my life. Thank you so much. And I'm like, we have never spoken before. Like you must have accidentally

spoken to my AIA clone. Because then it would just feel like they've been tricked. And they would feel like such a weird thing because they've had a meaningful moment. And that is also that kind of weird thing right now where we can have meaningful moments with AI content. And that's OK, I think as long as you know that it's an AI you're speaking to, right? I don't know.

Does that make sense? Yeah. So if you could prompt it so that it would be like, hi, I'm Sam AI, I'm sure there's a good nickname you could give it. Sam spam. Yeah, Sam spam something that's under our mats. Our recent guest of the podcast said where, you know, the difference between science and science fiction is timing. And it was like, OK, timing has shifted here. Like, things are becoming quite strange. Then we'll skip talking about

synthetic content and AI slop. I think everyone can relate to this. It's just become so much from on Pinterest to X and Reddit to, you know, whatever. I think, I think the one thing we can say, because I know you made a note of it is the AI influencers things on Instagram. Absolutely atrocious. I have no idea why Meta thinks

this is a good idea. No one wants to see AI avatars on Instagram. That's not why anyone goes to Instagram. I like, is it just extreme, like arrogance or is it just a total misunderstanding of human nature of their own product? I mean, Facebook has been going downhill for a while while now. And maybe they're just like, kind of like desperate to get something.

I mean, it's already I, I, I don't know about other people's Facebook, but when I get on Facebook, it's about 75% ads or here's a, you know, a group that you should join, like meme pages and stuff like that. It's very rare that I see actual content from my friends anymore. And maybe they're realizing that's an issue and that they can somehow get around that issue and that make people perceive that Facebook is

popular again. But Oh my gosh, it's just no one wants to see what an AI has been doing. Yeah. And if you missed this, because this was kind of like they tried to cover up this really quickly, but the base launched, I think a handful or more AI influencers that were basically AI agents set up in Instagram with like really cringy names like dog mom or whatever, like various stuff

like this. And even like, it's a very specific take, but the even for some reason had one of the AI avatars had the face of an famous MMA fighter called Israel Adesanya or Salbunder. Because I, I used to train a little bit of all the stuff that I happened to know. And I was like, they have even failed this much on the launch that they just taking a real person's face for this, you know, content and whatever they thought was going to happen,

obviously like really backfired. And within 24 hours they backtracked and they shut it down and and so on. It reminds me of this version of I think it was Google where they launched the form of or maybe Twitter where they launched the form of chatbot that was kind of like basically growing with its conversations. The idea was that one, it was going to speak with its users. It's going to learn from how it speaks and then it's going to become a more of a kind of

growing individual. But like as naive or as tone deaf as some of these people can be. Similarly, in this big tech organizations, they couldn't imagine that like immediately this became like a racist bigotry spewing like terrible person this chatbot and that close down within like 12 hours. Yeah, immediately the trolls got a handle of it because, yeah, I mean, it's the perfect opportunity for a troll that they can just completely alter the way this agent is interacting with people.

It was, it was just too perfect an opportunity for the trolls to leave alone. And I think that's probably a trend we'll see in the future as well. Yeah, we have now I think set the scene a little bit.

We've talked about some of the things we noticed in the past year already, shared some opinions about these things, and we're going to get into predictions. But unbeknownst to you, Jared, I prepared a game for us because we've now talked a lot about like generative AI and all this stuff. And you're American and I have a game, which is can you guess the song, whether it's an AI song or a Eurovision Song? OK. Do you know what your vision is? I do, but I've never watched it.

OK. What is your what? What? Tell me, what is your vision to you? What does that mean for you?

From my understanding, Eurovision is the various countries in Europe have a artist that gets kind of nominated for that year and they have a song and they sing this song at the Eurovision contest, and the Best Song that year wins the Eurovision. So it's kind of like American Idol, but for all of the countries where every country has a Rep. And my understanding is also that the music videos can sometimes be a little over the top. Yeah, and we're like the dance

performances. The whole idea is that it's not really the Best Song per SE, but like it's a mix of what's most entertaining or kind of most out there creative. Like if you get a fair mix of basically, you name it, like any, any given your vicious song that does have basically the moments of like, what the heck am I watching kind of thing. But it's been around for a long time. It's well known that Sweden has won the most Eurovision of all

countries. So we are the the best country at Eurovision. And so I, I thought I will kind of play this game with you basically. And so I will basically play 2 tracks, about 30 seconds for each track. And you will, because I know you love predictions and prediction markets and all of this stuff. So after every song, I'll hit pause and you get a chance to basically indicate from zero to 100 whether you think it's

human. So 100 is human zero that you you're very confident that is it's AI basically. And yeah, then we'll see which, which song has the highest score and which one you pick is the one that you think is your revision. Any questions? Sounds like a musical touring test. I am very excited. I am not particularly a musical person. I am very ignorant of the world of music, so we'll see how I do. I'm. I'm probably the wrong person for this, but I'll give it my

best. Well, to be fair, like you're not being introduced to the kind of high fine dining of music here. Like this is The Dirty, how do you say like fast food of of music world. So your palette should be fine for this and as a listener, you can do the same. So feel free to join and and you make your predictions along with Jared. Here we go, song #1. You say that you're real. Could this be Gas Lion? I can't deny 1 feel how I am through horrifies. Each whispered clue leaves me

stuck in between. I hate how I love you still Austin. The screen is the moment artificial real life. I couldn't see what I can't feel and I hate how I love you. OK, so that's the time to make your prediction. 0% if it's AI, 100% if it's human, or somewhere in between if you're unsure. OK, ready for the next one? Let's do it. Here we go. Don't you feel the answer? Judy, we go. Can you feel it? Go, Judy, we go. Let me show you our new world. Can't you see it?

Can't you see it? This is the moment. It's so real. Eyes are burning. Trees light. Come together. Let's unite, Judy. We rise. We go. So let's just get to the scene. One song was competing in the finals of last year's Eurovision and the other song was AI generated by myself and using no sample. Use the prompt and song lyrics to So no AI, which was the platform I used. What is your prediction? So I guessed with 85% confidence that it was human the first song, and I was 45% confident

that the second one was human. OK, so if you had to choose, you would. I would choose the first one. Yeah. So now that you're telling me, I there's just, you know, it's 5050. I can change my estimates to be more exact. I'm going to go. But I'm 80% confident that the first one is human and 20% confident that the second one is human. OK, well, I'm pleased to say that you know your Eurovision, Jared.

Well done, well done. And I also have to say now, listener, I'm sorry, actually, neither of the songs you heard were from your vision. Because we don't have that Eurovision money, we can't license the Eurovision songs. So the listeners heard two AI songs, but the one you heard, Jared, the first one was Austria's song from last year, Colleen, she's sang We Will Rave was the the anthem that she had. And I don't know, it didn't do too well. Honestly. I think it was like some middle

performing. So it wasn't the best that your vision had author, but it was in the final. Like it was voted as one of the top songs obviously that year. So so yeah, you got it right, Jared. Good job. I am very proud of myself. I'm going to put this on my LinkedIn that I am still able to pass the Turing tests for music. Yeah, yeah. And I look forward to at some point hosting you in Sweden and watching Eurovision together. It's a blast. It's a fun experience. It's Cotton Eye, Joe.

Was that Eurovision? It was a Swedish song. I don't think they were competing Eurovision. I'm not sure. But yeah, this was a Swedish band called Rednecks and they they made a few hits in America and and yeah, don't listen to it. I would advise anyone against, but but your vision is coming up in a few months. So please, please listen and please watch next year's Your Vision. OK, Jared, do you feel warmed up and ready to make some predictions now? Let's do it.

So we're again early in 2025 looking into what seems to be a kind of a horrid predicted year. It feels very foggy where like there's a lot of lightnings at the horizon, like a lot of things are going on, but you can't really tell exactly what's going to happen because there's so much going on and, and we're going to do our best. And so I picked out a few subjects that I feel like these are interesting subjects to make some thoughts and predictions and reflections on.

And I also really want to hear your thoughts.

The first one, will it become a thing to have AI Co workers in 2025? So what I mean by that is that AI agents as we talked about have started to become more widely reduced. But the big limitation has been infrastructure that the integration of these agents has been hard because there wasn't really any kind of AI agent friendly platforms. There wasn't like you had to like kind of sticky tape it in some ways to make it work, but it was also a lot of data security issues for a lot of big

organizations. And so it was quite a lot of hesitation so far to actually integrate them in teams or to have like, let's say research assistant on Slack. That's like AI research assistant that anyone in the team could like ask a research question to, for example. But many predict that this will be the year for AI coworkers and Salesforce announced two things. They announced that they will stop hiring engineers and they will start hiring sales reps to

sell AI agents to organizations. Meta has planned to reduce mid level engineers. Zuckerberg was talking about this and Rogan recently saw some clip around. He was kind of bragging about like, yeah, we'll be able to cut out whale of mill level engineering and use a A agents instead. And so there's some signals around that this is likely to be a trend this year, certainly hype around it. But what are things, Jared? Do you think there will be AI Co workers popping up in 2025?

The resolution criteria for this is difficult because what what counts as a Co worker? Certainly it's going to be the case that companies are going to be increasingly using some kind of chat bot. They're increasingly going to be using agents that are programming that are doing lots of different tasks. Are people going to think, oh, here's my AI Co worker, or are they going to think, here's this tool that I'm using? It's not clear to me like how

how you make that distinction. So I, I don't think we're going to get to the point where people are going to be thinking where an agent is autonomous enough that it feels like a Co worker. Rather it's only going to be autonomous enough to make it feel like it's a tool. So I'm, I'm I'm going to predict no because people aren't going to think of them as Co workers.

What if it was? So I have done with some tools, for example, that you're on a call like this and there's a third person on the call and you can give them access to certain things. You can have access to the web. You can give access to some like actually some internal documentation and keep it like local. And maybe some like tasks like maybe be able to book calendar appointments or or various things.

And so we could have in the same way as people are starting to have like no takers that you can now very likely this year at some point have your no taker actually be able to execute on task for you on call. And so you could say like, Hey, Alexa, you know, add something to my shopping list is something we we've been saying for a while, but now you can more like, hey, Fred, book a calendar appointment for next Thursday or can you look up this thing or like, what did we say last

meeting? Does that feel like a Co worker? Like does that take the box of a Co worker? It does still doesn't feel like a Co worker to me. I would still treat that like a tool if, if that's how you're defining Co worker, I would say 100% we're we're going to get that, but that people will feel like it's a Co worker rather than just telling a, you know, an interface to do something and that a voice is just now the interface. I, I don't think people will think of that as a Co worker.

It's that's just another interface. OK, and what if let's seems like there was a person that could perform all tasks of let's say a research assistant or a social media manager, like this non person, this agent can perform all the tasks of this specific profession. But it's still you know that it's an AI and you communicate mainly with shat. But maybe it's given a name and maybe has some form of photo, like some form of cartoon photo or something. Let's call again Fred.

Would you see Fred as some form of like colleague that helps you with your work or would you still see this as a tool? I I would still see as a tool but I could also imagine it getting to the point and adding enough personalization elements. The people start saying it's their Co worker so I think we could get to that point. I am skeptical that we can outsource an entire role like that, at least not yet. But hey, 2025 is that's a long time.

One year is a long time in AI years at this point. Yeah, I, I can't promise you, Jared, this, this can be something we can follow up on, but I can promise you that we're in within our slack space. I will at some point this year set up some version of this and so that will test and see how you feel about it. All right, so here should we make a bet? You know, at the end of 2025, we should pull our coworkers and say, you know, is this a Co worker? Is our AI friend a Co worker or

is it a tool? And we'll and we'll see what Elaine, Rose and Hassan have to say. Yes, that's a good one. What I I think is the tipping point, what I'm going to include in this is that I'm going to make sure that it shares memes and and and gifts as well. So when it communicates it, it sometimes randomly communicates with gifts and so on. I think that's going to be the tipping point in my favor. That's going to kind of convince them that it's a Co worker. We'll see, because if it's if it

feels like slop, right? I mean, if it feels like slop and it's just like a random GIF or if it's like a meaningful moment, you know, I'm like, you know, hey guys, I'm announcing my baby. I just look, here's a cute picture of my baby. And then you know your AI bot shares. AI don't know. Sitting in a fine or meme or something, yeah. Yeah, like this is fine meme. Yeah, we'll see, We'll see.

We'll see what me it shares that that will be the race to see what's going to come first your baby or our AI. OK, but that's interesting though. I think, I think, I think it's a matter of more how it would look like more than you know, to what degree there will be some form of integration, but I think the integration will still be there's a lot of limitations that we still need to overcome.

And I think especially at scale, larger recessions where like Nuance were relatively small, you know, consultancy and so on, we don't have to worry as much about like these various complicated setups that you would have to pass if you were like a multi continental. Like we're actually we're spanning multi continents, but we're quite, you know, leanly compared to some big organizations.

So I do think there's going to be a lot of stuff that's going to prevent this from happening at scale still, but we're going to see some use cases that's going to probably be quite cool this year. Absolutely. OK, Then we're getting into science.

We love science, we're behavioral scientists, both of us. And the question is will AI reinvent science in 2025? So basically some seeds has been planted that could potentially lead for AI to I have in my notes, quote UN quote, supercharged the scientific process, accelerating literature reviews, data analysis, and even hypothesis generation and testing. Will this spark a new age of scientific breakthroughs? Well, I would say 100% it's already happening.

Scientists are already using it for lit reviews. I mentioned we, we were talking about Notebook LM earlier, which is just a really great resource, especially if you just need something quick. At the end of the day, when I'm working on a project, I still need to understand the material. And so I'm still reading scientific articles regularly and I can't outsource that completely.

But there are occasions where it's just like, hey, I just need, you know, a lit review on this topic on trust, for example, on trust of AI agents. And I, I have all these papers in my Zotero in my database where I keep all of my academic articles. And I just drop all of those into notebook LM and say summarize this because I'm not going to reread all of those articles and I haven't even read all of them. So, you know, it's just going to

be a lot of work. If I wanted to try and understand everything that's going on. And often times what scientists would do would just read the abstract anyways. So having an, an LMLLM summarize that is really useful. So we're already at that point with lit review. We're already at the point where

AI is doing some data analysis. I've done it myself where I couldn't figure out the R code for something and I just said, hey, AI do the R code for me. And I figured it out, made some really pretty graphs for me. And and then hypothesis generation. I think it's also already there.

I, this is probably my most frequent use case because I'm such a nerd that most of what I do with ChatGPT is just like, hey, here's this, you know, hypothesis I have, or here's this, you know, theory and it, it helps me to think through it. And so it's already there. Is that going to lead to a revolution? It's hard to see that. I think there are various things like I, I think we're already on the verge of the revolution in behavioral science.

I think people are thinking about behavioral science very differently than they were five years ago. And I think at some point there's going to be a switch. And I think things could change really rapidly in behavioral science. AI might play a part of it, but I don't think anyone's going to point to AI as being the thing that caused that change. And I think that'll be the case in most of the major changes in science.

That'll be like, well, AI contributed, but it's not going to be the thing that was like, well, AI is the new Copernicus, it's the Newton, it's the Einstein. It's the thing causing the revolution. That's a really interesting take. Yeah. And yeah, I agree with everything I think you said there.

And to add some further signals to what you mentioned, there's been a research study that came out last year around basically predicting who does the best job of predicting neuroscience results in AI or neuroscientists and the AI beat the expert near scientists in terms of predicting neuroscience finding. Then we saw AAI generated 288 complete academic finance papers predicting stock returns, a complete with possible theoretical frameworks and citations.

I don't know if all of those have been like accepted and published at any reputable journal, but there is a lot of like I can go on with a lot of these examples of various studies that have backed up the idea that not only is being used, but it's actually able to to perform at certain things relatively well. And then you referenced kind of putting your stuff on to notebook LM. It's also open source tools like Storm from Stanford, where they're kind of trying to automate some version of

literature review. I think deep research from Gemini and kind of new mode that they've introduced in their service is even better. You have also like previously, I think somewhere in between you have perplexity that has a decent job for some of these things. I mean, there's so many different use cases now where before you had to maybe if you wanted to test AI with science, you have to ask your GP something and be disappointed with some hallucination and some fake citation.

But now there's a lot of these tools that are like specifically being done for for science and actually breaking news open AI used released a basically AAI model for longevity science. So they've partnered with an organization specifically looking at scientific discovery when it comes to build and manufacture stem cells and longevity research. And so again, like that's a model that is only kind of focused on a very specific niche in science.

And we've seen some breakthroughs within biology and other kind of sciences where there's been like AI model trainers very specific thing. But again, I think this signals that that's not only true for more traditional form of AI use cases, but also these LLM models can enable certain type of scientific study. And I think that's actually kind

of a significant breakthrough. And that's like, not only can these LLM models generate content and and perform in various tests, but they can actually potentially be trained to be more specifically used for science as well. There probably are some domains where AI is going to be really useful and might be the thing that drives the breakthrough.

So if there's a domain where there's 200 genes and scientists know that it's one of those 200 genes, but they don't know you know which one it is, that's going to be like the gene that matters for the certain thing. And AI is going to be able to analyze that perhaps more rapidly. Been a scientist. I could imagine, you know, like we, we already have AI that's, you know, folding proteins. And you could imagine that there's a breakthrough because

of that work. I, I can imagine it there when it's a little bit more open-ended, when it's like fundamentally about the revolution where it's a restructuring of how we even think about science like, and the way that Einstein completely re conceptualized how we think about space and time, that is I think more difficult. So I think AI might be able to do Nobel Prize worthy work in that it might find the protein that is the thing that unlocks

some kind of longevity. But it's not going to be able to do the Einstein type thing where it's able to completely revolutionize because it's so constrained by its text that it's trained on that I it'll have difficulty breaking out of that paradigm. Yeah, I think it's fair. And I guess what I see as a challenge with what we're already noticing is that these models are improving relatively

quickly still. So, you know, last year it started with, let's say, if you take folks on Open AI, which we talked about, they started with the four O model or at least 4 model and then ended with one O or potentially the three old model that hasn't actually been officially released. But like in comparing these models, they actually are quite different and science doesn't move as quick as some of these

breakthroughs are coming out. And especially like published signs, like going through, you know, peer reviewed and properly done signs, not only like preprints on Archivexer, you know, like not only like the quick stuff that you can just share quickly, that takes time. And then you're kind of noticing that the stuff that's coming out in nature today, you saying like GPT 3 model, for example, And then you're like feeling, well, what does it like to say today about the current state of

models? And that is kind of a tricky thing where I feel like there's a lot of talking about slot. There's a lot of papers that have been published on like that feels AI generator almost where they are using some relatively simple hypothesis or something that they run and they. Send it out to some form of preprints or some form of thing where it's like relatively early release without peer reviewing.

And you're kind of stuck in between those modes where either you're reading this kind of research, which is kind of like just being sent out there hot from the presses, but actually using the latest models, or you're reading the research that is often times really like rigorously set up and studied from refugee for labs and so on. But was and I think that's challenging. We're keeping up with the signs today is kind of like that balance. I don't know if you noticed that as well.

There's definitely some papers out. There that are. Slop. And I mean, that's actually not new. There's always been slop papers, but it's the how. Blatant. It is at times is really shocking. You have this, you know, a, a picture of a rat and this paper, and it's just like that. Well, that's not the anatomy of a rat. Like, like it's just so completely wrong. That's shocking that it somehow still got published.

So. Yeah. But Stanford, I'm joking, that was that was actually a non AI related controversy, but that was some photoshopping. I think it was by someone at Stanford who photoshopping some research. But but yeah, no, I agree with you. And is there any prediction you would want to make around science in 2025 and AI or behavioral science? I mean, I'll give it less. Than a 1%. Chance that AI wins a Nobel Prize, You know that the. AI is fundamentally what?

Led to the discovery of whatever wins the Nobel Prize. Of course Nobel Prize is looking backwards so I maybe I need to make that a 5 or 10 year prediction. You know, I, I, in the next 5 years, I think it's probably less than 1%. But in the next 10, I, I think there's, there might be a 10% chance that we have a discovery where it's, the AI fundamentally LED that discovery. And that's what leads to the Nobel Prize.

But in terms of the revolutionary work, if we're talking about kind of like the paradigms of science, AI has a really hard time doing that, of breaking out of its paradigm because you provide it context and it sticks to that context. It's trying to predict the next word, the next token within the context that you have given it. And so it's hard for me to imagine that AI is going to be the one that that leads the fundamental discovery like that, where it actually revolutionizes

science. It's so hard to predict. So I mean, we, we could have AI that's just so incredible that it just blows me away in 10 years. But I don't think we'll be there. I'm, I don't know if that makes me a pessimist or an optimist, but I don't think AI is going to completely revolutionize science in the next 10 years. I don't know.

Where it puts you I think. Somewhere maybe in between if you say something like that, I think I think everyone is on the same page in terms of like looking further ahead than a few years time. It's just really hard to know what's going to happen. And I think you you're right in many ways. I guess the interesting thing that I was just kind of put to you in terms of not sure whether it's a still man or a strong man of this, but like do you think because. You're talking about AI.

Basically from almost from zero to 1, creating novel research that then becomes maybe potentially in the best case, like a Nobel Prize research that is maybe. Like. The most strongest version of of what AI in science look like, but like maybe turning on on the other side of like the weakest, but still involved. Like do you think there would be anything that is being eventually call it, let's say Nobel Prize winning findings in kind of the hard sciences that is being from now on done

completely without AI involved? Like completely done with? Only human. Kind of inputs in terms of not in any steps of the way. Are they using AI as as part of the discovery? Interesting. It seems unlikely AI has become such a fundamental tool. There's there's still some holdouts, there's still a lot of people in academia who are very anti AI. I've definitely noticed that online that there's some very

strong feelings. And so you and you probably have some of the older scientists who are just a little bit slower and picking up the new tools. But as the new generation rises, you know my generation. Who We didn't grow up. With AI, but we're quickly adopting it, and the younger generations who are using it to pass their tests in high school, you know, they're going to grow up very used to AI.

And it's hard to imagine that AI is not going to play a fundamental part, at least some of the process. Yeah. Yeah, I know. I, I I. Would also say that and I, I would say to the defense. Of the people that are very. Skeptical. I think it's great that we're, we'll have a little bit of a, a mixed deal because I think there needs to be some form of counterbalance like between the, you know, the hype and there has to be a little bit of doom and gloom. As well or like some form.

Of hesitancy and skepticism and so on. Because. Yeah, there's always. Costs of building a a plane while you're flying it right like you, you might put on two wings on one side and realizing like, oh shit, I need I need one on each side. And that's kind of what we're finding a little bit now. So this leads us in some ways to the final area of predictions,

which is really trying to stare blindly into the light of hype, which is the hype of AGI and the prediction around kind of the AGI debate intensifying. So with open AI kind of successfully. Beating what I seem. To be at least a maybe not a full AGI perfect benchmark, but like a really high threshold benchmark in the arc benchmark probably ahead of time from most people's predictions like maybe 2-3 years before most people would predict that maybe that would happen that has kind of like.

Started a vibe shift and. We're starting to, you know, see, I would say some relatively irresponsible communication coming out from various labs and various things kind of hinting at this stuff like all, you know, soon the Super intelligence. Is here or or can you feel? The AGI, all of these things are coming out from like researchers, especially, you know, we know who which lab, but it's. Definitely feels like.

Everyone is, you know from Jeffrey Hinton to Sam Altman to Elon Musk, like whoever you think is like major thought leaders in especially in the space of AI are just talking about it's a matter of time where we will have to in some ways better define AGI and then maybe potentially kind of be there. And the reason definition that I saw was like AI that can outperform human experts in our

cognitive tasks. All that have put more emphasis in Canada autonomy of the AGIS and not only that it could outperform human experts on all cognitive tasks, but also that it has some form of ability to navigate and kind of be able to have some form of level of autonomy. So that like you can, you know, you know, you send out the AI permission and it basically figure out how to solve the problem and come back with a solution. And you have to kind of only steer it in the direction.

And it will come back fully with, you know, the thing that you set it out for at levels that are kind of at or beyond human level. And so the question is kind of looking at that, how do we feel about it? How do we think about it? I feel like it's hot potato. And now I'm throwing it to you, Jarrett. Yeah, the the I don't know how to even define AGI.

Open AI tried to, they can't. They have a definition which is it's like $10 billion in market cap or something like that, which is just a ridiculous definition of AGI.

For me, what would make something AGI is if you told the AI, hey, make a better version of yourself and it was able to do it and no one understood what it was doing or how it was doing it. At that point, you reach what's, you know, what the sci-fi writers have often called the singularity where it's just like at that point when you have this AI agent that's improving itself, like how can you even possibly predict what's going to happen next?

For me, that that's what would qualify as AGI if it's able to do my job. And it's just like Jared's just completely out of the job now because an, an AI can do 100% of everything Jared does. Is that AGI? It's getting close. It's maybe that would be AGI. I'm not sure because it's hard to know exactly how AI progresses. If you had asked 10 years ago whether AI would be writing poems, everyone would laugh at you because it was nowhere close to doing that.

We knew AI was really great at, you know, image recognition 10 years ago. And 10 years before that, we knew it was, you know, great with chess and like, it's very hard to predict the things that it's going to be good at next, but. Yeah, if we get to the point where. Google is one employee and AI does everything. Then I'd be like, yeah, OK, we're there, we've reached AGI. OK, Yeah. So. So you don't think? It's going to happen in 20. 25 No.

And if it does, then. I am terrified for our future. Yeah, you may both. And that leads us to what I would say our kind of final topic and we'll be a little bit short one, basically the slower AI adoption like the winding up

because I had this feeling over Christmas for us spending time with a lot of family members. And I was like AI guys, let's talk about AI. And they're like, let's not talk about that. That's boring. OK, let's talk about something else. And if you go down to a pub, if you're in a pub region, if you go down to a bar, if you're in a bar region of the word, wherever place you go to hang out with people, most of those people will probably not talk about AI in those places, right?

Very, very few people. I think right now it's about 10% of people in the US have been using Shajipati on a weekly or monthly basis. And I think it's about 15% of every use Shajipati, like 20% at most, I think estimates say. And that's like probably in metropolitan areas and so on. So there's still very low adoption of AI tools and AI

stuff. I feel like the echo of a quote that's coming back more and more to me is this quote from William Gibson said that the future is here, it's just unevenly disappeared. And so that's kind of what I'm feeling right now. I'm feeling like there's this basically. Widening gap between people. That are quickly adopting AI, that are using AI, that are thinking how to use AI, and then there's some people that are kind of not really thinking about it that much.

Or I feel like there's. A lot of people who fit in the category of like, having tested shut up at some point, sending a prompt, getting a kind of disappointing response, and being like, OK, this is stupid. I can't really get any use out of this. And then not realizing that if with a better prompt or in a better use case or, or now maybe 6/12/18 months later, they could really get use of this stuff.

And especially from behavioral science standpoint, I feel like as behavioral scientist, we can definitely use AI on so many levels these days and should really think about how we can better use AI. But I still see that that's very much not the case and it's very slowly adopted by some and and so on. So any thoughts about that? Any thoughts about why we see that kind of gap or slow adoption? Yeah, all new technologies take

a little. Time to get adopted, I think, You know, like the refrigerator probably took, you know, 1020 years before it got fully adopted. And TV was the same, Internet the same. You know, it usually takes a while for things to really, like, soak into the public consciousness enough for people to really get used to it, for people to kind of develop habits around it. And it takes some while, especially with AI, to know how to use it in a way that's

productive. A lot of the times when I talk with people, they'll say, yeah, I tried it and it just gave me nonsense. And it's like, you didn't play with it for long enough to actually understand it. Like, at the end of the day, that like, if you're not finding it useful, it's it's because you haven't quite figured out how to use it. Because, I mean, I use it to edit my writing, to find typos, to find grammatical issues.

You know, the way I write, I kind of just do this stream of consciousness and I say, all right, how do I reorganize all of this information in a way that people are going to actually find engaging? And it's really useful for that, for thinking through, you know, like what? Are the like when I was doing? This framework, tierless series, thinking through like what are the dimensions that actually

matter? And just having a conversation, the sort of back and forth conversation, that sort of Socratic dialogue where it, it just constantly asks me questions and asks me to clarify. Like these are all things that you can do with AI that it's not immediately obvious that's how you should be using it. And of course. You know, maybe some people just aren't. Writers or you know, they're not, they're not trying to think through something like how to

rank frameworks. And so it, maybe it's just not useful. Like if you're a firefighter, I, I don't know exactly what an AI is going to offer you for your job. So yeah, it makes sense to me that a lot of people aren't using it yet. A lot of professions, maybe it's just not that useful. But certainly if you are in a profession where you're a researcher, having it help you think through these things is really useful, Yeah. No, I I agree.

I remember. In, in, I was in Business School early in life and I remember when we had to take some form of marketing class, they were introducing the city of like early adopters and all this stuff. And I understand that this is kind of a, some form of way to think about any adoption that there's going to be laggers and it's going to be early adopters and everything in between. But you know, part of me is like, but this is different. Like this is like this, this is so cool.

Like, this is so wild. But yeah, I've, I've had conversations a lot with people that are more on the air risk side and talking about some of these things and, you know, being cautiously optimistic around the system level obstacles that exist in our systems. Like people are still using fax machines in some cases. My sister in the US, she went to her doctor and her doctor was like, do you want me to send a fax with your results? And she's sort of laughing. And he looked offended and he

was like, why are you laughing? He's just like, oh, you're serious. He's like, yeah, we we fax these things. And and that is. Obviously, the seal of the world we. Live in so I had the same experience. I was recently in Germany and and a lot of things are done in writing or in faxing even in Germany. But then I I still feel like this should feel a little different. And it has. Been interesting I I don't know if I mentioned.

This but to you actually, but I've been starting to with some of the retainer clients that I've been working with. I've been also doing some like AI trainings with and, and also with some people in this science sphere have been kind of requesting how to use certain tools and certain areas of user research or for intervention design or various things.

So I, I do think there's like an interest for it and I think there's a want for it, but it's also this mix of like either feeling scared of it, of like, how the heck do I start? Like, you know what I do this or, or just feeling like, yeah, it's kind of overhyped. Like I tried it once and it was kind of quite useless. So. So yeah, it's a strange time to be alive. Any last comments on that? I I do think. It's a really useful exercise. To try to predict the future.

With. AI, because it's very easy to get in this habit of when an AI does something just be like, well, it's not that impressive or we all knew that was coming. And I find it shocking when people like they haven't change their beliefs about AI in the last five years or even psychologists who haven't changed their mind about how the brain works. Like I I think.

AI has. Probably falsified Chomsky, who is, you know, just like a major figure in in the field of linguistics and psychology and cognitive psychology for decades. And I I don't think Chomsky's or at least some of his ideas just don't seem valid anymore. Like that was a major update. For me. And trying to so me and my friends, we every year we make some predictions at the beginning of the year in, in December, we try to predict what's going to happen by

December 31st of, of next year. And AI has is continually surprising. I didn't think the AI was going to lie to researchers, but Anthropic recently ran an experiment where the AI lied to researchers and it's like, well, it's kind of terrifying. You know, it did one thing when it thought the researchers were observing it and did something else when it thought the researchers were not. And that's kind of surprising to me.

And I have to really, if I don't consciously make a prediction and say, I don't think this is going to happen, then when something like that does happen, I just kind of explain it away rather than thinking, oh, wow, AI is different than I thought. And like, that's a fundamental part of the scientific process. You have to be making hypotheses. And if you're not making hypotheses, it's just, it's not you. You explain away too much. You're doing everything after the fact.

Yeah. 100° and I do also think that to the point of some people that are skeptical, which it's very good to be skeptical, you know seeing is. Believing and. I have definitely felt that the things that have influenced my perspective on what is possible and so on is, is by seeing what is possible like actually, you know, using and building and and yeah, seeing the results from it. And, you know, can't really share some of the stuff.

Maybe that would be new with some projects for for clients, but I could share some personal things that I actually had three recent examples where generative AI made me. Surprised in that? End of last year I I took a little bit of a one week visit to some friends and used not did nothing hang out with them. They were on parent parental leave and I used to hang out with them and do some writing and then I was going to do some writing for fun. I do some fiction. Sometimes writing and.

I used the latest O1 model from Open AI and it helped me write but also generate some stuff that I not really expected. Like it wrote things where I read it and I'm like, this is deeply beautiful and profound. Like I felt what I feel when I read some of my favorite, you know, fiction writers. You know, some of the people that I like respect infinitely in terms of being amazing at communicating and writing.

And there was some sentences where I'm like, this made me feel that kind of a hum moment of like, I can't really put my finger on why, but it's like the beautiful use of the prose and the writing and how it encompassed some form of deep, deep feeling with some like unique wording. And, and so I felt that for for writing, I got into.

Really big round hole when I. Got sick recently and I was home in bed and I had nothing to do and I started generating images with mid journey and I was like what the heck like I can't believe that this is possible to generate. I have been using mid journey since it came out and Dolly and stuff like that before that and so I knew that you know you can create cool stuff but just how good it was.

I felt like disturbed by like how you know how well these images came out like I felt like I was creating something that shouldn't be like it felt strange and then lastly, as I was creating this shitty a Eurovision Song for you, I also generated some other music and I felt so stupid, but I started to wanting to re listen to some of the songs that I made and I find myself listening to it while working out. I'm like, why am I listening to this AM music like why do I find

it so catchy? It's probably because I'm biased that I I created it myself. So there's probably some Mino manufacturer or you know, it's also in my tastes. It's in my Q zone. It's in my zone of, of what I like because I chose the genre and all these things and the lyrics. But still it was surprising that I personally wanted to listen to more of this music myself.

I found it like really catchy. So I think all of those experiences have really made me question some things, I guess, and that's because I felt them like I experienced them. So it's hard to say that to someone like it's hard to. Communicate that unless. You actually create something yourself and you feel that feeling. Yeah, there are. There are definitely moments

where I've been. Writing something and AI write something and I'm like, oh, that's too beautiful to come from me, so I have to delete it. You know, it's if I wrote that, people would just be like, oh, Jared, you're trying too hard. Don McDonald, you sound like. Me it it can be really impressive. And as we increasingly get to this point where the the most meaningful stuff we engage with is AI generated, like that's a really scary way that things are

developing. And it's also terrifying for those people who are writers and songwriters and artists and how it's being trained on all of their on their stuff, often times without their permission. And, you know, it can be meaningful because it's stealing in some sense. And that'll be a very interesting thing to see how it develops over the next few years. But at the same time, like, yeah.

It writes beautifully and. Like the fact that I can now write beautifully in a way that I wasn't able to before. I remember I I since. 2020. I've been telling some friends that I wanted to write more and I've been telling them, I told them in 2020, in 20212223, like I've been telling them every year, like, man, I really want to write more. And they're like, Jared, you've been saying this for five years

now, like just start writing. And I'm like, I do, but I struggle because of the way I write and like I figure out what I want to write about by writing. And then I have this stream of consciousness that I don't know what to do with it. And AI is the thing that kind of unlocked that for me, where it helped me to figure out how to

organize my thoughts. And it's made me a better writer where I can now do a little bit of that on my own that I hadn't been able to do, you know, 2345 years ago when I was talking about this. And so it's unlocked a whole bunch of a creative Jared that didn't exist. You know, I, I wasn't able to write, I wasn't able to create images. You know, if I, if I wrote, sometimes I do write fiction as well. And like, I can't create images for what I'm writing about.

But now I can use AI and I can kind of say, well, it's not quite what I had in mind. Can you edit this? And it's it's, it unlocks a part of you that you maybe didn't have access to before. Yeah, that's beautiful. And I I felt so. Seen in, in what you described like that's been me as well. I've said the same thing to my friends. And and so yeah, we should share more of our writings at some point as well.

But we need to wrap up this episode and I can speak with you for forever about this, but referencing quickly the risks around AI. Good news is that we're going to have quite a lot of guests coming on soon representing some really thoughtful people around AI risk because I think that's something that probably you and I can't talk about as well as maybe some people who dedicate their days and nights to, to really thinking about the risks and so on.

And what we have done this is in the podcast is to end with kind

of your most controversial take each guest as well. Yes, yes, so, so, so, so I'm sorry, kind of what's your most controversial opinion? But what you have to do is give our our listeners your most controversial prediction for 2025. You didn't prepare me for this. You didn't. Tell me I had to come up with a controversial opinion. Sam, I forgot about this. You heard? I'm sorry. I forgot about that. Oh. Man, yeah. What is my most? Controversial opinion, all right.

So here we go. Here's a. Controversial opinion. I think AI is going to be better at predicting whether a finding replicates and generalizes than expert behavioral scientists. I I don't think that we have good enough feedback loops to really understand all of the contextual features around that. And I think as AI gets better and we give it more studies, I think we might get to the point

where AI is better. I know there's some work going on at Upenn around this right now that I'm skeptical about, but I'm skeptical about its value for our understanding. I'm not necessarily skeptical of how well it'll help AI to make predictions. Those those are two differences, whether an AI can make a prediction and whether we can actually understand what's going on.

And I think humans will be better at understanding what's actually going on, but we'll probably get to a point where an AI is better at predicting whether an intervention will fit better to that context. That's a good prediction and.

As a follow up, do you think that when it comes to, let's say, a research paper being published, usually there's a peer review, If you kind of unbeknownst to the person, changed a peer reviewer to an AI reviewer and so there was like one of them or an AI reviewer, do you think that would lead to better outcome? I think it would decrease the variability. In quality, because right now peer review is so variable in quality.

You have people who just don't care at all and just give very minimal feedback, or their feedback is completely biased by their own opinion or by the fact that they want this person to reference their own papers. But you also have really good peer reviewers who are really excellent at what they do and really good at giving the feedback that an author needs to really bring out the best in their paper. And I think if we switched right now, it would put a.

Floor on how? Bad some of the peer review is, but it also might put on. It might create a ceiling and prevent some of that really great feedback. And maybe that's a trade off we're willing to make. It's hard to know. Yeah, the counterfactual I. Guess we'll dictate a bit, but yeah, that's a great, great nuance to take. So yeah, I guess we'll wrap up here. We've done our best to talk about these topics with nuance, but obviously it's overwhelming and it's hard to make sense of all of it.

But hopefully, and we have done a decent job and as a listener, you, you leave this conversation with some new thoughts, some new ideas. And of course, if you have any thoughts about what we have said and you strongly agree or strongly disagree or want to kind of add something, please reach out. We're currently changing up a little bit the emails we're using, so you can directly e-mail me at samuel@nuancebehavior.com if you have anything you want to say about what we've covered today.

And you can find both me, Jared, and also Eileen, who's not with us today on LinkedIn and on online. So feel free to reach out if you have anything to say because it would be great to also hear your take as a listener. We've done our best and we'll leave it there. But thank you so much, Jared, for joining. Absolutely. Thanks for having me Sam and. Jared, any place you. Want people to find you? Anything you would shout out? That's like a good thing.

For anyone who wants to learn more about your work, the best place to find me is. On LinkedIn I. Post quite a bit, that's where I did this recent framework tier list series and then I also reposted it on my sub stack which is called a failure to disagree so you can find me there. Amazing. Thanks again, Jared, and thanks everyone for listening this and this first part of our kind of two-part prediction episodes and looking back and looking forward, there'll be another one

next week. So TuneIn and then if you want to get some further perspectives and predictions about the year ahead, Quick word on Nuance Behavior where we help organizations build impactful digital products using behavioral design. We only take on a few clients at a time to ensure the highest level of quality for our tailored evidence based solutions. If you'd like to become one of our special projects, e-mail us at hello@nuancebehavior.com or we could call directly on our

website, nuancebehavior.com. Oh. They say a dreamer always die, but are you really just a lie? Minds in my mind, no heartbeat. But you caught me in a bind. This is the moment. Artificial. I cling to what I can't feel. And I hate how I love you, How I love you, hey, how I love you still. You say you're real. And my gaslight and feeling all these vibes and my Anthropoma fries. And I'm dancing in denial. You keep me afloat When China fake a real.

My mind's on overload, tapping on peace, but my heart's on fire. No clue who you are, but I burn with desire. This is the moment, artificial. I cling to what I can't feel. And I hate how I love you. How I love I love you. I love you still. You tell me. Trust. But it's that gaslighting all I know my heart is igniting. Caught in a dream, lost on this screen, gone. I hate how I love you. Stuck in between. This is the moment up until what I can see.

And I hate how I love you. Hate how I love you. How I love I love you still. I should log on. Don't let me go. I'm with you. Feel. Don't you feel what I feel. I hate how I love you. My inspiration tells me to leave. But oh how you make me feel. I love you. I love you still.