Most AI rollouts looks something like this, a big announcement, lots of enthusiasm, and a few months later things have gone one of two ways, either barely anyone's using the tools, or everyone's using them to churn out AI slop. Sometimes both and leadless to say, the huge investment in licenses isn't paying off. Now, for today's episode, we are releasing a recording from a live webinar Neo and I ran recently on why AI rollouts fail and how to set
yours up for success. We get into why these problems are happening, including the surprising research on how little time AI is actually saving most people, the training goes that does move the dial, and the question every leadership team needs to answer before spending another dollar on licenses. Well, let's jump in by starting with the biggest problem we are seeing organizations making Welcome to how IAI with me Doctor Amantha Imba and Neo Applin, head of Inventium AI.
Each episode we share one practical way to use AI better at.
Work and in life.
No fluff, no tech jargon, just things you can use straight away. We are of course talking to you today about AI rollouts and why we've seen a lot of them fail and how we can actually make them work. So to start with, you might be like, oh my god, who are these randoms? Or maybe like you followed us for years and you're like you need no introduction. But anyway, I'm going to assume that, like we're two random people,
So I'll introduce myself first. So my name is Amantha Imber and with my initials, well, it just qualifies me to be.
Talking about AI, right, So what more do I need? Anyway?
Aside from that, I'm the founder of Inventium and Inventium dot AI, which is our sister company where all we do is help people with AI skills, building and rollouts and all that sort of great stuff. I was awarded AI Consultant of the Year last year, so there we go. I'm award winning, so hey, you should probably trust me.
Neo, who are you?
I'm neo and matrix jokes going in here. So I've been in it my whole career. I've been building stuff for big companies, small companies, startup, scale ups and that kind of stuff. But AI has been the biggest change I've seen over the time. So AI is all I do these days. I'm the head of AI and invent him.
AI and I trained companies on how to get the best out of this, and I usually start at leadership teams and boards and things like that and then go throughout the rest of the company to be able to implement AI awesome.
So together Neo and I and the team at.
Inventium, we have worked with a lot of different organizations.
These are just a smattering.
Who we have helped build AI capability, help them with roll outs, all that sort of great stuff. So today, like we're talking from firsthand on the ground experience plus all the research that AI has of course helped us get across.
So let's start. Let's start with a really big problem.
Great, So first problem is actually top down. A lot of leadership teams are saying this AI thing, Yeah, we'll just get someone else to take care of that. So it is a problem that needs to be solved at the leadership level, and it needs to be a problem that needs to be solved as far as here's what we need to have for our company, but also needs
to be embodied by the leads as well. So one of the challenges we've seen is the company might have an AI policy, say, but everyone meets the policy and goes, oh, no, I think you can use it. No, I can't use it for that, And people get confused on what you can actually use it for because the policy is not specific about the types of data or the types of use cases you can use it for. So one department feels great to be using it, another apartment doesn't use
it much at all. That's one of the challenges there. So the AI strategy is the first challenge, but within that strategy challenge, the CEO needs to know what these AI tools are, where the warts are, where they wanted to go, where they think it's going to actually change your company. And we found a lot of leadership teams will say, oh, that AI thing, will get that done by it department, but the CEO themselves don't use it.
They don't get their leadership team to use it, so everyone else in the company is a bit scared to use it as well. I even had one lead who said every time anyone walked behind them, if they had AI opened on their laptop, they'd all tab out of it, which is an absolute wrong way to do it. You should be saying this is how I've used AI, and you should be able to use it in these kind of ways too, So we're going to walk the talk
here with AI. Otherwise a whole bunch of your staff are going to be too scared to use the damn thing.
Now, let's move on to another problem near I know you are seeing this a lot as am I.
Yep, Absolutely, buy the licenses, go right ahead. But you've got to be able to train your people on how to use these things. Where is it twenty thirty bucks a month per user? A few times that buy your workforce. This is not a cheap investment, but you can get a lot of this back. We think we get a hell of a lot more than that back return on investment for us and the people we train. So thirty bucks is actually pretty cheap for what you can get
out of AI and the people's time savings. But just giving them AI and saying now go is actually going to give you less productivity. What I mean by that is you're going to get more people getting confused with it, just writing big emails, creating AI slop. So you need to be able to train people on how to use AI. Licenses alone is not going to solve the problem.
Now.
Of course, the big promise with AI is that it's meant to save us all time.
But when we look at the latest research.
Around is coming out of HBr, they found that the average time saving is just like guess in your head before I reveal this number, two point five percent, Like that is nothing, And there are many reasons why this time saving is not happening. But I mean, what we are saying, particularly when you give people the licenses without training, is that you get this huge proliferation of volume of work,
and generally that work is slop. Like you will have people, you know, typing a prompt into AI, going write me a ten page report and here are my three bullet points, and then that report then gets distributed to people people presumably have to read that report, or even worse, they're putting that report into AI and going summarize this report and they're getting back three completely different bullet points. And then companies are like, ohy am, I not saving time.
Well yeah, next on our problem list. What we see is a big problem in how companies are measuring the ROI of their investment, which is obviously like you know, really it's a decent sized investment. When you think about licenses and training and doing this stuff properly.
And what we see a lot of companies doing, and.
Particularly like I know, like on sales calls that I'll be on, I'll be like, well, how's you know, how's AIU is going?
Now?
Like where are people at with adoption? And they'll go, oh, well, you know, we can see the x percentage of stuff for logging in daily to their AI tool, and some companies is a little bit more advanced, like they're measuring token usage.
But the thing with this, this tells us nothing whether people are logging on or not, Like they could be logging on and creating slop and then logging off, Like that's not helping your company. So these vanity metrics are not helping you. We need to think differently about the metrics that you're using to go how much value is AI actually giving us?
Okay, next to agents, Yeah.
That's another vantage metric we've seen as well. Some companies are going great, we're getting more agents created, and we want more agents created. So, just for those who don't know, agents are effectively AI experts that do things for you the same way every single time. So you might have an agent that might create for you like a little the way we create our briefing here, or it might be an agent that does like a style guide checker, or it might be an agent that helps you with
your email writing. Those kind of things. Everyone's building agents, and that's great, really good to see. In fact, we train people on how to build agents. But here's the thing. If everyone in your company is building agents, you're going to get one thing, which is a proliferation of agents. And the other thing is my agent will work differently to your agent. And so what you're going to find is that there'll be too many agents out there. They'll
all work a little bit different to each other. And a lot of people are building agents. Don't know how to build these things, so they work the great way every single time. And so it might work great for me when I use it, when I give it to you, you might use work different because you use different words when you created your agent. So those kind of problems. The other problem is, because we've got so many agents being created, no one's actually looking after the agent. Who
owns that agent? If it needs to be updated or changed? Who owns that? How do I find that within my organization? And so what we've actually got is a whole bunch of agents out there floating around that may or may not be giving value to the company. The other thing is we're going to have a whole bunch of legacy problems. Like if I'm got this old agent, if anyone hasn't used it in two months, can I delete it? If
I do delete it, does that break someone's workflow? I don't know, because all these agents are just been thrown around. It's one of the big problems we've got. So agents are great, but too many agents not done particularly well. Isn't innovation. It actually is more confusing than anything else.
And interestingly, like on the topic of vanity metrics, I certainly hear this a lot, particularly when I'm speaking with leaders who are looking at AI rollouts, and they'd be like, yeah, yeah, I've got like heaps of people they're bill agents. And I always wonder, hmm, what do you mean by that, because like, there's good agent building and there's crap agent building.
If you've spent maybe like five or ten minutes like putting a system prompt into like agents on Copilot or a GPT on GPT, and then you're like, Bam, agent created, it's probably not the best agent because like Neo.
How long do you take to build an agent?
Typically half a day a day. Yeah, and that's because I'm building it, I'm refining it, I'm testing it, I'm trying to break it. I'm giving it different inputs, different outputs need to be put in. I need to make sure that this works reliably for me and everyone else
I give it to every single time. And the other thing is I then need to manage and version control my agents because I'm going to come back to these ones when the engine changes, when it goes from chetpt five point three to five point four sometimes day break. Same thing with Copilot and all the others. So yeah, agents do take time to build correctly.
I'm curious, folks at your workplace, who's working in an organization where like there's a bit of a vibe that like, oh, is AI going.
To replace us?
Like just give us like a thumbs up if that is your organization.
Yeah, I can see LA lots of them's going up now.
Okay, So I question, like, why would anyone adopt a tool that they think is there to replace them? This is a culture problem that very much needs to be managed from the top, because you know, if I'm working in an organization, I'm like, Oh, I think AI is going to replace me? How excited am I going to be to use the technology and really embrace it?
Probably not much at all. Now. The final big.
Problem that we're going to talk about before we get into solutions and then we'll get into your questions is brain fry. So I know that this has kind of been headlines in AI news.
If you're like us.
Inclined and like just you know, follow this stuff every single day. Brain fry was something that I was really awesome pair of research and published in HBr, led by this amazing researcher Gabrielle Obrezen Kellerman, who I've actually had on how I.
Work a couple of times. But brain fry is what happens.
And I'm curious as to anyone that has experience to this. When you have got multiple let's call them, like, you know, multiple AI tabs tasks going on at the same time. Maybe you know, you asked chat GPT to do one thing and then you've opened up another tab and you've got it working on another task and then another. And this is how a lot of people are working with AI, and people are getting to the end of the day,
and it's different from burnout. They are feeling like their brain is fried, like they have intense cognitive fatigue and not surprisingly, like when we are fatigued like that, it causes us to make more errors. It causes us to obviously produce like lower quality work. It is not good for the output we are delivering in our job. So different to burn out, which is a little bit more emotional and some other aspects. But AI brain fright is a really big problem. I don't know, you know, if
people have started to experience that. I know, I when I read that research, I'm like, oh my gosh, I am feeling that.
I feel like the way I work with AI.
Sometimes I am context switching because, like god forbid, I have to wait, you know, thirty or sixty seconds for the AI to you know, be in like proper thinking votes and vasually takes some time to think about what I'm asking it to do.
But yeah, you've probably experienced brain for as well.
Yeah.
The other part of it also is that that constant checking, like is that thing that it's given me good? Is it actually correct? And you're checking your changing your job from being a creator of email or document whatever to being effectively a boss and a checker and a policy and a reviewer and all those kind of things. And you're constantly vigilant as well as constantly between tabs and tasks and things you got to do and things you got to check. So it's a real thing.
So's that's a few problems.
When Nao and I were preparing for this, were like, we listed about twenty problems that we're seeing all the time, and we just like got the top ones that you know, I'm assuming that you can probably relate to at least one of those.
So let's get into solution mode. And again, there's so much that.
We could talk about here, but we've just you know, picked out a few.
Now I am curious, particularly.
For people that are sitting on a senior leadership team, whether you can answer this question. So before you spend a dollar on AI. Finished this sentence, we are using AI.
Two.
Okay, this is this is your why why you're doing AI.
And a reason is not because well everyone's doing AI, like that's the thing that you do. It's not a reason. It's so it's a reason. I guess it's not a compelling one, no great one. So you need to be clear on your why, because everything else when it comes to your AI strategy will follow.
Right doing AI to you free up time.
To really innovate, to service your customers better to maybe let your staff work god for be like a thirty eight hour week.
So be really clear on your why. Now another solution near.
We'll have to mention this one earlier, which is we often start with and I encourage you start with the senior leadership team because that dovetails into the why, but also dovetails into the leadership and what we can do as an organization with AI, because the finance lead will know how they could implement it in finance, the operations likewise,
et cetera, et cetera, et cetera. So if those leads don't understand AI, then there's pretty much no chance that they'll be able to lead their teams to be able to use this effectively. So we often start there at
de Lasia teams. Sometimes we start a board, sometimes board and leadership team, those kind of things, because what we're finding is that the trailblazing leaders are spending up to eight hours a week just continually working on AI and working with their teams on AI to make sure that it's going to be implemented, so it's upskilling themselves, but also working with their teams to actually see how it's
being used at the coal face as well. So the leaders using it in their own routines does a couple of things. One is keeps them up to date, rather than being the Ivory Tower leader who doesn't really know about the AI but thinks it's a great idea. Instead, they're actually embodying that and they're showing their teams and everyone below them. Here's how I'm using it. You should be using it too.
And I think neo because you're like you spent your entire career in it and tech like you have seen many organizations go through digital transformations, like ALI is really different here, and I'm keen to know because like previously, a digital transformation, it's like you just delegate that and.
CEO of a medium sized company if you've got a new CRM, like a customer relationship management software, so like Salesforce or something like that, where we've got our sales and our deals and our products and all those kind of things in there. Traditionally, it was get the it to department to go and buy it the IT department would then buy it, they'd implement it. They do a little bit of training for everyone, and then the training is done a couple of handholds with a couple of people,
a couple of manuals there, and then it's sorted. So eighty percent of the work is about just putting the software in. II is really different because most of the work is actually with the training and embodying that within your workday. But also this thing changes. This AI thing is continually evolving, so it's not a one and done
training problem. It's actually an ongoing training thing. And so if the leads aren't keeping that ongoing themselves, they've got no hope of being able to lead that for the rest of the organization.
Now on the topic of training, neo, what should we do? And we're a comfort like you know, because we see a lot of bad training. We're often bought into fixed training that was not particularly effective.
But yeah, what tell it? Tell us about this?
A lot of the training, this part of that delegation thing. A lot of the leaders are delegating to one department. They're saying it this is yours to manage and roll out, or they're saying HR it's a learning and development thing, so Hi, you're going to roll this thing out. It's not just one department to do. And what we find is a lot of the training that happens and sometimes is with the bigger providers as well. They're very much based on features. So here, click this button, you can
do this, you can do it. Here's deep research, you can click that and do some things. But they're not showing people why they use it to how they can put it in their day job to be actually able to change the way they work. So being taught a whole bunch of features is great, but not knowing how I can improve my day job, improve my outputs, my clients, my customers, my internal stakeholders. That's where people get stuck.
And we find a lot of teams get already trained on the features, and then there's not the kind of traction that people are seeing who are wanting to see. They'll see people are using emails. That's really common. You don't need to be trained to use emails, and so these get lots of emails and lots of big emails, but they're not actually getting better work done. And so what we recommend is people train to tasks, not to features.
So initially there's going to be some kind of acclimation here's what the buttons are and whatnot, But then quickly it really shu you go into familiarity of the tool and how that's going to help me a as a worker, and so I really go to tasks that people do and particularly if you can then get it to be embedded with your workflows as well. So what we found is there's actually a bit of a threshold with training with different organizations, the researchers out there and saying that
there's a minimum viable training amounts. They call it the minimum dose if you'd like, and it's actually five hours of hands on training. So this is not read a manual. It's not come to a lunch and learn and watch someone do it. It's training where people are being taught their features and how to use that, but also taught how to put that within their day job and practicing that minimum dose five hours. But those companies that are actually getting benefits out of it, they're training up to
eighty hours a year. So that's not just here's what the feature is. It's like, let's talk about your workflow. Let's see how we can build AI into those workflows. Let's practice these things and building an agent and then sharing it and then tweaking it and changing it, and let's use different scenarios here, those kind of things. Yeah, they take some time to sit down with those those people.
You've got change champions, you might have external providers like us, but that's where the benefit really comes in.
And if we look at that research that Nio just mentioned that eighty hours a week training is like a great dose to aim for in the research that showed people got back fourteen hours per week terms every week in terms of time saving.
So if you're like going, I am.
I'm going to like do some a train and do like a one hour lunch and learn and like.
Hope for the best.
Just like think about that research and any research that we mentioned, just like hit us up. We'll share links with you. Okay, dooksys. So quite often.
Again, when I'm in meetings.
With various leaders is they'll be like, oh, yeah, you know, we're just we're just you know, rolling out the licenses and yeah, you know, give us a few months to do that and then you know, we'll we'll get you guys into do training and that that scares me. I mean, we've got we've got plenty of works, so that's totally fine. But what scares me is if you've got people with
licenses like free or paid. Obviously paid is better ACTU you can do more stuff, and then there's a real lag between when people have access to the tool and when they're getting trained in the tool. To me, like my organizational psychologist, habit change brain goes, oh man, that's a lot of time to set some really bad habits and also to produce a lot of AI slop that
is going to reduce productivity, not increase productivity. And interestingly, some other research that Neil and I have come across has found that people with AI I think there's something like over one hundred percent increase in the amount of
emails that they're sending. And that is because it's now super easy to write an e like there's going to be a crap email if there's been no like human judgment or involvement in that email other than write this email with these three messages like that is slowing people down. So make sure that do try to time the training very close to when people have the licenses to reduce bad habits forming and a proliferation proliferation. I think I
said that right of aislot. Okay, neo, what's this one about?
Yeah, it's on literacy first, so maybe there's a step prior to that, but really it's about first focus on here's what the talking do and he hey, can actually get it to help you to challenge your thinking and use it as a buddy or an expert rather than just use your slave, which is a lot of people like write me the email that kind of stuff. But yeah, absolutely focus on that literacy first and get that good baseline knowledge of how it can fit into mind day
and help me. Then the benefits really come into leverage, which is one of my workflows. How can it help fit into my workflow? How can it speed up that thing that I do that annoys and the crap out of me? How do I get AI to help me
with that or that big report that I do. Rather than doing one a month because it's so big, maybe I can do five a month because we're using AI so we can get better insights on our customers or the ones that I love is how do we customize some of these things so rather than every customer, every client, every department getting the same response. How do we get AI to customize these things for people? So, but first it really depends on that literacy part. So I'll put
it a little diagram here. So here's where a lot of people start, just at access. So that's where here take co pilot. There's a common one of companies get and go for it and people have got it and they think, yep, AI is now done. We've got our it guy. He's pretty good at AI, I understand, and our it guy is going to show people how to do it while I have a luncheon len. That's great. Done, that's not bad. It's not a bad start. But really it's about then giving people the tools and showing them
with buttons to bress and things like that. You're not going to get any business benefit there, but it's required, right. You need to give people licenses, you need to give them their people access, and you need to tell them what these things are. We can certainly help with that, but you can do that as well. But the benefits really start coming in at this next stage, and that's the literacy thing. Now you'll notice here this isn't a box.
This is a box with an arrow. The reason for that is this literacy thing is now an ongoing challenge. Like the AI you're using now is the worst AI you're going to use for the rest of your life. It is going to continue to change, not just with feature. It's going to get smarter, it's going to be able to be plugged into more systems and all those kind of things. So the literacy part will have to change over time. This is an ongoing thing that people need
to be involved in. But certainly get people up to speed with what the tools are, how they work, how they think, how they hallucinate from time to time, how to best work with them so you get fewer hallucinations, how you can fit it anyr work day, all of those kind of things, and so that literacy is there
as a baseline for everything that comes next. From there, we really see those kind of two things, and the first is I'm calling an individual leverage, which is how do I get it to help me in my day job? Knowing what I do, and I've then got some great literacy on how I can use AI to do that. Then I can start building things like agents. I can start building workflows and fitting how putting it into my
workday and getting some benefit out of that. And the other key on this one is actually sharing it with your team as well, so making sure that individuals know how to get the best out of this tool and know how to look after these things that they're built as well, so we don't just get old agents that kind of work, that kind of don't work, and processes and things like that half broken. So that individual leverage
is the first next step. And then after that, what we generally find and what we recommend and do help companies with is that organizational leverage, which is we have this workflow, how do we get AI to help that? And the organization and individual leverage often starts with I'm doing this thing, how do I get AI to help me do this thing? In the same way just a bit different. The companies that are really going ahead, they're the ones who are saying, now I've got AI, how
do I re envisage this workflow? How do I really re envisage the inputs the outputs that we're getting from these How do I actually move the company's dial rather than just shove AI into a step? How do I say, now I've got this an amazing tool. How do I change the way we're delivering so it's more effective, more productive, those kind of things, and that's where the big benefits come from.
And just on that. It's not just about.
Finding people who are really really good at AI or really heavy users. It's about training people or finding people that think in terms of workflow, like like what is the process? How do I unpack a process step by step?
How do I know what AI can do and what it can't do?
Because what Neo and I find and the rest of the team when we're working with organizations on this, there's a lot of people that think AI is magic, like it can create high quality data where no data exists, it's not that good. So it's really important that like when we're looking at individual leverage for individual workflows and organization or team or function workflows, is that these people not only no AI, but they also think like a
workflow designer to get the best stuff. And like, obviously you know whereby us we're do great training on this, but you just need to make sure you've got that skill set in mind.
It's not just about knowing AI, and.
Often we find that it's in different people. So you might have the IT person of helping out, and you might have someone who's like a business analyst kind of processing kind of person helping out. And you might also have someone who's like that into the SME expert who knows their organizational lens and the three of those people are often needed to pull in the needs and the
goods all those kind of things. But yeah, it's not they get one person to do it or a self serve, particularly without training, because then you're going to get a lot of broken processes and upset customers and things like that. So yeah, absolutely you need to be a whole organization behind this. And you'll also see this diagram and really provi should go on forever because this is now an ongoing challenge, so it's not once again. It's not a
one and done. You might have improved this process, but the tools will change over time and so we need to continue revise and look at these to see if we need to optimize them again.
Okay, one more solution before we go into questions.
So AI will give people their time.
Back, but guess what happens if you don't have a plan for where that time goes.
They're going to fill it with more work.
And that's probably going to lead to brain fry because generally we're filling it with like deeper, more cognitively intense work, particularly when we're reviewing AI's output, which is now a large part of our day that requires vigilance and critical thinking and stuff that requires more brain power than the task that we're outsourcing to AI, which typically require a little bit less brain power. So this is really important.
So leaders need to really give some thought to this and communicate it, like, do you want people with all these time savings that you're getting or are going to get or have been promised? Like, do you want them putting that towards innovation where previously they had no time? Do you want them putting it towards the customer and learning more about the customer, speaking more with customers, trainer
up your customer satisfaction scores and net promotor scores? Do you want them putting it back into learning and development? Because I'm yet to meet an L and D professional who's like, oh, people have so much time for learning at our organization. Or do you want them putting it into god forbid, work life balance? So crazy idea, So be really thoughtful around that, because if you are not people will just cram it with more work. Thanks so
much for listening. If you are in the middle of an AI rollout and want some help getting it right, you can find us at Inventium dot ai.
We would love to chat. See you next time.
How i AI was hosted by me, Amantha Imber and Neo Applan. A big thank you to Martin Imber who does our sound editing, and Jem Rubio for production support, and thank you to John Kilby who composed the theme music.