¶ AI in L&D: The Implementation Inflection
Hello and welcome to the MindTools L&D podcast, a weekly show about work performance and learning. I'm Ross Dickey and this week we're speaking to Don Taylor about his and Aglo Vinescaita's most recent focus report. AI and L&D, the race for impact. Hello, Don. Ross, hello. Great to be back. Great to have you with us as always.
As you point out in the introductions report, this is the third report that you and Agla have produced together. The fourth, I think, since the release of ChatGPT. Acknowledging the limitations of... the data that you've been able to gather and how that's sort of changed year to year based on the respondents. What's changed since, you know, over time? What's changed in the last few years?
I think it's, thank you for acknowledging the differences. I think that's really important. I think the key thing that's changed this year, and as you say, we don't know if it's the same data set. There are more people this year. So we have. different people voting. But the key thing that's changed this year is that we passed an inflection point in the past 12 months. In the previous three...
previous two annual surveys, and this is the third annual survey, we ask people a question, how far are you down the road to using AI in what you do? And it's a multiple choice question with six answers, which can basically be grouped into, are people before? trying it? Are they experimenting or piloting it? Or are they using it or using it extensively? This year, for the first time, more than half the people who responded 54%, as opposed to 40% last year.
said they were using it or using it extensively. That's a big jump. And all the other categories are squeezed down. What we have is people now regarding our surveyed population. regarding AI as being part of the toolkit. It's gone from being a shiny toy to being something we just accept now, or at least our survey population. And I keep saying that because I'm well aware that the people who respond to the survey...
Other people are enthusiastic about this and excited about it. It doesn't necessarily reflect the general population today. I do believe it will reflect the general population in three years' time. I think that we're just at the sharp end of the... diffusion of innovation care with the innovators and the early adopters. And we know from experience that those things do pass down the line. This is not a train that's going to be reversed. Yeah.
You refer to this moment as the implementation inflection in the report, as we've now passed more than half of people using AI regularly in their workflows in L&D.
How have we got to that point? Is it just greater awareness of these tools? You mentioned social proof in the report as well. What to you are the kind of key factors that have... have got us to this point where it's becoming almost the norm uh for lnd to use ai i think two things and then you've got a caveat the two things are yeah utility and social proof
And the caveat is, what exactly are people using it for? When we say you're using it or you're using it extensively, what does that mean? Yeah, I was wondering that... as well as extensively will mean maybe different things to different people. And one of the things I was thinking about while reading the report was AI is just increasingly...
embedded into the kinds of tools that people would be using day-to-day before even ChatGPT was released to it is now, you know, if you're using Microsoft, it's there with Copilot in your Word documents and your emails. If you're using Google Suite, you've got Gemini. I don't know if it's yet, but, you know, Chrome is going to, Gemini is going to be integrated into the browser. I read this New York Times article recently, which was, the title of it was 48 hours.
without ai and i thought reading the title i thought okay this is going to be 48 hours without chat gpt and it's going to be this you know the author's reflections on how dependent he's become on it actually it was about
48 hours without contact with any type of artificial intelligence in general, including machine learning. Machine learning has obviously been with us for much longer than generative AI, which is what people naturally think of when we talk about AI. And he ended up... foraging for food in Central Park because machine learning is just embedded in absolutely everything that we do from, you know, he lives in New York City.
the way that water is managed in reservoirs, for example, that machine learning is involved in that process. We're obviously quite a way off from generative AI being that prevalent to the point where you can't escape it. But it does feel to me that it is just... slightly everywhere i think maybe the last time we had you on the podcast we talked about how it is a sort of foundational technology that whether people choose to adopt it or not it's just going to start being
absolutely everywhere and you're not going to be able to avoid it i love our chats ross because you have a very balanced well-read and researched background and well-thought-out view of what this means. And yes, you're right. For most people... I mean, these compliments are why we keep inviting you back. For most people, AI. It's synonymous with generative AI, but of course, yeah, machine learning. And I try to just refer to computational power. And I say, look, we've got computational power.
increasing in scale by several factors since about 1930, when you have your first mechanical and electrically supported devices. To the extent where they're now, I think it's something like you can buy 100 million times as much processing power now for a dollar as you could when I was born in 1963.
So, and that's in the areas of transistors. And it's not just about, it's not just about chat GPT. It's about all you can do if you've got this huge, huge computational power. But our mindsets are very much, you know, that's my lifetime. Our mindsets are very much in the sense that, oh, well, we expect these things to happen this way because that's how it's always happened. But that experiential growth in scale of computational power for $1.
So it's not that you can do it, you pay more for it. It's just that the price has plummeted. Combined with huge data sets being available for free on the internet, is that wise? Should they be available for free? They are, unfortunately, and they can be plundered. combined with smart algorithms, combined with really smart people going to work on it, all those things together get us to the point whereby you can't live for 24 hours without...
unless you're foraging for parsley in Central Park. So I, by the way, I spent increasingly some parts of my life living in Wales, in a small town, which... It has an economy which you can pay for things with cards. You also do a lot of stuff with cash. And there is very little AI in life there. And there is much more interpersonal conversation.
And it's a salutary reminder of the stuff that is important in life. I realize we're getting off topic here, Ross, but I think it's not bad to step back and look at the big picture because it is so fundamental. Like you said, it's a foundational. technology. It's a general purpose technology, like fire, like writing, that is going to transform and is transforming what we do. And we're seeing snippets of it. It's like we're on a boat.
heading into new territory and we're looking out through the portal and we get a glimpse of new territories passing by. We don't see the whole picture, but even those glimpses should be enough to tell us we're in a new uncharted territory. Yeah, absolutely. I mean, I enjoyed the tangent. I think it's the thing is when we were talking just before we started recording about how a few years ago when ChatGPT was released or a bit more recently when say NoGlukelam was released.
There is wonder with these technologies, what they could do. And very quickly, it's become this sort of norm. You're like, yeah, what else is new? And it's amazing that we've got to that point. So I think to your point about... this community in wales i think it's uh it's very easy day to day i think to to not to see the bigger picture and how much things have changed just in a short space of time and where we're going
¶ Evolving AI Use Cases: Data Analysis and SME Collaboration
To bring us back to the survey, what do you think are the kind of most interesting changes in the quantitative results of the survey? And what do they tell us about the direction of travel or where the boat is headed, to use your analogy? Yeah, that analogy came out of nowhere. I'm not sure how durable it will prove to be.
And I think that for me, what's interesting, the quantitative side of it, so the numbers side of it, if we think about how things have... changed in terms of the numbers that we get in the survey initially looking at it just on the surface it appears almost that nothing has changed so we ask people what do you use AI4, and the uses seem to be very similar to what they were in last year's survey. And we give people a list of 12 things.
They can choose as many as they want. Interestingly, there's a slight increase in the number that people choose. It's just, I think it's, I can't remember what it is. But anyway, there's an increase in the number that people choose this year very slightly. So they're using it for more things. content dominates and content design dominate the top half of the table. So of the top six things on the table, I think all but two of them are content and design. Yeah. And so you think, well, okay, so...
well, people are using it for content, so what? But then two things are apparent. Firstly, there's one big change, which is that qualitative data analysis has come up. Last year it was ranked, and we did an interim survey of April. 2024. So that's what I'm comparing this with. In that survey, that choice, qualitative data analysis was ranked eighth. This year, it's ranked fifth. And that's a big change. That's the only major change we've got, but it's substantial because it's talking about data.
And it's people looking at the words they get. Maybe it's survey results. Maybe it's something else. Maybe they're analyzing transcripts of phone calls, but it's quantitative data they're analyzing. And that change reflects something else. When we look at the other way we ask people about what you're doing with AI, we ask them to fill in a free text box. How do you use AI in LMD? And we had over 600 responses from over 50 countries.
20,000 words written in total in response to that question. If you read Kafka's Metamorphosis, it's something like the length of metamorphosis, right? So it's a book, right? A slightly weird, otherworldly book, okay, of what people are doing. And you look at it, and there are two things we do with that. We do our own quantitative analysis. We look at it, we check what people are saying, but also we just do a straightforward...
Quant analysis, we look for certain words and word stems, and we find that some of them have risen in importance over the past three years. And the three that really stand out that are in the top 10 most important words this year and were nowhere three years ago, they are analysis. data and research, and variations thereof. Analysis was 28th of all the words that were chosen in 2023. Now it is fifth, and so on. Data was 37th. Research was...
The 53rd of all the words that were being used in 2023 now is tense. So this shows that when we do the analysis of looking at how people are using AI... I'm always worried that we have a bias that we're introducing into our analysis. But just looking at the words themselves and how frequently they're used, we can see that, no, it's real. But underneath that apparent lack of change...
there's a sort of groundswell that's shifting of increased sophistication. And it's not about content, or if it is about content, it's about using things like data analysis and research to better inform how you produce content.
That wasn't something I was expecting, but it's something that's really happened over the past three years. Yeah, and I think I'd imagine part of that reflects the evolving capabilities of the systems themselves. So you have... tools like open ai's deep research um which i don't think was available in 2023 or if it was it would have been available to a smaller sort of subset maybe of of paying subscribers
And then I think tied to that, we've probably also got the kind of social proof aspect of it where over time you maybe hear about more people using it for research and then that kind of drives usage up in that area. But I do think... I think that was one of the things that I found interesting and kind of heartening from the survey results is that people are using it in this way. I mean, out of interest, did you use AI to support...
The qualitative analysis of the data in the report. Yes, a little bit. The qualitative, not the quantitative, the quantitative stuff. Unfortunately, the numbers, I still do old school. It's not great numbers. Yeah. But yeah, the quantitative, yes, we do. All the writing, all the writing is done by hand. It is handcrafted lovingly in ink and steel nibs. Yeah, I mean, I ask because it's something that I have...
used it for. And I think, you know, if you think about use cases like user research or focus groups where you have all of this unstructured qualitative data. and you want to do some sort of thematic analysis on that to draw kind of the key themes that people are talking about, doing that manually is just very complicated. And you maybe could have done it previously if you'd had some sort of software, but...
It's incredibly time-consuming, incredibly time-consuming. And, you know, you have to go back and double-check things. You have to ask yourself, hang on a second, is this real? Show me quotes to illustrate what you're talking about right now. I'm going to go back and find the quotes and check.
Do I really believe what this is saying? And you have to be pretty rigorous about using it because it is so easy to be seduced into thinking the answers, because they're always presented in a very polished way, but the answers are... true. They may be true, they may not. You have to go back and check. But Egler did some really good work in going through these answers. And we categorized the four main uses that people are using AI for is content and design.
operations and other things, strategy and insights and workforce enablement. These things have not shifted overall in the sense of that list remains pretty unchanged. But what people are doing inside those boxes has shifted so that we're seeing, for example, just one very simple thing about content and design. People increasingly talking about looking at using...
AI, generative AI tools, to check the gaps in what they're producing. So whereas last year it was like, well, I want to do something, make me a nice picture. Now it's, does this fit? What am I missing? And they're using it much more as a collaborative tool. There's a lot more interaction taking place with AI and not accepting the answers verbatim, but challenging them and expecting AI to challenge you as the author.
The other thing that's really big for me that I'm seeing in the content and design side is the change in relationship with subject matter experts. It's now possible to get much better briefed before you go and talk to an SME so that you're going and asking the right questions.
to get information out that is really helpful, rather than the whole discovery phase being done entirely manually. A lot of the discovery phase can be done using AI. And there's one particular company, it's not actually mentioned in the report, but we did an interview with which...
is a legal company in Manchester. They have a research and innovation team that produced a, that had their own internal LLM. Of course, legal companies deal with huge amounts of words. This is perfect. And this is all obviously. air-gapped from the internet. So it's private, it's safe. It is effectively the repository of knowledge of the organization. It's used by everybody, from senior partners right away down to newbies, and by the...
L&D team, the L&D team go and use it to say, right, okay, if we want to go and write some content for something, what should we be doing? They treat it as an SME and then they can go off and they can use these people's time because it's extremely valuable very effectively.
to check what they're doing, rather than having to go through an expensive discovery phase. And I think that's one example of a shift in how we're using AI now, the content that certainly wasn't taking place two years ago. Yeah, I like that a lot. I mean, I think in L&D, we often have to know a little about a lot of different subjects. So we'll be going and speaking to subject matter experts, who are experts and subjects that we in L&D...
probably don't know very much about until we start having those conversations. And so as you're saying, that discovery phase, part of it is just asking questions that the answers to which might seem obvious to the SMEs, but we need to kind of spend time clarifying.
in L&D and so you can do some of that initial legwork to get yourself up to speed and then go and ask the SMEs more direct questions about what are the specific performance challenges that we're facing in our organization around these issues rather than explain this concept to me or what does that mean or how does this work yeah exactly yeah that idea has sort of then then
goes through other iterations. So if you can do that with an SME, what else can you do it with? Well, you can do it with preparing documentation. So when you're preparing documents to report what you're doing to the board or to go to have a stakeholder meeting.
use the same approach. And we're seeing people taking a very proactive view to getting their pitch done right for an audience. So if you're going to go and talk to a stakeholder, you're not going to talk about the learning stuff. You work with your... typically generative AI, large language model, in order to get something that has the right tone and points of interest for that audience, rather than doing it in L&D speak.
that makes a difference to the likelihood of your success. So the idea that you're using this to transform the way you interact with people as part of O&D is something which I wasn't expecting, but it's... how it's evolving. And I think that's quite exciting. Yeah, absolutely. I think on the one hand, you could be quite sniffy about the fact that that remains the primary use case and we're using it to churn out more content and to do it faster.
¶ The Race for Impact: L&D's Strategic Imperative
On the other hand, you could make the argument that that is a lot of what's expected of L&D by the businesses in which they operate if L&D departments are... Using these tools in a way that allows them to do what's expected of them at lore. cost more efficiently maybe more effectively in some cases if they're using you know some sort of learning design co-pilot that is a kind of impact it's maybe not learning impact but the title of this
of this report is the race for impact. How do you and EGLA define impact or what kind of version of impact did you have in mind when you were thinking about this report and writing? And that's, it's a great question, Ross. Let's go back to the previous reports and what the titles were. So the first one back in November of 23.
was the state of play we were a year after the launch of chat gpt we just wanted to know what the hell's going on and we just want to say is this all mumbo jumbo nonsense or is there some reality there so that's that's that title Where did that come from? The next one, which was an interim one six months later, I think, April 24, from talk to action. We'd seen a shift in that period where people were still talking about it, but actually things were starting to happen.
October 24, the title was Intention Reality. Well, we want to achieve something. Are we doing it? No, there was a bit of a mismatch. And then we got to the stage now where we've gone from talk to action. People are much clearer about.
what their intention is, and they are really achieving it. The race now is to show impact. And we've chosen the word impact. You could have chosen value, but impact is the word because there are... so many ways in which Ellen is under pressure at the moment that people are seeing AI as a way in which to show impact, but also that there's a race to show impact because...
If you don't, then why the hell are you using it? And I think that there's a sort of, so what we're trying to express there is this dichotomy that you're under a lot of pressure and you've got to prove yourself and you've got to prove this tool you're using is worthwhile.
And at the same time, we know that it is possible to do it. In the previous report, we had a report of HSBC using a conversational... training tool for customer support staff where we were able to show a 10% increase in customer satisfaction ratings for people who'd been trained on that and also the operators themselves felt more comfortable and confident in their work.
through having gone through that. Something like a flight simulator for customer support staff. So it is possible to show it, but I think the race is on. And there's another reason why there's race for impact. And it comes back to the content thing. Yes, L&D is expected to produce content and is part of the job. But the problem is anybody can produce content now with a good prompt or even a mediocre prompt. And the content may not be very good.
but it will do a job. So L&D is now in a world whereby it is not the sole producer of content. Lots of people can and will and are doing it. And so if you are measuring yourself and your activity... And if you're seen to be measured against the amount of content, it's never the quality because people can't judge that from a distance. The amount of content you produce, then you are on a losing game. Content has become, like it or not, largely...
a commodity. And for L&D to stand out, it needs to focus not on producing content, that may still be the job, but on producing impact. And that's why we have to have this race for impact. Because otherwise, if our job, if we see ourselves as producing more stuff faster, then we are comparing ourselves with the sea of material in an undifferentiated way.
¶ Future Vision for L&D: Transformation Triangle
And I'm afraid the result will be fading away into obscurity. I completely agree. And I think that brings us on nicely to, in the report, you kind of sketch out a vision for the future of the profession moving beyond. content production can you give a brief overview of the transformation triangle i should know also that there are several in-depth case studies in the report they're well worth a read if anybody hasn't read it yet
and wants to see some kind of tangible examples of what people are doing in practice. What could the future look like if we seize the opportunity to do something different? Thanks, Ross. I think it's important to say this is what it could look like. And we're not trying to sit in an armchair and wave our arms around.
and pull something out midair. This is trying to reflect those 20,000 words that people came up with. The full case studies that we've got in the report, we've got only eight full case studies, 10 snapshots. and all the stuff from the previous three reports as well. What we're saying to ourselves is, where do we go? And from the focus on content, we see three possible futures that may change in the future.
but three possible futures. One is the skills authority, one is the enablement partner, and one is the adaptation engine. I can do a quick sketch of those very quickly so you get a sense of what that means. So the skills authority is where the... L&D department is focused on skills as a business resource. Let's say you're a consultancy, your people need to know and be able to do stuff. And it's the L&D department as a skills authority that makes that possible.
And it's a critical business resource. If that doesn't happen, the company can't make money. So that's one approach, skills authority. And it's different from just having a competency framework, which I'll explain in a minute. Second one, enablement partner. The L&D department is a supporter of the great work that is done typically in a large distributed organization by people who know how they do their job well.
and can find small increments in productivity locally, perhaps large increments. And then the LD department supports in producing that in capturing what we're doing, and then... distributing and sharing that more widely across the organization. So they're not the bottleneck producing content, but rather the enabling partner, and they can focus more on strategic things.
Adaptation engine is very different. That is the most radically different of these three nodes on the triangle. The LED department doesn't exist anymore. It is part of a group that exists. to focus on performance and ensure that performance issues in the organization are tackled using whatever is necessary.
approach to talent, it could be approached to systems, it could be an approach to learning or performance aids or whatever. But it is not starting from the point of view of learning, starting from the point of view of performance issue. and a multi-talented team that goes and tackles it. So that's the three options. And we see these, these aren't parked out the edge, as I say, we see these reflected in the case studies.
¶ Practical Steps for L&D Transformation
and the work that we're doing, including interviews that aren't published, the work we're doing to understand what the heck's going on at the moment in our field. Great. I mean, I'm conscious of time, and this might be a whole other podcast, but Hedge L&Z seems... get there so people might be listening thinking that sounds great but in my organization we're seen as the trainings that sounds great but don come on be realistic how how the heck yeah yeah
Well, we're seen as the training department and we're measured in how many courses we've developed and delivered and that sort of thing. And I would love to do what you're talking about. Where do you start, I think? Because we've maybe got time to talk about that. It's a perfectly legitimate and fair question. And I would always take something like that Arthur Rash quote. Start where you are with what you've got. And...
For example, the skills authority, you might say, look, do I have to start a three-year program of overhauling the entire organization? No. But for all of these things, you need to start with one. person, a manager and leader that gets it, that understands that L&D is about supporting the business goals and helping the business and individuals in it flourish. And they can do it in a number of ways.
Your ability to persuade that person to take you seriously in doing this will rely on you having had previous L&D wins. and on understanding the business. So that's where you start from. The success will then depend on the culture. And typically we found in all three of these nodes, a culture of trust on all sides is a really important part of this. So the manager has got to trust that...
you can deliver. And they will do that if you have delivered. And from that, you build a reputation internally. And this is true for all of these. You build a reputation internally for solving business problems. which help that manager or that leader get the job done and look good. They then become your advocate and you're on the road to starting change. There's nothing new about this. This is how it's always happened. The thing now is that it's imperative.
Because the old bedrock on which L&D stood and could retreat to, well, we're just going to go back to creating courses, is not going to exist in two to three years' time. So change is now a fundamental requirement. Great.
I think that's a good place to wrap up. I genuinely enjoyed the report. I think there's obviously a lot of talk on LinkedIn and other places about AI and what's possible and you get this big sense of... what other people are doing but I think the report put some meat on those bones and it's well worth a read so we will pop a link to the report in our show notes
¶ What I Learned This Week: Trivia and Etymology
Thank you, Ross. Thank you very much. Okay, so we'll move on now to our regular feature, what I learned this week, where we share something we've picked up over the last seven days. What have you learned this week, Don? You're laughing because I...
I always forget you can ask this question. Can you answer it? And then I'm going to think, what have I learned in the last seven days? I'll come back. I'll answer it, yeah. So the thing that I learned, I have been taking... more flights this year than I would really have liked to and I don't know I think this is maybe something that a pilot announced at some point but I hadn't appreciated that
Planes fly at different heights depending on their heading. So eastbound flights generally will cruise at odd thousands of feet. So 31,000, 33,000, and then westbound flights will cruise at even thousands of feet. So 30,000, 32,000, that sort of thing. They always say, you know, we're now cruising at X,000 feet. And I thought, well, I don't really care.
But apparently there is a reason for that and it totally makes sense. So it's all about safety. So planes flying in opposite directions should not be flying at the same height in theory. So that's what I learned this week. That's brilliant. That is actually really interesting. And it would then cause you some concern if you were heading east and you were told, yeah, we're heading at an even number of feet. That would be scary.
Yeah, exactly. Run up and knock on the door. Are you sure you should be following that down? Even thousands of feet. That was quite a short one, I'm afraid, Don, but has it given you a minute or so to think of what you've learned this week? Exactly, exactly. So I'm really booked on the...
the English language, Loving or Amorous, Amorous or Loving by Rupert Gavin. It's really, really good book, very funny about what happened, where the English language came from and so on. And there's a very short chapter. It's only about three pages on the forgotten tribes. And basically it talks about how Celtic, some Celtic words, and I've learned a lot from it, but some Celtic words have come into the English language.
Something I hadn't appreciated is that the word for Wales, which was something like reals, was originally an Anglo-Saxon word. And the Angles came in and they saw these people and they said, well, they're Wales. They are foreigners. They are different. And that is why Wales in English is... which is the word we use in the English language, means foreigners, which is rather harsh. After all, these are the original inhabitants of the islands living in their own country.
It tells you so much about our history. You could really pick a scab if you wanted to, couldn't you? Well, I'm not going to go there, but yes, it does. Yeah, let's all go there. And the Welsh word, of course, for Wales is Cymru. or gunglu, depending on how it's formed in sentence. And of course, that means community. But interestingly, of course, that word Wales is the same word you see in Cornwall.
That means that's the Welsh people in the horn, down the bottom. And it's also from what the Walletians, I think, from the Germans or Anglo-Saxons called the Walletians. the foreigners, because they were on the border from the Germanic-speaking countries. So there we are. That's what I, one of the things I learned this week that I can sort of remember.
Amazing. That sounds like A Book the Bureau of My Street. We'll pop a link to that in the show notes as well. It's a super book. You'd love it. You'd love it. And it's very funny as well. Yes. And that's it. That's all from us for this week. If you'd like to share thoughts on anything we've said on the show, you can find me and Dom on LinkedIn. For more from us, head to mentos.com slash business. Thanks for listening and bye for now.