Cognitive Science & Choosing the Right Media for Learning with Clark Quinn

I was thinking about the context for this, and I realized the perspective from cognitive science, from understanding how our brains work, uh, cognitive science was an integrative, uh, way to start pulling together different people thinking about thinking— philosophers, neuroscientists, uh, cognitive psychologists, anthropologists— more learning science emerged from that, similarly trying to pull in instructional design and, and educational psychologists

and cognitive researchers. And that perspective isn't, I think, well known and used enough. And yet it gives us insight that's really powerful like that, the properties of media. And so that's the broader perspective I'd like to share is that this is coming from a way of thinking and looking at the world that gives us really useful handles. And I found it extremely

useful. I have the ability to give people insights that they wouldn't have gotten in other ways just because of that understanding how our brains work, which I think is, increasingly going to be important because that's where things are going. We're going to find out what technology can do, what brains can do, and we need to understand that really well to figure out where we fit in this continually evolving world.

Good morning, good evening, good afternoon, wherever you are and wherever you're watching from. We're revisiting an episode we recorded 2 years ago with Clark Quinn. When we recorded this episode, AI video tools weren't creating content in seconds. Today you can generate a video with a prompt. You can clone your voice. You can build an entire training series without ever even turning on your camera. Pretty amazing. Maybe. But it also

makes this conversation even more important. Just because you can generate video in seconds doesn't mean video is always the right medium. And I say that as someone who loves video. I work at a company that builds tools for video. I believe deeply in its power. Video connects, it teaches, it builds trust in ways few mediums can. But it's not the only tool in the toolbox. Screenshots matter, images matter, diagrams matter. Sometimes a static visual communicates something faster and more

clearly than a 5-minute video ever could. And today, with short-form video dominating feeds with reels and shorts and constant motion everywhere, The question isn't just, should I use video? It's what kind of media best supports learning? And even more specifically, what kind of video best supports understanding? And that's why this conversation with Clark Quinn feels even more relevant now than when we recorded

it. Clark challenges us to think beyond format and into cognition, how our brains actually process information. Static versus dynamic, concept versus context, diagram versus animation versus video. In a world where creating content has never been easier, choosing the right medium has never been more important. So hopefully you'll listen with that lens, not just how do I make more video, but how do I help someone understand better? With that said, here's our episode with Clark Quinn from

February of 2024. Uh, Clark, I, I, we always like to start

the show with a kind of the practical approach and you know, you obviously you're not necessarily a media creator. I'm sure you've created media in throughout your career, but as you think about from a learning perspective, what's, what's a tip you would give to folks out there who want to use video? They want to use it for helping people to learn. Is there a tip that you would give them to help them to be maybe more successful or, uh, maybe have that hit the mark a little bit

more? I guess my perspective is don't use video for video's sake, for making it more compelling or more interesting. What you should be using video for is to convey dynamic stories. Um, too often we use video when we don't have to, and it's a high bandwidth and high production cost, uh, effort in many times, in many cases. So, I would like to reserve using video for when it makes the most sense, not as a panacea and end-all.

We'll talk today about the specific ways in which certain different media most opportunely support certain types of learning outcomes and cognitive outcomes, and then we can dig into, you know, How can you swap them in and out for variety and a variety of other things? Well, I, I

appreciate that as someone who, uh, you know, I use a— I make a lot of videos. I work for a company that promotes tools that you make videos. I, I appreciate the advice though, because I do think there's, there's this balance there, kind of time, cost, effort, and always, I, you always got to ask, is it the right medium to deliver the message? So I appreciate it. No worries.

And it's just our brains have evolved to, uh, collect certain types of information in certain ways and process it in certain ways. And we want to ideally match for that, particularly when we're, you know, trying to convey the most important message. There are, you know, video can capture us talking like I'm doing right now, and we can communicate thoughts and elegantly And, but too often we can overuse it. We can use it without

controls. I just saw an example of this the other day where they were providing things and they didn't really give you pause and restart options. You could go back, play the whole thing, but it didn't automatically just say pause it because, you know, somebody's just come in the door or something. We need control over the media. But think about, Um, Ken Burns and the Civil War, uh, movie he made, it was a whole bunch of static images sequenced together.

Those static images were what they had at the time, but they also communicated in powerful ways when he sequenced them together. We don't— we can communicate those stories in multiple different ways. We can use graphic novels, we can use narrative prose, we can use sequences of images linked together with audio or text, or we can use video. When does each make sense?

That's the important issue for me. Yeah, I love that. And, you know, I mean, we, we often refer to Ken Burns, and he's such a master of doing those things, right? But those are not, uh, not outside the scope of anyone's capability. People can use imagery and they

can tell stories, and that— so I love that. Um, and we could probably talk about just that alone for a long time, but I want to get into this idea of cognitive properties, because when you— when you— we— I connected with you and you said, hey, here's some things that we talk about this, and I'm No, um, but I think we need to do a little defining up front because my guess is most people listening to this are saying, cognitive properties? What

does Clark mean? And so lead us down this path of like, what is this idea of cognitive properties and as it relates to, to media, right? I was actually thinking you were going to ask me to define the different types of media because people describe those differently. Oh yeah. So I actually want to start with that because it's at the front of my mind right now.

What— I have left of one. Um, when you think of, uh, diagrams, and that was one of the things we talked about discussing, they communicate conceptual relationships via spatial relationships. So when you say up here is this and down here is this, and we have a linkage between them, and the other things are linked this other way. We are communicating information. That's not video, that's a static image. And two things, um, cognitive properties I want to separate out now are static versus dynamic.

So there's information that exists, you know, the relationship between, um, roots and, and branches. There is dynamics that the roots feed up and grow the branches, but the static relationship is that, you know, the roots are the nourishing from the groundwater, whereas the branches are exporting the leaves, which get energy from the sun. And together they put those together.

Then we have the more dynamic story of an environment where, you know, the, the water cycle, uh, evaporation, condensation, precipitation, and whatever the word is for where it flows down the hill and gets up in the, in the body of water to evaporate again. We have that. Cycle. That's dynamic. And sometimes we need to communicate one, and sometimes we need to communicate the other. When do— so I like to distinguish between diagrams and

animations. And when I say animation, many people think about, you know, cartoons moving around. I'm thinking specifically of animating a diagram. Mm-hmm. And then we have photos which capture context. And videos which capture dynamic context. Then we have text, which captures, uh, prose, you know, is a narrative, and then we have audio, which is also a narrative, and each of those are different. Diagrams and animations capture the conceptual.

They're not tied to real context, they're an abstraction, whereas videos and images capture the actual context. So you can see behind me what my room looks like because it's actual context. We could just have an abstraction— I mean, a cartoon with a blank background or some arbitrary abstract background which could communicate something different. There, what I begin to talk about the cognitive properties, and when you want to communicate context, a photo or a video makes sense.

But when you want to communicate a concept, you may be better off with a diagram or an animation. —and when you want to communicate narrative, you may want to tell a story. You can— now, the point I mentioned earlier about, you know, having still photos together become a, an, a story, um, video can be used to communicate narratives and is— why do we watch movies and TV series? But we're showing the context as well, and we don't have to infer the concept.

Unless we were laying it over. So, from a learning perspective, the cognitive properties we're talking about are: we communicating concept or context? Are we communicating it dynamically or statically? Relationships? Those are the things— the cognitive properties I'm talking about. And then we need to start thinking about which

we need to communicate and what media makes the most sense. And then, of course, we then think about And then how do we mix it up so it's not completely dull and boring the whole time?

Well, okay, so there's, there's a lot here and, and those are— I think we're, you know, you're doing a great job at simplifying what is a very complex kind of concept in terms of, you know, there's these properties, these media, this media, these types of media that are good at typically doing. So if I, if I'm at the, the, you know, I'm the instructional designer or I'm thinking about creating media for, for learning,

I'm hearing what you're saying. And so now I'm thinking like, okay, what are maybe the questions or decisions that are going to help me down that path? Because like you said, I— if I'm always using a diagram— diagrams are great, they do a great job with the thing. But if I'm always trying to use a diagram, it feels like, like you said, the— you get fatigue. Like, I— we actually struggle with this in video, right? If I see— put a video up online and it looks the same as every other

video, people say, I've already watched that video. Or, you know, so I'm curious from your perspective because there's the, the practical application here going from this like understanding of these cognitive properties to now I've gotta, I gotta make stuff. Like a lot of our, our, a lot of our roles is as instructional designers, particularly if you're doing, uh, really good in-depth design is say understanding those things, but then Now it's like, okay, what do I do with it?

So what, what would you say? How do we translate this to— what, what can someone do now if they understand these things? Well, two different takes on that. So let's talk about the traditional instructional experience first. You should be presenting an underlying model. Cognitively, models give us good basis for, uh, comparing our performance to what the model predicts. It gives us a basis to make decisions You know, models help us explain what happened, uh, or predict

what will happen. And then we can say, if I did X, the model says this would occur, and if I do Y, the model says that will occur, and this is better than that, so I'll do this. The model tells us that. So we want to communicate a model, and the diagram makes sense, but then we want to show how that plays out in context. We have evidence, research-based evidence from cognitive load theory and the like, that showing examples before we give people opportunity to practice

makes it easier to then take it on yourself. You see and abstract a few examples and it gives you some guidance. Examples need context. They show how the model plays out in the real world. So suddenly we're talking about watching a video or seeing a static image or a sequence of images telling a story. Examples work best 'cause they really are stories that show, I faced this problem, I applied this model, and this was the outcome.

That's a story, that's a narrative, and we could use video or, or st— sequence of static images to convey that. Then we need practice. We need to immerse the learner into an environment and give them opportunity to make choices. Interactivity is something I haven't mentioned in terms of the cognitive properties here. This is more for the media instead of the interactivity, but then we need to allow the learner to make choices and see the consequences of those.

So suddenly we're switching again to— and we might— one of the things I didn't talk about in my elicitation of the types of media, I talked about, you know, video and our, you know, context with photos or video. And I talked about concepts with diagrams or

animation. I talked about narrative with prose or audio, but I didn't talk about graphic novel formats and comics and those types of things where they're semi-conceptual and semi-contextually because they strip away some of the details of context. And yet provide enough that the learner can recognize the context, but that allows the opportunity to layer the concepts on top of it as well.

And so we might mix this up. We might have a graphic novel format of an example and a video for an example, and then we put the learner, and it might be photorealistic, or we might start with a graphic novel. So we're mixing up the media to tell the story. Now, I said sort of two different pedagogies. One was sort of the traditional, you have a model, example, practice. The other one is you put the learner in the situation. This is more a problem-based type of approach where you specifically

chosen an important problem and you provide scaffolding. You've simplified it in certain ways early on, and later on they take on more capability. But we might make resources available in the environment so you can pull up a diagram or you can pull up a little example. We might embed that in the story. So there— the environment in which you're performing in might have a library of, that has the diagrams, and it might have a history of case studies that the organization has done in the past that

you pull up that are really examples. So we are mixing up media to meet different needs because for learning purposes, we have different cognitive roles at different points in the experience. So we avoid boredom in that sense, but then we should be thinking about how are we making sure we're communicating each at the right point in time.

Wow. Uh, you know, what I'm thinking about is, uh, so I've noticed this trend and, and, you know, obviously we can talk about instructional designers, corporate learning all, all day long. I've noticed in, in the world of non-instructional designers, there's a lot of people out there making training content. It's, you know, we would, we would probably balk at it a little bit what it is and, you know, but they're on YouTube, they're YouTube creators, they're teaching, uh, cooking or fix a car.

But what I— in that first, particularly in that model, right, where you've got, uh, you know, you got— you're gonna put these things together, right? Model, example, practice. What I've noticed that they're really good at on YouTube, if they're doing well on the channel— so not everybody's good at it, but you know, the ones that are really good, they, they, they hook you with that. Here's the problem, right? They somehow draw you in, but

then they, in a lot of ways, they're essentially doing this, right? They're modeling something. They're Showing the example, but it's not a linear boom, boom, boom. They— it's almost like a lot of loops because they're coming back to the, like, the thing. Here's the problem. Here's, here's something you could do that then the next thing. Um, and I'm curious if you thought about how you might like, because obviously you could say

like, do these 3 things and that's all you ever do. Is it something that we should be looking at if we're creators? So we're creating instructional content. To like build this into, you know, almost templates that we could then repurpose? Or is that too structured and it's like everything is going to just— because a lot of times

it's like, well, it's so specific, it's going to just be different every time. But are there kind of general frameworks that you, you've built out or you think we could build out from, from these kind of this approach? Um, there are templates that, you know, there are structures for what good instructional design is. Saying, giving the model, then give examples, then give practices emerged from empirical research, and that serves as a good template. We want to avoid too

much templatization. I remember a company, uh, many years ago was following David Merrill's dick tips, and they created this very rigorous system that you talked about the type of objective you had, and then it limited the types of things you could do all the way through. And it made really rigorously accurate instruction, and you'd rather pull your eyes out than actually go through it, or That was unpleasant. So, um, but you have to be careful because some of the—

you know, I repaired my dryer with a video from YouTube. I have no idea what I had. It wasn't a learning experience, it was performance support. It led me through the steps and diagnostic and figure out what to do. And then— but I didn't— I haven't had to do that again for years and years. It would have been silly for me to learn anything from it. It

was just perfectly propor— sport. And you pointed out a bit of a, a evolutionary selection process going on, that the good videos are the ones that get viewpoints because there are people who naturally draw upon some of these principles, um, and, you know, that iterative cycle you were talking about. They're showing this step and talking about what leads to it, and in their narrative they talk a little bit about

the models that gave them the guidance. And Andrew Schoenfeld at Berkeley did some of the greatest stuff of talk— actually going down the way and talking about the thinking behind each step, which experts often don't have

access to. Um, cognitively, the research at the University of Southern California's Cognitive Technology Group by people like Richard Clark showed that 70% of what experts do, they don't have conscious access to, which is really problematic for instructional design because you've got these experts doing things and they don't even know what they're doing. They can tell you what they but they struggle to tell you what they do and you really have to work hard to pull it

out. These people, somehow, the best YouTube providers are managing to articulate the underlying thinking and then showing how it plays out in context, step by step by step through a process. And you may learn something from it if it is something you do frequently and you go back and view it several times each time until you've

sort of internalized it.. But we have to think a lot about what is the context, what is the need, how frequently— when you look at the principles of, you know, designing instruction, the criteria that determine how much practice you need and how much articulation tend to be, it's how inherently complex is it, how frequently you perform it in the real world, um, how important is it

if, if you get it wrong. That will determine a lot of these factors that are sort of not articulated explicitly in these YouTube videos and the selection between them, but end up playing a big role in whether you're talking about performance sport or act— you know, are you happy leaving the information in the world? Did you absolutely have to put something in the head? And if so, what and

how? Well, I, I appreciate that distinction because I think it is one that, uh, and it's not as nuanced, but it becomes kind of— is this learning? Is this performance support? And there is a lot of things that are just performance support, right? Just show me how to do it. Don't— I don't have to, I don't have to know it. I don't have to, you know, draw on that knowledge at a later point or build necessarily build

from that, that framework to understand this other framework. Um, so I think I want to call that out because I do think that is a really insightful point. And I, and I, I know the difference, but like hearing you say it's like, oh yeah. Duh, a lot of

this is not instruction. It's, it's really that. So, um, I'm, I'm curious because we've covered— I feel like we've covered a lot, but what else are we missing, or what else do we need to know to, again, to kind of keep us moving forward here with this understanding of the, you know, cognitive properties of, of, of using these things in a way, uh, because I, I do think we sometimes get into this cycle of it, particularly instructional design world where it's, you know,

there's an academic field for a reason. And it's— academics are good, I think, and we need those, we need that work, that research.

But oftentimes then, uh, you know, just in even my own experience, I'll be reading a paper and saying, okay, these are good things, and I— the application of it is like, well, I don't know, it doesn't maybe try to make the bridge the gap to like what I actually can do, or how would— how I do it, or even how my audience responds comparatively to the, the the study, you know, the body of the study of the people that

went through it. So is there other things we need to understand as about cognitive properties? Yes. So I, um, suggest that, uh, really learning science is, is rocket science. So the brain is arguably the most complex thing in the known universe, and trying to systematically get changes in it by, you know, you know, sort

of random perturbations is not the way to do it. We have The research you're pointing to, we have very good prescriptions in general that have teased out the importance of practice and the importance of deliberate practice and spacing of practice and a whole bunch of nuances. But putting it together for any specific circumstance that you're designing to support, there will— won't be one study that tells you how you should do that. What you're doing is putting

together different aspects. Which is why, by the way, you're starting to see also a move towards more iterative approaches. Um, so you need to create your best first guess, but then you should test it and say, oh, this part's working, that part's not, let's tune it a little bit. Okay, it's working better, let's do it, you know, but fine-tuning this thing, look, it's doing it. And you look at Michael Allen's SAM, Successive Approximation Model, or the Megan Torrance's LAMA,

lot like agile management approaches. They're moving to more create a first draft. Even David Merrill has moved from, uh, bone splay theory to ID2. Now his Pebble in a Pond is very much, you know, focus on the core practice first, get that right, to— testing and tuning, and then add other stuff around. So, you know, the point I'm trying to make is that research gives us good prescriptions. We need to apply it

in a creative way. Focus on engagement as well without violating the principles, and then test it and tune it to see what's happening. Um, because we're not like concrete, we don't have totally predictable properties. Our brains can even change, you know, a little bit of the Heisenberg uncertainty principle, right? Uh, when you're observing people doing things, it changes their understanding of what

they can do. Many people have created— built technology solutions and tested it and found that the people go, oh, well, now that I can see this, I want you to also do this and get rid of that. I don't need that now that you can do It's like, whoa, this is a dynamic process. That's cool, but it does require paying attention. Yeah, I, I feel like the, the pushback that then comes— because I, I completely agree, and I can remember being in my master's program, you know, working a lot of theory, a

lot of study of, of different models for learning and things like that. And of course, uh, you know, one of the— we had a journal we had to keep and we'd write ideas, and so Very— this great process, but I remember trying to create my own models, thinking, how am I gonna create my own models? You know, but I, I remember one thing that is, is common is that kind of that iteration, right? That there is, there is no real learning process without like getting better

at developing without iteration. And I think I wanna draw back to the, the conversation with the creators as well, because I think that's where they get us. They're really good at this. They, they are testing stuff all the time. Whereas I think in a corporate setting particularly, or even maybe in an educational setting,

it's really hard to do that. And I know just from my own experience, if I have to make a video or some kind of piece of media that's gonna go out, the, the, the cycle time for me to come back to iterate on that is probably not very quick. It might be a year, it might be 2 years. And so, you know, from a practicality standpoint, What advice would you give to folks who are saying like, yeah, I love this idea of iterating, but I can't iterate that fast because all— there's,

you know, 400 courses that they want. There's these pressures, that pressure, you know, it's just the reality of, of the work is that these things take time, cost money, things like that. So I'm not expecting like for you to have a silver bullet because I don't think there is one, but I'm curious, what, what advice do you give to folks like in that situation? I, I give a varied forms of

advice. The, the problem is, is we put ourselves in this position or allowed ourselves to be put in this position where this expectation where if we give you PowerPoints and PDFs, you can pop it in this program and turn it out on the web with a quiz and we're done. They have unrealistic expectations of what learning is. They have unrealistic expectations of what the process needs to be. And they have unrealistic expectations of what

learning can do. If we give people information, they'll change their behavior and we're done, right? Which empirically isn't true. We're not formological beings. Otherwise, if we get— got new information, we change our behavior, but we don't. It takes practice. It takes a lot of development. It takes getting rid of old things. We don't unlearn them. We learn over them. Uh, sorry, can't resist addressing a myth along the way, but we need to Uh, change the perception, and that's going to

be hard. So I've told people, you know, do a little bit of— it's easier to get forgiveness than permission. Focus most on making more meaningful practice. Just make a better written multiple choice question that's just a scenario, mini scenario, where there's a situation they have to make decisions instead of asking them to respond to pull out new information. We actually have evidence that, um, just pulling out information doesn't lead to behavior, but

actually making decisions. You don't even need— there was always this belief you needed to make sure you knew the information before you applied it. It turns out that requiring people just to make the decisions requires them to pull that information out of memory and supports the learning as well. So you only need the high-level questions. So it's, it's a dual

our, our multiple front attack. We need to change people's expectations, help them understand what learning is better, why we need to put less in the head and more in the world when we can. That's a much more effective solution. And we need to be subversively, um, creating, uh, learning that's more effective within the constraints we work in, but also start

measuring and showing that by making these changes, we're making a bigger impact. I know we resist evaluation a lot, and yet it's going to be the key towards

moving ourselves forwards. Well, I, I've got a quote for my, for my day: less in the head, more in the world. I love that. That is, uh, that is— that's, that's, that's a great statement. So, uh, no, thank you for that advice. And I think that, uh, what I think we'll, we're gonna— we'll end our formal conversation on that. I want to get to speed round here in a second, but I just want to say that I

love that, right? That there's this process that we've got it, we've got to just— you got to do it, ask for forgiveness and otherwise we'll never move forward and we keep— we'll keep delivering the same things. And I, I love that advice because I think it applies to this conversation. I think it applies to a lot of conversations I have around video and what's true about like how

people are making video, what types of videos they're making. So, uh, Clark, thank you for that good, good piece of advice. I, Appreciate it. No worries. Okay, so we're gonna, we're gonna jump into speed round questions. If you're new to the show, these are quick, uh, fast answers to questions that we decide by rolling a dice. So we're gonna play our stinger and we'll see you in

just a second. All right, Clark, here we go. We're gonna bring up our dice cam because that's right, we have a dice cam and we're gonna roll for— we got a 12-sided die, 12 different questions. And the first question is number 7. So here we go. Question