[Music]
[Music]
Hello everyone I am Raquel Bertiz from the NLN Center for Innovation in Education Excellence. And today I'm very excited to chat with a fellow nurse educator and literally a fellow from our 2022 NLN Sim Lead fellowship program and let me introduce Dr. Laura Klenke-Borgmann. She's currently the director of the University of Kansas School of Nursing Simulation Center and she is a clinical associate professor of the same program
as well. She had published and presented various works in nursing education such as a clinical judgment, nurse educator competencies, and currently also a several projects on competency based education. But I really admire Laura's fashion and work in simulation education. More recently, the work that was acknowledged by the Clinical Simulation in Nursing journal as article of the year in 2023 was that work on transferability of clinical judgments learned via simulations
in the classroom to clinicals. Yeah, that's right. The transability of that to their actual clinical practice. Right. So there is so much to talk about in nursing education and simulations today but I am very interested to learn and I'm sure many of you are as well, our listeners, about initiatives in operationalizing competency based education in curriculum and nursing programs such as the one Laura and I will talk about today. So Laura,
welcome. Thanks Raquel. Always good to chat and catch up with you as my fellow fellow. Right. Yeah. So yeah. So the nursing education has so many questions on how to implement competency based education and of course the 2021 AACN Essentials so I would start with with these questions. So what projects or initiatives have you implemented in your nursing program and what were your reasons for doing so. Well, other than we know it's the 2021. Yeah. Other than we
need to. Right. We need to do. Right. Yeah, well thank you so much and I like I said, I really appreciate being here. I could talk forever on these topics as you know as well. So one of the things to your question that we really have been working on and implementing as a new initiative here at the university, the KU School of Nursing, is we did see the need to increase the presence of competency based education and competency based
evaluation within our simulation curriculum. And again, like you said, all based on that 2021 AACN Essentials and saw specifically the SimZones approach as the perfect theoretical framework in
which to do it. So last spring, spring of 24 ,we implemented for the very first time in our curriculum and specifically in our sim curriculum, the SimZone's approach to competency based evaluation through an SOCE, so an objective structured clinical examination for our third year students at the end of the, at the end of the semester, at the end of the year. So I hear a SimZone's approach here and many of our listeners may know it or may not know it, but
I'm curious about it. So can you talk a little bit more about the SimZones framework? Yes, absolutely. So originally the SimZones approach was published by Roussin and Weinstock and then more recently it's been kind of modified for specifically the
nursing curriculum and education. And basically it is a framework that we chose because when mapped together, SimZones and actually if people are familiar with Miller's pyramid of competency based evaluation, it's a really strong scaffold for what authentic learning looks like, specifically in a longitudinal simulation curriculum. So basically, to evaluate what learners can do, not necessarily what educators teach them or tell them, right.
So the SimZones system is an organization for longitudinal simulation based learning and it's comprised of basically five zones, in which the authors talked about how there's specific learning goals in each zone. And as I mentioned, additionally, those zones were mapped to Miller's pyramid for competency assessment. So you start at the bottom of the pyramid or bottom of the SimZones and you just worry about what your students know
right. So basically requisite knowledge - what do learners need to know from didactic courses, from their previous Knowledge from readings, to be able to then elevate them through the next zones. So zone zero basically of the SimZones talks about that of just what requisite knowledge do they need to know. Then after they know, you kind of take it one step up into zone one and
that's basically what they know how to do. So through the lens of SimZones and through the lens of sim, that's talking about things like you know repetitive deliberate practice of just like the psychomotor skills. So once they have that requisite knowledge and zone zero you can move them through the next zone of repetitive deliberate practice. Then once they sort of have been in that zone for a while you can move them to
zone two, which is more contextual learning. So now we can actually kind of start to throw them into simulation based experiences because they have that other knowledge from the other zones. But you can still kind of stop and start them and provide feedback along the way and that's kind of the contextual learning that happens in zone two.
In zone three that's what we think of the type of learning that happens in a simulation where they start the simulation, they do the entire simulation and then they stop and we debrief them, so there's not that stopping and starting. This is that traditional, when you think of sim, you know, stopping and not stopping until the simulation is over. And then the last zone, zone four is basically taking all of that knowledge that you've accumulated throughout those zones and
being able to actually do in practice. And so this is actually like evaluation that would happen in the clinical practice setting, in the wild so to speak. So for this project that we did last spring, we specifically used principles of backward design through the lens of those SimZones that I just described to ensure that our curriculum
met the desire outcomes for our students. So our focal point was that individual OSCE, a comprehensive evaluation to that we use to assess the clinical competence of our learners, but worked backward from that through backward design throughout to create an entire semester long curriculum that moved them through those SimZones to ensure that they were prepared to be successful in that end
of semester OSCE. Yeah so there's a lot of elements of competency based education to digest from what you just said, so you definitely mentioned backward design, which is of course, you know, is almost synonymous with competency based education. But I hear some elements there as well. So in terms of what you said about OSCE, so I'm thinking, hmm, formative assessments, summative evaluations, right, and of course the simulations are teaching
and learning modalities as well, right. So would you want to expound on how you did your OSCEs, right. Yeah! OSCEs, what were they for and what came afterwards. Yeah, absolutely, for sure. So to your point of talking about, you know, we kind of covered the topic backward design already, but I would like to kind of piggy back onto what you talked about regarding like formative and summative evaluation throughout this
process. So as I mentioned, our initial step was kind of a thorough analysis of zone zero, basically we assessed our students baseline knowledge and did a preliminary analysis that revealed gaps which kind of told us how to tailor our instructional sessions accordingly. And next a curricular road map was essentially created that enabled our learners to achieve the necessary competencies that we wanted them to be able to do by that end of semester OSCE.
So the road map basically included preparatory lab sessions that aligned with each SimZone framework zone for our learners. So for example, we held an OSCE prep lab one toward the beginning of the semester that we sort of envisioned as being in that zone
one of our SimZone. So that was held one month before the summit of OSCE and it was a two-hour session that incorporated some small group practice where students focused on deliberate practice of some of the psychomotor skills that were going to be required later in the semester in the OSCE. So specifically things like primary and secondary IV tubing setup, doing a physical assessment, doing ISBAR and a handoff, communicating with the healthcare team, doing safe IV medication
administration. And in these two-hour sessions, in this OSCE prep lab one, in zone one, learners were able to do deliberate practice of those things and received real time facilitator and peer feedback throughout that session. Then two weeks after OSCE prep lab one, students returned back to the lab for what we referred to as OSCE prep lab 2. So again we envisioned OSCE prep lab 2 occurring
in that zone two of the SimZones. So this time, this two-hour session for the learners allowed them to engage in small group practice scenarios designed to mimic what the final OSCE simulation was going to look like. So students were allowed to kind of pause and stop and ask questions of
the facilitators along the way. And again, the simulation that they actually participated in was very close to the one they were going to do for their more summative evaluation, but it allowed them to stop and start and continue to get feedback, so again that contextual learning in
SimZone 2. And then what we referred to as zone three involved the actual summative measurement of those nursing competencies through that individual simulation that each student did at the end of the semester for the actual OSCE and every student took care basically of the same patient. It was a post-operative patient experiencing some post-op complications. So does that kind of answer your question about how we moved through those zones
to prep them for that final evaluation? Yeah, it
makes sense, right. So if you have the SimZones as your frame for moving your students through like maybe zero or no knowledge to eventually, oh at the end of this time frame, whatever that time frame was they need to be able to do all of these things, which of course goes back to our prior conversation of backwards design, where while in the beginning we need to know where they would be at the end of a certain time frame, which traditionally for many of us it's a semester
or a trimester, half a semester, whatever that is right. So right. Right and yeah. And yeah interesting. So and then I would like to circle back to your reasons or your aims for this final project. Yeah, yeah so essentially we had two major aims
for this pilot project. One the first aims was to evaluate the individual clinical competencies of these third-year pre-licensure baccalaureate nursing students who were going to be in a primary nurse role while caring for this post-op patient. So we as the faculty involved created a faculty- designed checklist where the students either did or did not complete observable tasks that we were going to be looking for and evaluating throughout
that OSCE. And it was the expectation that the learners demonstrated competence for every line item on that evaluation checklist to be determined quote unquote competent in the OSCE itself. So to be clear, the final version of this checklist that
we designed had four subscales. So really we were looking specifically at behaviors that would fall into categories of patient safety, clinical judgment, physical assessment and communication, with obviously required observable behaviors in each one of them. And so measurement we for every one of these students they came in and did a 35 minute simulation caring for this patient for which we measured then these competencies on this checklist. And this is this was the interesting
part, Raquel. So performance was determined as either competent or developing competency so to be in alignment and use that same language as the AACN is putting out for their competencies. And if students were deemed still developing competency this was not a good/bad, pass/fail kind of evaluation. This was just, okay, great, you know
you're still developing competency. So then if that was the case, they were brought back to the lab one week later for additional deliberate practice and then did another summative OSCE and this process, sort of an iterative process, was repeated on a weekly basis as many times as necessary until competency was reached for all of the learners. And in total this process was completed over a
three-week time frame. So we were able to get 107 of our learners through this iterative process of you know doing the OSCE, coming back if they were still developing competency, working with them a little bit more and then having them do another OSCE and it it took three weeks, so not as long as maybe we thought it maybe would take. That, that's
amazing right. So and I hear the words like all students and like individual students and I also hear certain activities on the end of the faculty in terms of the creating the road map, designing the checklist, evaluation checklist, determining behaviors. That sounded a lot of work and so therefore and of course like oh three weeks right, oh were you able to do all of that in three weeks! So I'm curious to know, how did you get this to work? So your faculty for example, I'm curious
about that. Yeah, sure, absolutely. So 100% it was a lot of work and a lot of work to make sure that we did the process right from the very beginning right so that we were setting ourselves up to be successful all along the way. And we definitely gained valuable insight into not only the challenges that we as faculty faced but challenges that student faced with competency
based evaluations along the way as well. So one of the things I will tell you is that very quickly in the process the faculty recognize the importance of transparency regarding the evaluation process for our learners. You know, so we made sure from the very beginning that we provided students with a really clear description of how evaluations would be conducted. We covered that checklist
that I just told you about that we designed. We covered the checklist criteria with the students from the beginning. I think a lot of us in education sometimes are a little worried about sort of quote unquote teaching to the test but I think transparency and competency based evaluation is actually really, really important and we were transparent with the students at the very beginning of the whole process from the first
day of one of those OSCE prep lab ones. We provided the students the evaluation checklist that they were going to be evaluated on at the end so they had that checklist through all of those prep labs and all of their learning so that they were, it was very transparent to them what they were going to be evaluated on and what competencies we were specifically looking at. We learned that was really
important. We also provided timelines for feedback to students and reinforced that student data would be used to develop personalized learning plans for them and this message was repeatedly communicated to students throughout the entire semester so that's the one thing I can share with you that we as faculty learned really quickly of how important transparency is all along the way for learners in
this new methodology. The other thing we learned really quickly was unlike traditional, like as I kind of as I mentioned before, like unlike traditional pass/fail or A-F grading systems, competency evaluations were a new concept for many students. You think about the way they've learned since kindergarten, right. Good/bad, pass/fail,
get a 97% on your test. This shift and assessment methodology was met with certainly a range of emotional responses from students and I imagine and one of the ways we sort of came to know this was they also had to once they were deemed competent in their evaluation we had them watch the recording of themselves doing the OSCE
and fill out a guided written reflection. And in that written reflection we heard a lot of comments from students including stress, obviously anxiety, uncertainty, with this type of evaluation process because as many time even though we told them over and over again throughout this process this is not good/bad, pass/fail this is either your competent or your developing competency and we'll bring you back they still thought in their head when they
were told on the day of evaluation that okay great, you're still developing competency several of them still saw it as, oh I failed, and that was a really hard hurdle to overcome with their level of sort of understanding of competency evaluation. So that was really important to us as faculty
as a learning piece as well. And so I'll say despite the faculty efforts to create a really supportive environment, many students still did equate developing competency as like receiving a poor letter grade and that wasn't the case at all, right. So that was one thing that, that's a big thing that we really learned moving forward is how important that language is for this shift in methodology of how students will be evaluated.
Yeah, I can imagine if our students had gone through years and years of our traditional education and suddenly we're shifting them to this new way of well of course evaluation is is the biggest I would say component there and I am used to getting a grade, like what you said. Yeah. Or pass/fail and I can sleep at night knowing I pass, but then right what do I do if you tell
me I'm still developing competency. And of course I can't imagine the cascade of activities that would happen afterwards to get to my competencies, get competence, but the first thing I would say is the acceptance of, what do you mean I am not competent yet? Right. Yeah. So I kind of like get what you're saying there in terms of, oh, let's
not forget our students here. Right. Yeah. We're all very excited about developing our road maps and curriculum and yes we have a nice well-written program here but what about our learners right. But okay, so let's circle back to your faculty though and I heard earlier that you said that the faculty understood the importance of this process. Yeah. So what about the learning curve
of faculty, if there is such right. Yeah. Yeah. No, absolutely, and you know I will definitely call out my team that I did this project with. They were fantastic. We were, you know, always in tight communication. Communication and collaboration throughout this process was huge. And I think it's in beyond just your team of faculty. You have to think about the
faculty at large as well, right. And so you know historically OSCE were oftentimes thought to be these very like high stakes pass or fail type of competency evaluations. Exactly. Because that's what we went through. That's right, right. And so I guess my question to that and my little
push back to that is - do they have to be, right? To better capture the essence of competency based evaluation I think we in nursing education should maybe consider reimagining quote unquote what an OSCE is, right, to include iterative coaching and several opportunities for learner success like
the approach that we did in this project. And so I think that's one of the things for faculty development at large, not just obviously here at my institution but at large is it'll also take a little bit of retraining the way we think about some of the ways that we evaluate
in the past right. Right. And to our listeners I would like to kind of like point out what you said in terms of transparency of learning and how it goes in contrast to our traditional belief that we should not be teaching to that
test. Right. Which is why not. Right, exactly. So if we're testing them we want them to be able to pass the test, but of course not to take it literally to teach them what's in the test, but what should we be thinking about as educators for our students to get there successfully get there and and then of course, again, I'm not sure if the is the professional culture in nursing and of course I digress a little bit, but the anxiety and fear of learning which occurs so often. Yeah.
Right. Right. In our, I would say, roles as faculty and future faculty development initiatives, I think we should also be taking into consideration a kind of like reflection at our own culture as teachers, educators, and faculty and I think that's a huge element of initiating any change, which we often forget because we dive in, right, into oh this is what needs to be done, but like we leave ourselves alone for introspection which
is necessary. We would like to be conscious of our time boundaries here and I know that this conversation can go on and on and it will. We will end this part one now with the conversation of CBE and simulation based education. Thank you for your time and we hope to see you in our next episode. Bye for now and see you in part two. [Music]