Welcome to this episode of the NLN podcast Nursing Edge Unscripted. I am the host of today's episode. I'm Raquel Bertiz and co-hosting with me is Dr. Kellie Bryant and we're both from the National League for Nursing. In this episode we will discuss generative AI in teaching and learning, specifically ChatGPT. Our guest today is Dr.
Rachel Cox Simms. She is an assistant professor at the MGH Institute for Health Professions School of Nursing in Boston and she had completed interesting projects in AI published them also in various peer reviewed journals. She is also completing her PhD in health professions education at the same institution. I therefore welcome Rachel or Dr. Cox Simms. Thank you for joining us. Thank you both for having me today.
So excited to talk about those are really exciting topics something near and dear to my heart and my everyday life. So I'm very excited to discuss this with you both. Okay. Yeah. And I cannot wait to get this conversation going. So let me ask you the first question. So what research projects have you done in AI and what led you to do this projects? I'm curious about that. Great question. So let's talk about what led me here first. So honestly,
what led me here was I felt like survival. I was pregnant, exhausted, and trying to figure out how to streamline my academic life and my career before a little tiny human arrived and took over my life. I have a brother who's a computer scientist very excited about AI. He like sent me a message saying "This is really cool, check it out." So I started testing AI tools to see if there's anything I could use to help lighten my load or be more creative and just take some things
off my plate. And as it turns out, ChatGPT was a fantastic tool. I didn't need sleep or snacks or maternity leave. So it was a win-win. It was a very helpful tool. It helped me balance a lot that I had going on at the time and it was really exciting to see everything it could do. So that's how I started with it is just sheer need and then I kind of ran with it. It started with doing some research studies and comparing the quality of NCLEX questions created by ChatGPT with those
created by human experts educators. And from there moved on to designing assignments that work with AI instead of against it. Think things like case studies quizzes, tasks where we can use ChatGPT to actually improve learning rather than shortcut it. So think about how we can use its weaknesses, the fact that it makes things up. It's not a perfect tool. Use it to help our students learn and think critically and then I've got lots of other things
going on in class. Things like custom GPTs for for my pharmacology course and comparing different AI platforms. So definitely a lot of irons in the fire right now. Yeah, I actually saw a most recent article of yours and it's really the call for AI literacy for nursing educators or in general nursing education. So I was thinking that's kind of like, oh, pushing this forward. So what was that all about if you can like expand a little bit
on that the call? Absolutely. So I think that AI is going to become more prevalent in education as time goes on. It's going to be I think baked in to some degree as like an assistant, something like autocorrect, something we use without a lot of thought. However, as these tools become more prevalent we have to retain our critical thinking clinical judgment. I think these tools are going
to be integrated into clinical practice. We're already seeing them being used in hospital settings to help take notes, coding, billing things like that. So our students are going to be stepping into a world a future where health care and AI are completely intertwined. So I believe that generative AI literacy for both faculty and students isn't just helpful, it's crucial. We want to prepare students not just for exams but for the like health care system that's evolving
in real time. Our students need the skills and the critical thinking not just to use these tools but to critically examine them their outputs and use them safely ethically. And it starts with us as educators making sure that we understand even if we don't love it. Not everybody has to love it. Not everybody has to use it. But knowing it's good and bad and making sure we explain that to students in context is really important.
That is great. And again, I also read quite a few of your articles and I found them really, really insightful because they're, like you said, there's a lot of educators out there who, you know, you have the gamut. You have some who are novices who have never used it or a little bit worried about using it and you have some people who just dive right in and pushing it to the limits and seeing you know what other ways it can make their life a lot easier, more efficient.
You just mentioned though that it's not perfect. And I was wondering if you can expand on what are some of the challenges and and obstacles that you see when it comes to faculty incorporating AI into the nursing curriculum? Absolutely. So like any tool there are pros and cons. Like any medication, I teach pharmacology, I'm always talking about risks and benefits. And this is a tool just like anything else. There are risks and
benefits. One of the big risks right now is that AI produces factually incorrect information and it does so very confidently. It can fool even if you're not being careful. It can fool you into believing something that is absolutely not true. But it sounds really good. It sounds great. So we have to be aware of the fact that these tools can provide unreliable, inaccurate, or even biased information. So it's our job as faculty to use
our expertise. We know a lot and it's our job to use our expertise to carefully look at what AI creates. Keep what's good, reject what's bad, and then use that to help students. One of my favorite things to do is have students use AI to generate practice questions. I give them some guidelines on how to create them and then I make them factcheck those practice questions they create. Is it a good question? Is it like something you'd see on the
test? Was the answer correct? Was the rationale right? And then I ask them to site their source from the book or my slides. And then at the end, just to make sure they did it correctly, I have them submit their transcript. ChatGPT can allow students to basically copy their entire conversation and share it with me. So I can look to make sure that they're using it correctly and wisely and give them advice to guide them towards
better use. So right now it's not a perfect tool, but I like to harness its weakness as a learning strength for students. Not only am I teaching them about AI and how to use it safely and wisely, I'm strengthening their critical thinking skills having them look at text look for bias look for factual inaccuracies and then correct it. Think bigger than just memorizing. We're not just memorizing wrote memorization. We're thinking
about things. And I think it's going to help potentially make students stronger test takers if they understand even the questions better. So a lot of opportunity there. Yeah. So I think that you really kind of like touched on several challenges there and at the same time advantages of utilizing the technology that we have right now, which is AI. So for example, I heard you say oh, it's going to make them stronger test takers.
So and I know that you mentioned earlier that you did a particular study on writing test questions. So can you kind of like share a little bit of like what surprised you if any from that study because I found that very interesting. Absolutely. So one of one thing that students like a lot, for anybody who teaches in nursing education, you
know students love practice questions. They're always asking for more and more practice questions because that's a great way to learn how to take a test. And I'm always coming up short. I never feel like I have enough practice questions. I started using ChatGPT to create custom practice questions for students based on things they were struggling with. And I found they were actually really good,
a really great jumping off point. So I teamed up with some other faculty at the school at the IHP and what we did is we did a comparative study. We used ChatGPT to create practice questions and then we paired them with human generated questions .So same topic different questions and we coupled them together. We then sent out these questions to faculty really all over New England.
It was a New England-based study and we had them evaluate these two questions not knowing that we're comparing AI or human, but we wanted to know did you like the questions, how clear were they, correct were the distractors good, was there any bias? So we had them evaluate all of these questions, 20 in total, and then we compared the human the scores on the human-based questions with those created by the AI. Interestingly enough, we found that the scores were almost identical
or very statistically similar. We found that the questions created by AI did not necessarily score better or worse than those created by humans, but we did find that there was a slight preference towards more of the AI questions than the humans. So several of the AI questions were preferred, but we did find that the human created questions were maybe a little bit harder, but the hard the degree of difficulty and the preference were not
the same. So it's not like we preferred the harder over the easier or anything like that. So we found that the questions were pretty similar. And this was back in 2023 with ChatGPT's original model. So things, a lot has changed since then. It's been a few years, but we found that even the more primitive ChatGPT was able to create some really great questions. It's a lot of opportunity for
us as educators. We know how hard it is to write a good question, to write lots of good questions over and over and over again. And we care about integrity of our exams. We don't want to always be reusing questions. So we see this as a potential avenue to help stimulate ideas for questions, generate new questions, mix it up on your exams. You know, make sure that we're always giving our students up-to-date fresh questions and
keeping them challenged and engaged. And I can imagine our learners are writing exams for themselves as well, right, if they learn how to do prompts or prompt engineering of their own learning. And I think that's great, but that brings back the point of you to say human oversight over the teaching and learning processes and if nurse educators, human nurse educators
have been creating more difficult questions. So that brings me back to how ... how will we make sure that these questions actually align to the level of learning that we would like to have our students achieve? Although, like at this point, like you said, we're now at 4.5 or or even higher, maybe ChatGPT can do that also. So to me, that's really just interesting to see all of these changes happen over time. Absolutely I think there's a lot of avenue for future research
discovery comparisons. There's other platforms now other than just ChatGPT. There are a lot of really other great platforms who are probably capable of creating other great questions as well. I think we're going to probably look towards a future where maybe an AI is even customized towards health care nursing even creating and learning and training on content specific and relevant to
us. Maybe that could even enhance our ability to create resources, materials, assignments for our students to help them, you know, learn in this new AI-driven world. Because one of the concerns for nurse educators is that students are going to use AI for everything and it's going to decrease their critical thinking skills and that you know if we give them simple questions such as fill-in- the blank or multiple choice they're just going to put in a ChatGPT and get the answer
and not really. you know. think about the answer themselves and just kind of use it to spit out the answer. So you brought up a good point and a great example of how you can as a nurse educator use generative AI to create assignments where you're embracing using generative AI and because it's
not going anywhere. So I'm just curious if there are any other examples of assignments that nurse educators can adopt that will help the student number one teach them the AI literacy skills that they need and competencies but also create a meaningful assignment like the one. the example that you gave of writing NCLEX. Do you have any other examples for our nurse educators out there and how they can use generative AI in their
assessments and their assignments? Absolutely. So another really great opportunity for learning and exploration of AI is having AI generate patient cases like case studies, so having students create case studies using generative AI. You know, have them try to produce the best case study they can using iterative process to have them refine and go back and edit their case study using generative
AI. Always having them though site their sources using the book, lecture slides, whatever resource you deem most appropriate for your class. Make sure students are citing their work anyways but have them create case studies, unfolding case studies, even ones that require them to think a little further ahead, something further than AI could easily do. It could definitely be an
assistant. I like to think of AI in this case as almost like a scaffold of learning, something to support students as they think and grow and learn, giving them a little bit of backup until they know everything or know as much as they can and create great case studies. So using it as an assistant to create great case studies, helping them think more about a clinical setting and about their patients in in real time. Also having students
interact with the AI like it's a patient. So ask the AI to be a patient and roleplay with the AI and ultimately have them share their transcript with you from the AI so you can look and see how they're using it and provide them feedback. One of I think the most important things though is to ask students to provide a reflection at the end. Have them think reflectively about what they
did, what they liked, what they didn't like. Not something that's graded heavily but requires them to kind of meaningly think about what they did, what they liked about it, what was incorrect, what was correct and even have that, share that with their peers have a discussion about it. All of that's a great way to kind of use it to learn and use it to learn about AI at the same
time. Those are great examples. Thank you. I know and to me, like listening to how you design or integrate AI into those very specific assignments really reflect how you would still have to have reflections in there, the value of self-reflection perhaps, and then giving your feedback to the students is still an integral process of learning. And then looking at AI as the scaffold. Yeah I
I really like that example. So with with that said, I'm curious about how it changed your your teaching practices or even the learning patterns of your students and how long have you been doing this in your practice? That's a great question. So I have a lot of thoughts about it. So I teach pharmacology primarily, which is generally an exam-heavy course. Truthfully, even before A,I I didn't incorporate a lot of essays so there wasn't a huge impact on the types of assignments that I
gave. We still have in-person testing and I think that's really important, in person testing where I can view my students. I can see their screens. I know what's going on. I know they're not using AI. But for other assignments that I do in class, I tend to make them either low stakes or not graded so that there isn't as much incentive to try to use the AI to cheat. We're using it as a
learning experience. We're doing it together. We're trying to meaningfully engage with the material together. So there isn't that incentive to try to use that. I know this is a bigger challenge in courses that use more assessments and assignments that are essay based or they get sent home with assignments that they could easily cheat with or use generative AI unethically. And I think that's a, it's a, it's a big barrier and it's a, it's a problem that we're going to have to try to
mitigate as time goes on. We know that AI checkers like through Turnitin aren't always the most accurate. They can unfairly penalize especially students whose language, their first language isn't English. We know that these AI checkers can unfairly like mark them as AI created material. So
I think we're going to have to be creative. I think it's going to require us to push a lot of our work back in person on campus maybe even pencil and paper someday, which seems counterintuitive - more technology and we're going back to pencil and paper! But I do think that if we are going to be creating assessments and assignments that students could easily do with ChatGPT we're going to have to be creative about it. It's going to be
a challenge. Right and yes, so creativity and still like it brings me back to your point of human oversight, right. The expertise of the nurse educator there. Absolutely. Another quick thought is the idea of doing video assignments. So having students create recordings and videos. Much more
difficult to use an AI to complete that. So that's another way I've seen nurse educators do this is have, either give their assignments via video or have students create assignments that are videos which is a great way to see how your students are engaging with the material. Yeah. And we've seen several articles of how students or learners are responding to the use of AI or or their preferences. So based on your own experience, how are your learners accepting or reacting to AI
in their learning process. It's a great question. So surprisingly, a lot of students at least say that they don't know anything about AI or have not used it much yet, but for those who are using it are using it regularly. What I hear from students is that they are using it to help them create study guides to explain concepts that they didn't understand in class in a different way. You know, having AI kind of simplify things. Students use
it to create mnemonics. I know that's a popular way especially in a pharmacology class. Creating mnemonics is often really important. I also know that students are using it to create practice questions and you know other study suggestions to help them prepare for exams and interact and role play with like almost stimulated patients using AI. So there's definitely some positive experiences and it's interesting to see what creative things
students will do with it. I know they're going to do something great and interesting. For sure. There's a great article out there where it's defining how the students use AI. In some ways, I think they're probably using it more than we are as nurse educators. And all those wonderful ways that that you stated. I had a professor who had a student who did very well on one of the exams. And so the professor talked to the student say, "Wow there was a big jump from your last exam
and such an improvement in this exam. Like did you change your study habits?" And the answer was "I used ChatGPT to help develop test questions." And it proved to help because the student had a much higher grade and the professor was like, "Well that's great. You know if it worked and it helped you, you know, to get a higher grade, you know very supportive of using it." Absolutely. I
also have a student who reached out to me. She uses an AI called NotebookLM to take basically research articles and turn them into a podcast. So the AI reads the articles, summarizes it, and creates it in podcast form that she listens while she commutes to campus. So she found herself keeping up with a lot of new things through listening and then the AI was able to help streamline that for her, which I thought was really interesting and innovative.
We can learn from our students also. Right. So like if we are really looking ahead and kind of like be two steps ahead, not just a step ahead of the innovations that we're facing right now. It should be both ways right. So we're preparing ourselves and our students are on this trajectory as well. So with that said, I'm curious. So what directions do you see your works in AI going in the near future or even like maybe if you have a forecast
of what's to come with your AI projects? Well, for my work specifically, I've kind of actually gone back to some of the basics during my process of trying to assess the quality of NCLEX questions. I realized there wasn't like a great standard for equality. So I've gone back and I'm working with experts to create like what makes a really great
NCLEX question. That's the work I'm doing with my PhD. And then I hope to kind of expand and start looking at other AI platforms learning about which ones are best suited for the things that we need as nurse educators? Which ones are creating the most accurate medical information? which ones have the least amount of bias? So I'm really hoping to kind of dive more in to a lot of the different AI platforms, not just ChatGPT, which
I love, don't get me wrong. It's the fantastic, but I want to learn more about all the different options. What's the best and think about a future where maybe we create an AI that's maybe more custom to what we need. I'm working now on creating a custom GPT using ChatGPT for my pharmacology course, one that I've baked in all of my own materials, my own syllabus. I'm trying to create a resource for students that I have a lot more control over can
provide a lot more accurate information for them. It's something they can use to kind of supplement their learning in my course. So I think the future is probably going to involve more personalized learning but also expanding our horizons beyond just ChatGPT, looking at other opportunities to kind of grow in this area and expand our teaching materials and our opportunity
there. Yeah. And definitely evaluating the quality of this apps technology is one fertile ground for more exploration or investigation. Yeah, so I'm excited for you and you're in this path of AI and where it's going and yeah, we'll definitely be looking out for more articles from you in the future. So we're towards
the the end of our podcast here. And so for our final question, I would just like to kind of like to know if you have some lessons learned or some nuggets to share with our nursing educators out there. Absolutely I would encourage every nursing educator to try. Try ChatGPT if you haven't already. Just try it out. Play around with it. Spend an hour, book that hour in your day, block it off and spend some time with ChatGPT. Talk to it like
it was a person. Ask it to do various different things just to see if it can. Try not to be afraid. I know a lot of faculty are nervous and afraid and I think all of their concerns are valid, but not to be afraid when interacting. Ask it to do all different things. I'm always blown away by what it can do if I just ask. AI can mimic a lot. It can do so many different things. It creates all these different things. And I think there's a lot of
opportunity for creativity. I know nurse educators are creative people. I know nursing as a field always steps up historically. We're always evolving and I think there's a lot of really untapped potential there for us as nurse educators. So I'd say keep an open mind, be curious, not judgmental and can go in and try it out and see what you can do with it. Then share it with us because we're all excited to see. So I shared your findings! Yeah, so thank you very much
with that set of nuggets. There's, there were a lot of nuggets there and I know that Kellie, Dr. Bryant is also steeped into AI and ChatGPT. You've done so many presentations on this as well. So any last words for our educators also. Yeah, I just I just want to say I agree with Rachel. It's just ... you can't break it. Just the more that you use it and play with it the more comfortable you get and
you can push it to the limits. You can just play around and do things that aren't related to work. Tell it I have these ingredients in my cabinet. Help me make a recipe for tonight. So start with something innocent. But I know we're running out of time and I just wanted to say thank you Rachel.
You really gave us a lot of pearls of wisdom and that you know there's so many things to look forward to in the future with AI, but I love how you talked about all the benefits, but also talked about the things that we need to be aware of and and the limitations and some of the challenges. So I wish you the best of luck with your PhD also with your project. Thank you so much for having me. I really enjoyed this conversation with you both.
Yes. So thank you. Thank you Kellie, Dr. Bryant. Thank you Rachel. So that's the end of our podcast and until our next episode. Bye bye everyone.