Episode 575: Feedback and Focus

CAROL SUBIÑO SULLIVAN: Yeah, so we call it an early classroom feedback session. And those sessions have two parts. The first part is where I would go into the classroom room and sit down and observe what is happening in the class. I take a look at what the instructor is doing. And I also am paying attention to what the students are doing. And I'm making notes about what I observe.

And I should back up and say that even before I sit in the classroom, I actually meet with the instructor at least once or twice before I go in to get an understanding of what the course is, how the instructor has chosen to structure it, what their instructional goals are, and any of their observations about the classroom dynamic. I might also take a look at materials, like the syllabus.

I might take a look at the Canvas page and see the ways that students are being set up for interaction with the content, with the instructor, and with each other. So when I step into the classroom and I'm getting to observe it myself, I don't presume that what I'm seeing on that particular day is everything there is to know about a class. A class goes on for many weeks. And what happens on any particular day, you know, it could just be a random thing that's-- something happens.

And things maybe don't go the way that an instructor wishes on a day that they being observed. And so I never assume that what I'm seeing that day is the whole story. And I do work to get the context. Yet, coming in person is important. It lets me see the instructor in action, see the classroom culture that has developed. And I'm able to provide some feedback. I like to use a tool for recording those observations. It's called the COPUS, which is something like the Classroom Observation Protocol.

It's a tool that lets me structure my observations of what's happening in the classroom. So for example, if there's a period of time where the instructor is lecturing and the students are listening and taking notes, I can observe that. Or if it transitions into small group activity, I can make note of that. And it's broken out into timestamps.

So at the end of the observation, I can tell the instructor, It looks like you have spent about half of your allotted class time doing lecture, using slides. And then you allocated about 15 minutes to interactive exercises. And then there was another 10 minutes of administrative work. And sometimes that breakdown is helpful for an instructor and can also help guide the type of feedback that I provide.

So since this is not like a prescribed thing-- it's not like some administrative policy at Georgia Tech says, hey, everyone needs to have their class reviewed-- you're kind of relying on instructors to think, Oh, you know what, I could use some help from the Center for Teaching and Learning. So what do you find is the motivation, the most common motivation, of folks that teach to come to you? Is it because they want to-- are they afraid that their class is going downhill?

Or is it a moment of crisis?

But before I do, I think it's really important that I finish the description of what an early course feedback session is because the observation is just the first part. The second part is actually a focus group that I run with the students.

So I ask the instructors to allocate the last 20 minutes of that class session that I come to observe to leave the room, leave me in charge of the class, and then I will organize a discussion with the students to find out their perceptions of what is helping them learn the most in the class. So what is in place? What is the instructor doing on a day-to-day basis?

How have they set up their Canvas course-- all kinds of things-- and then the other side of that, which is, what ideas do they have for changes that could be made that would improve their learning experience? They meet in small groups and discuss those questions. I then ask them to share the ideas. And I find out how much consensus there is. That's a helpful data point for instructors.

Sometimes when you don't get that consensus building, like if they just get the end of semester evaluations and they get a random comment from one student, they have no idea. Do all the students feel like this? Or was it just an outlier? And so this consensus process gives them that information. And then we end by-- I ask the students, Given everything that you've just seen and heard from our discussion, what's the most important thing that you want your instructor to know?

And so the two parts, the observation and the focus group, make up the early course feedback session. I take all of that information. I write up the report. And then I meet again with the instructor afterwards, not only to share the insights from the process but also to have a brainstorming conversation with them about how they would like to respond to that student feedback. Sometimes there's feedback that's easy to respond to, an easy thing to change.

And the instructor can happily do that because they can see that it's going to make a difference to their students. Sometimes what the students are asking is more involved. And so that's a great opportunity for the instructor to express appreciation to the students for those perspectives and tell them their plans of how they might want to implement that in a future semester. And then sometimes what the students are suggesting is not something that the instructor wants to do.

And they have usually thought carefully about that choice. And that's another great opportunity to invite students into the thought process that goes behind teaching because many students have never had the opportunity to teach. So they might not have ever thought about it from the instructor's perspective.

CHARLIE BENNETT: You are listening to an interview with Dr. Carol Subiño Sullivan, assistant director of Faculty Teaching and Learning at Georgia Tech's Center for Teaching and Learning, also known as CTL. We'll be back with more about early classroom feedback sessions after a music set. And you can file this set under PA3015.V84.

What's that experience like for you? CAROL SUBIÑO SULLIVAN: Well, I can understand that feeling of trepidation and not knowing what they're going to say.

What not just my experience, but what the research on early course feedback sessions has shown is that that feeling of being valued and being heard-- just that, in and of itself-- even apart from whatever feedback the students might share, improves the dynamic of the course because suddenly the students are getting this signal, My instructor is making the space to really hear me and is doing it in a way where I don't feel like if I do have a critical perspective to share,

I don't feel like my grade is going to be in jeopardy because they're bringing in this third party. I don't know their names. I don't record any names when I'm doing the feedback. Everything in the report is anonymous. So there's no way for the inherent power dynamic that happens when you have a faculty member who is grading a set of students, that doesn't come into play. So students feel safe to say what they need to say. And they also feel like they are going to be heard.

It's really not about points on your end of semester evaluations. But we do know that those are still used in important ways in higher education. And there has been a documented positive impact on end of semester evaluations when faculty do take the time to do these early course feedback sessions.

Have you ever encountered a situation where there's been a real-- I don't know if "disconnect" is the right word-- or if it's been real hard work for the instructor who maybe thinks one way about how a class was going and then gets the feedback, and that feedback is much different than the instructor's initial mindset? CAROL SUBIÑO SULLIVAN: Certainly. I honor the teaching decisions. It's an incredible amount of thought that goes into putting a course together and a teaching approach.

And I work with many instructors who are so incredibly thoughtful.

When the students have all kinds of suggestions based on their experience and they don't have that perspective, it can be difficult to reconcile the two, which is why having the separate meeting before the instructor goes back and discusses the results with the students-- I think, actually, that's the most vulnerable part of the whole process, is when the instructor goes back to their students and says, This is what I heard you say. And this is what I'm going to do.

And so all of the things that lead up to that help to support and prepare the instructor to be able to do that really important, vulnerable piece of it. I think what helps is that when I have those conversations with instructors, I am trying to honor all of the voices in the space, so the instructor voice and the students. So when a student says, Please change this, I'm not like automatically on the side of the student. I'm never blaming the instructor.

All I'm doing is trying to help figure out-- help the instructor, really, figure out, what is going on that might have led the students to make this suggestion? If that change isn't what you need to make exactly, you know, what might be the underlying dynamic? What is something that can happen so that it doesn't violate these intentional choices that you've made? And yet, can we meet the students where they are? What is it that they really need in this situation?

I would say, overall, these early course feedback sessions and all of the components are some of the most positive experiences that I've had as an educational developer in the time I have been. Almost always, we're able to find a concrete way to move forward. And maybe it's not-- it doesn't turn things around completely in that very semester, but it kind of gets things changing. I'm not going to reveal any details or names because these consultations are absolutely confidential.

But I have had experiences where the feedback from the students indicated that there were some major issues in the course that needed to change. And by this process of dialogue and conversation and listening across all parties over the course of a semester or two, the instructional team was able to make changes. And I ran into some of the instructors a while later and said, We worked together all these years ago, and I had never gotten a teaching award.

And now, suddenly, I'm getting teaching awards. And my students are totally saying-- they have very different reactions to the experience of going into the course. So it's a powerful experience. I have seen it be able to make a positive change for the instructors, for the students. And even though it does-- I won't pretend that it doesn't require vulnerability. It's something that is so valuable.

And I hope that the instructors feel so supported throughout the whole process that I hope it isn't a deterrent, that little bit of fear that might be along with it.

Go ahead and hit that feedback generator, Fred.

Certainly, the observation part of it will be helpful, getting some student perspectives. I would need to frame it differently because I couldn't promise the students you're going to see a change by the end of the course. That's the big difference. But you might have the same students that take multiple workshops. They could get to experience an improvement or a change that they would find helpful by the end.

I would still consider it to be a formative feedback process in the case of providing feedback on somebody who's teaching standalone workshops rather than an integrated course. The part we would need to think carefully about-- and I would invite the instructor into that conversation-- is what would be the best way to gather the student perspectives? That 20 minutes in a single class session, when it's happening in the course of a semester, things can get moved around.

But when it's in a single workshop, the workshop is all there is. So maybe it would be a focus group with students who have attended library workshops that's happening at a different time outside of the workshop. I don't know. We'd have to think creatively about how to gather student perspectives. I think it would be something that we could collaborate to create.

So it is the opportunity to hear students perspectives. It is a snapshot in time. We generally recommend between weeks 4 and 6 in a typical length semester, so before the first half, in order to give enough time for the classroom culture to settle, but not so late that changes real changes can't be implemented. That's what makes it formative, right? We're getting these perspectives in, and the intention is to use them to iterate on what's going on in the classroom.

These reports, I write them, and I give them to the instructor, and nobody else sees them. The instructor can do whatever they want with them. Some instructors have elected to use them in their annual reviews or send them to administrators who were asking about their teaching. But I am never going to do that. An important role of the Center for Teaching and Learning is to be a space that instructors see as welcoming and accessible. And so we're not going to engage in the process of evaluation.

We're not going to engage in that process of judgment. We really are in the business of trying to improve teaching and learning to support student success. And then my other question was the use of the word "early." Is that referring to the time in the semester when it takes place? Or is it referring to where the person is in their career or some other aspect of the word "early?" CAROL SUBIÑO SULLIVAN: Yeah, it's about the point in the semester.

And the contrast is to the tradition and the institutionalization of the teaching. In this case, I will use evaluations that are administered officially at the very end of the term, when the course is over. And so the early course feedback means we're not waiting till the end of the term. Anybody can request one of these at any point in their career. And I am delighted whenever senior faculty or people who've been teaching for a very long time reach out to us and request one of these.

I think no matter how many years you've been teaching in your field, it's always a new set of students. Staying in tune with student perspectives keeps the teaching alive, whether you're in just in your first year or whether you've been teaching for decades.

This is what you could do different. This is what you should keep doing. But you're sort of a go-between for the students and the instructors so that they have a better relationship after this process. CAROL SUBIÑO SULLIVAN: Yes, absolutely, it is sort of like a go-between. [LAUGHS] I think that's what you said. You were breaking up a little bit. So I couldn't understand exactly. But even if-- and I do sometimes have instructors that come and say, I want the feedback.

I just can't spare the 20 minutes. This time, can you come in and just do the observation? Even in that case, I still would never say, Here's what I saw, and here's what you should do. It's always a conversation. My goal is to really understand, what is the instructional goal? And how does this approach to teaching help the instructional goal, help the students be successful? In the conversation, as it's appropriate, I will say, Well, this is what we know.

This is the research I'm aware of that has looked at this particular approach to teaching, this particular set of students, and bring that information into the conversation. But at the end of the day, I'm supporting the instructor in being able to make the decisions that are going to be best for them. And there is no one single right way to teach.

It's always an intricate decision-making process about the instructor themselves, the content that they need to help the students master, the set of students, the context of the classroom, of the online portions of it. All of it is sort of a delicate balance that informs each other. And that's what ultimately shapes what the right decision is going to be for this particular case. So we're almost out of time.

And just from what we've talked about, it's clear that there can be no sort of summing up advice to instructors. But I wonder, what have you learned or been able to do with your own instruction after years of early classroom feedback? CAROL SUBIÑO SULLIVAN: Well, so I'll say that I'm always trying something new.

So even for as long as I've been teaching and I learn all these things about teaching, I don't like to share these ideas without having any first-hand experience of trying things out myself in a real teaching setting and working with students. And I have to say that if there is something that I have learned it's that I like teaching that way.

I like always being open to trying something a little bit differently than I did before, to reflect on the process, reflect on what I observe in the students, and using those reflections to guide me in what I might try out.

So for example, the last time that I taught my course in the honors college, that I mentioned before, for the first time, I decided to apply principles of-- I don't want to start throwing in terms that I have to explain-- but give some more choices in order to make the course more inclusive. So I had never assigned a set of readings, for example, where the students could just choose the one that they were most interested in.

Or I would like throw in some that were text and some that were audio and some that were video. And students could make some selections according to their interests, their learning needs. And I wove that all into the classroom discussion. And it was something that really enriched the learning environment. And so now when I talk with faculty about inclusive teaching approaches, I can share some of my own experiences as well as share evidence from the teaching literature.

Our guest today was Dr. Carol Subiño Sullivan, the assistant director of Faculty Teaching and Learning at Georgia Tech's Center for Teaching and Learning.

Our web page is Library.GATech.e du/LostInTheStacks, where you'll find our most recent episode, a link to our podcast feed, and a web form if you want to get in touch with us. CHARLIE BENNETT: Next week, we're going to talk about banned books. It turns out, that particular ridiculousness is coming back in a big way, like belly shirts. Oy vey. Time for our last song today.

As instructors, it leaves us a little vulnerable to let outsiders come in and tell us how we can change our classroom strategy.