Welcome to Ed Tech insiders. In this podcast we talk to educators and educational technology investors, thought leaders, founders and operators about the most interesting and exciting trends in the field. I'm your host, Alex Sarlin, an educational technology veteran with over a decade of work at leading edtech companies. Jordan Adair is the Vice President of Product at honor lock, an online proctoring platform dedicated to maintaining academic integrity with humanity. Jordan began his career in education as an elementary and middle school teacher. After transitioning into educational technology. He became focused on delivering products designed to empower instructors, and improve the student experience. Jordan Adair, Welcome to EdTech insiders.
Thanks for having me, Alex, excited to be here.
Absolutely. So, Jordan, you have a really interesting history before moving to honor lock in 2019. I looked at exam soft, these are both assessment companies in the EdTech space, tell us a little bit about what drew you into ad tech, and to the assessment space specifically?
Sure. I consider my start in education actually beginning back when I was coaching, football and basketball at the high school level, I coach for about six years. And that's what actually got me into teaching, I was looking to kind of keep coaching because I really enjoyed it. But I needed a career that had hours that aligned so that I could actually be there after school. And there's not a whole lot of jobs that offer that outside of teaching. So I went ahead and got my teaching certificate in high school social studies is what I was looking to get into, turned out that the school I was coaching at all they had open was a fifth grade teaching spot, not exactly what I had in mind. But it turned out to be a ton of fun, I really liked the Compare and contrast between getting to teach the younger kids in the classroom. And then coach high school aged kids in the afternoon. So after teaching for about four years, I had kind of gotten a little bit burned out, I moved over to a charter school, we were sort of incentivized to teach to the test for the end of the year, because there was so much riding on it at the school level. And I have lost a little bit of the creativity and some of the stuff that I really enjoyed about teaching. So I was thinking, Alright, what could I add, I love the idea of education, I love the idea of being able to make a difference in the students lives. So what could I do, if I'm not going to be teaching that could keep me in that realm. And Ed Tech seemed like a logical steps, I ended up getting really lucky in that a teacher who was teaching in the classroom next to me, she ended up getting a job at ExamSoft, she kind of opened the door for me put a good word in. And that's what sort of allowed me to get my, my foot in there and get started. And then it was just the right place right time, they were building a product team for the first time, kind of transitioning from old school waterfall methodology for building software, to a more agile methodology. And they were really willing to let me learn through a lot of trial and error and experimenting. So I got to see the product team grow there and be a big part of that. And then one of the my colleagues came over to unlock kind of a, it made sense for me to make a shift over and try to build things here from the ground up as well. So it was a odd course. But I'm definitely glad that I ended up where I'm at.
Yeah, actually, I don't think it is that on an of course, I think that's a fantastic, you know, we have a lot of educators who listen to the podcast, who are some of them are a little burned out on the classroom, or they're just thinking about what their career trajectory, and they often do want to move into ad tech. So that's when you talk about, you know, moving through a colleague from the classroom to to an ad tech company, and then moving up in, you know, up within it and moving into product within it. I think that's a really inspiring story for many of our listeners,
well, I definitely would say that the reach that you can obtain in tech far exceeds what you can have in the classroom, right? I mean, you don't have the same one to one connection with the students, which I do miss to some extent, I actually still do some training from a sports perspective on the side, just because I like that seeing the impact on the actual individual. So you might miss out a bit on that. But knowing that you know, a product or a thing that you're building is being used by hundreds of 1000s millions of students, that is a good thing to the EdTech space where the sky's the limit, in a sense in terms of what type of impact you can make on the industry as a whole not just an individual students lives but you know, many, many more.
Yes, absolutely. And that's what I love about a tech as well. You never get back that one on one thrill of helping an individual unless you're actually you know, teaching situate It shouldn't, but it is a different kind of thrill and helping many people. So speaking of millions of pounds, let's talk a little bit about on our lock on unlocking. This is an assessment proctoring platform that uses an interesting combination of artificial intelligence, AI, and human intelligence to ensure that students are working with anti academic integrity. Tell us a little bit about this model and how the AI and human intelligence work together.
Yeah, so our, our approach has always been this hybrid model, where if you're relying solely on human Proctor's, like many in the proctoring space do that that works. But you're always going to be hamstrung by the elements of scheduling and scaling. If there's going to be a one to one match between a Proctor and a student or any ratio for that matter. You need to know how many Proctor's to have on staff and you need to be exact with that staffing, which means the students typically have to go through the process of scheduling their test so that the company can have Proctor's on staff or the school if you were doing something in person. So with this hybrid approach, it gives us a lot more flexibility and ability to scale up without the student needing to schedule their exam. Because the way it works is, as the student goes through the testing process, the AI is feeding elements to our Proctor's to then observe a student when it only when it's necessary, we don't need to have a human watching a student from beginning to end, because the AI is doing that. And then in turn, the AI feeding the human, we are always reliant on the human making a decision. A lot of people across the board, whether we're talking, testing or otherwise, are reluctant to give up decision making processes to AI. And no one would feel too great about thinking that an AI system is going to make a determination on if they're cheating on a test or not. And we certainly don't want to go down that path with our products. And in turn, our Proctor's are not actually making a determination on if someone is cheating or involved in academic dishonesty, we're just simply observing the situation, taking notes, sending those notes on to an instructor, and then letting that instructor do with it what they may. And oftentimes, we don't even know what that outcome is. So we don't necessarily get the feedback on what's done with the proctoring results. But the AI feeding the human is one of the big elements that we it really is the core of our business from a lot of perspectives, it helps us maintain that human element, while also giving the students just a ton of flexibility. You know, if you're working two jobs, and something happens, you shift gets changed. Well, now your scheduled exam is gone out the window, and you might have to reschedule for another day, another week. Maybe you're up against a deadline, and you can't reschedule all that stuff is irrelevant with this hybrid approach.
Yeah. So the AI allows you to scale and to remove some of the scheduling friction that comes with traditional online proctoring. That's human only. But the human element allows it to be not purely run by artificial intelligence, which is an important aspect of making it humane and logical and and not as scary to the students.
Yeah, absolutely. It's really the best of both worlds, right, leveraging technology to make a more flexible testing experience, while also keeping that human element so that we have can make judgment calls and can maintain that human touch and actually interact with a student, right, like one to one via chat versus having that all be automated or all the AI generated.
Yeah, that one on one approach, I can imagine would be an interesting experience. I want to ask you about that a little bit later. But you know, just just to keep a wide angle lens for one more moment, the online proctoring space for assessments may not be the most familiar space. For those who haven't come into contact with that that part of the EdTech industry. It's pretty specific. Obviously, over the last years, and especially during the pandemic, there's been an enormous shift to digital learning. And that includes digital testing, instead of in person testing. Give us a little bit of an overview of the online proctoring space. Is it used more in higher ed or high schools, professional exams for adults, all three and how has it expanded during these pandemic years?
Well, our focus is definitely on really two major areas, the higher ed market that it makes up the majority of our existing clients. And then we've recently expanded into the corporate space really focusing on certification exams, there seems to be a growing demand for the online proctored certification tests. I think some of that is tied to COVID forcing that to go online and then companies sort of seeing a lot of the benefits of that and now wanting to kind of keep the ball rolling in that regard. And honestly, we saw sort of some of the same things on the higher ed front. It's really just the last couple of years have kick started or sped up a trend that was already in motion. A lot of schools that maybe had online programs or you know, about a third of their students were at least enrolled in one online course. So they were using some type of online proctoring solution. Now that more students were exposed to that. And not by choice they had to because everything was moved online, they've seen Oh, this is actually a pretty flexible way for me to, you know, take a class that was scheduled at a time I didn't want to show up for or a more flexible way for me to test that doesn't require scheduling or showing up in person. And so students are sort of saying, even if they're not going to transition from in person to being fully online, more open, and more seeing some of the benefits of taking at least a couple classes online. So we're certainly seeing that trend. Continue, like some examples. We had schools that as they started to move back and bring students on campus, there aren't campus Testing Center was not necessarily staffing backed up to full capacity. They were using those in person Proctor's to help facilitate online testing as well, because the students were still preferring that method if they were offering both. So we definitely have seen, you know, the trend speed up over the past couple of years. And I suspect that that will just stick and continue to expand as time goes on, because of the flexibility. And how much more convenient the process is for a student to test online versus heading in in person to a testing center. And we do have some K 12 clients as well. But ourselves as a company, I don't want to speak for the entire market as a whole. But we are primarily serving higher ed in the corporate space.
It's been an interesting through line of I think a lot of different areas in ed tech that the pandemic, as you say it accelerated trends that we're already moving in a certain direction. But then once everybody sort of tipped into it, and really experienced a different type of, of learning experience, like online testing, and proctoring, or being able to test without scheduling or going to a center, the flexibility and an ease of it became pretty appealing. And it's interesting to hear that that is exactly what you've been seeing in the proctoring space that schools may not be re staffing their in person testing centers, or that students are preferring online testing, because it has benefits that they may not have expected if they were given the choice, you know, five years ago.
Yeah, absolutely. And I don't think that in person education is going to go anywhere, right? I mean, there's a lot of benefits to that to the connection that you make with your classmates and instructors. But there's always that one off odd credit that you need, that's, you know, at an inconvenient time. Or maybe there's a class that your university doesn't offer that you're interested in taking somewhere else, right. So really, this the hybrid approach of in person and online classes, I think that that's kind of where we're at and where things are going to stay for the foreseeable future in higher ed,
I had a friend in college who didn't graduate for I think two or three years it was one was one credit or one class short for years. And I would imagine that the type of flexibility we have now might have really made a difference for her. The tagline for honor lock, I find really interesting. I call this the academic integrity space, sometimes online proctoring and it says, online proctoring with integrity and humanity. And we talked about this a little but I wanted to double click on the idea that you know why honor lock features, its humanity as one of the main differentiators in the space?
Well, one of the things that we've seen is across the market, there's a really big push in kind of like a lot of companies are looking at, alright, how can we charge a few cents less per exam? How can we make our AI 1% better, so we can catch one out of every 10,000 students more. And that's sort of where a lot of companies that focus their efforts. We didn't want to compete in that same game. And so we sort of focused on like you're saying the human side of things, right? We all of our life, Proctor's are our employees based in the US, whereas a lot of other proctoring companies outsource that. And some of that has to do with our hybrid approach and the fact that we don't require a scheduling, right, so we can manage it with a smaller staff. But what that really allows us to do is we can maintain a much higher standard of service with our Proctor's and with our support team, for that matter, and it really has proven to make such a huge difference, like we've heard from a lot of students that have used other solutions and have used honor lock and the way that they've been treated by our Proctor's always sort of stands out above the crowd, and they've mentioned how helpful our people feel our Proctor's have been for them and how the entire The entire approach felt as if we were there to help versus there to catch. And that's really is what we try to pride ourselves on as a company is we want to be a deterrent, we want to help the students stay on track. We don't want to catch students in academic dishonesty, if we, you know, we will if we need to, because that's certainly what school is paying us to do. But we know that the majority of students out there are honest, they're just trying to take their exams as seamlessly as possible. And that's who we want to help do that, right. And we don't want to be a burden on them or any type of additional element of anxiety or things of that nature. So that human touch is key for us, and something that we will always maintain. We've even gone so far as to have you know, our Proctor's go through some training with a student counselor to help our Proctor's understand Alright, how do we de escalate a situation if a student who's stressed out or upset is maybe being belligerent or pushing back on some of the things the Proctor's asking them to do. And we have some other plans for further training on that front, too. But that's the type of stuff that we really take to heart and focus our time and energy on
sounds like you're balancing support of the idea of the humanities, you're balancing the integrity, the ability to be sure that students aren't cheating on exams in various ways, but with not overstepping, so that students feel over monitored or overly anxious or sort of have academic performance issues based on the intensity of the proctoring. So it should be humane proctoring with a human behind it, but that still is accurate enough to catch legitimate cheating.
Absolutely, yes. 100%. And if there's nothing, a proctor approaching a student in an aggressive way, or in any way, that's not helpful, it serves no one, right, whether the student is attempting academic dishonesty or not, like I mentioned a bit earlier, we are not the ones making that determination. We're simply the record keepers, right? We're going to see what we're going to record what was seen by our Proctor's and send that on to an instructor. So because that's our role, we really, there's no other way for us to be other than helpful, right? Because that's really our task is trying to get the student back on track, record what we saw, send it on to the instructor and then let the instructor and student work out what happens from there, because that's the relationship that's should be dealing with that whereas we are simply like a source of information. So I think because of the role we play, it's in a lot of people don't understand that, right? Students don't necessarily know upfront unless it's been communicated to them. Clearly, that honor lock can't fail me during my test or honor locks, not going to be able to end my exam, right? And that's certainly not the position that we're in are the things that we can do during a test.
Yeah, maybe a feature a byproduct of the fact that digital proctoring and online test taking is such a new experience in general for people, you know, I wanted to dig down You've mentioned a couple of times the idea of students helping no one if a proctor is aggressive, or students feeling, you know, potential anxiety. And in your work at honor luck, you've completed a study recently about the student experience with online proctoring. And I think specifically trying to look at students emotional reactions to this and how you can make sure that you are supporting tell us a little bit about that study and some of the insights that you gather.
So we ran a study last year with partner one of our clients. And we ran a study over a little bit over 200 exam sessions, we followed students through two high stakes tests, basically a midterm and final exam. And what we were trying to figure out is, what elements of online proctoring are contributing to student anxiety. And what we learned throughout the process, we This consisted of a survey, we use what's called the West Side test anxiety scale, which had been used in some previous test anxiety studies. We paired that with some proctoring specific questions. And then we followed that up with a focus group with some of the students that had responded so you can really dig in and understand the context of their answers around anxiety. So we discovered some interesting things. Number one, students are going to be anxious, regardless of how well prepared they are for the exam. That was one thing that was clear pretty well across the board. That makes sense, no matter how well you know you studied, you want to make sure you remember what you've studied, and you're going to be a little bit anxious. So knowing that the last thing students were looking for is some other elements that's now going to add on top of their already existing anxiety. But the study showed that online proctoring can contribute in a negative way to that anxiety and in specifically three main areas. The students were feeling worried about technical issues that they would not have to have dealt with in person setting so nervous about their internet connection going out. Is their webcam going to work and their microphone do they have you know, their A browser is their operating system up to date, all of those things that come along with using any type of app. So students were concerned with that students were also they had a lot of misconceptions about AI, they were expecting and thinking that the AI number one played a little bit of a bigger role than what it does in the in our hybrid model. It didn't quite understand how the AI feeds into the human proctor. But they also thought that the AI would be suspicious of activities that it's not so as an example, students reference things like dogs barking cats, meowing dishes clattering in the kitchen, coughing, sneezing, they were worried that all of these things would trigger flags that would make them look suspicious to their instructor. And our AI does not flag any of those things, right, because there, none of them are indicative of any type of academic dishonesty. So we don't care about those things. There was a student who had mentioned that they had their entire family with I think a dog and a cat and a bird kind of rent out an Airbnb around the block because they were so nervous about any type of background noise, causing a flag or making them look suspicious. So obviously, that student was going through all this trouble to arrange the you know, Airbnb and get their family out of the house. And that's just distracting them from the test. And they were sort of sidetracked with all of these other things that they didn't need to be concerned about if they would have understood how the system worked. And then the third big thing was the Proctor and wondering how the proctor was going to treat them. I mentioned a bit before some students had experiences with other proctoring solutions. So they had already preconceived notions of how this might go. A lot of those experiences were not positive students that have never used an on online proctoring solution. They didn't really know what to expect, you know, is the proctor going to be on video talking to me chatting with me? How is that interaction even going to take place? And then is the proctor going to feel like someone who is there to catch me? Are they going to be nitpicking me on every little behavior, right. And so these were all the things that the students didn't quite understand that we're driving them a higher level of anxiety on these proctored tests.
That's really fascinating. So so it sounds like they were worried about both the both sides of the hybrid model in some ways, they were worried about the artificial intelligence, flagging false positives, or sort of noticing something random in their apartment, or their their house or apartment like a dog barking and then thinking they were cheating. They were also worried potentially about the human element about the, you know, a mean, or aggressive proctor. And then the sort of underlying technical concerns. And I think it speaks volumes that that, you know, from a sort of Ed Tech ethics perspective that as an online proctoring solution, you really went to the you know, went to the students and heard from from the horse's mouth, so to speak, whether your solution was actually increasing anxiety or, and increasing tests and finding to some extent, people were, there were some issues there. So I think that's really interesting. Just forgive me a slight, slight tangent, but it reminds me of a couple of interesting stories about the past. When new technologies came out, my father worked on one of the first generations of ATMs. And when ATMs first came out, as you can imagine, people were very distrustful of them. They said, You know, I'm handing my money into a machine, a robot in a wall that is taking it and what would happen if it just ate it, you know, what would happen if it miscounted it? So there was a lot of this sort of anxiety about the new technology. And I even think way back to the beginning of when the first movie projectors came, there's a famous story about one of the first movies is showing train coming at the screen. And you had the you know, the people in the in the audience, you know, dive for cover, because they just didn't even understand what was going on. And it feels in some ways, like, this is a little bit of a version of that, you know, the idea of an AI and a human proctor watching you in your house while you take a test. People have all sorts of their imagination goes a little wild about what that might mean. What do you think?
Yeah, I think you're right. I think that there definitely is. We're at the very beginning of people adapting to this new method of testing. And really what was driving us right is we wanted to do this study so that we could figure out how can we improve our product to address the things that might be contributing to anxiety? And really what we ended up another element that we discovered was, I mentioned that we had done the study over two exams. So familiarity with honor lock, went a long way in reducing anxiety on future tests. So it was once students got that first experience out of the way, and they saw okay, this was not what this isn't what I was expecting it to be. It wasn't as scary as I thought it was going to be. The technology was easy to use. The proctor was friendly to me. There was no interruptions during my test because of AI right now. They their anxiety was reduced in those future tests. So that kind of, you know, informs us on how we can build some better onboarding tools for students a better tutorial experience to introduce them to all of these elements a little bit earlier in the process, so that by the time they get to their first high stakes test, they're more than familiar and comfortable with honor lock, and they'd had all of these concerns addressed.
Yeah, that makes a lot of sense that sort of film familiarity and positive experiences with the technology and with the system, make it feel more trustworthy over time. So Jordan, as a classroom teacher, I wonder if there any lessons that you learn from that past life that you apply to your role as a VP of product,
there most certainly is, first off the connection with your students also relates to what as a product leader, you need to connect with your users, right and draw out the problems they're trying to solve and help them solve them in a different way. But that connection is certainly transferable. And then, on top of that, you are the user of so many edtech products as a teacher, you're seeing what's great about them, and you're seeing the flaws. And when you make a transition from that into the product world, that goes a long way. Because you you now have that empathy for the users because you were one. And you also really understand the use case at a deeper level that no one's ever going to understand just your interviews or market research. So absolutely, yeah, there's there's a lot that you can transfer over from that real world experience. And it helps you build better products.
I'm curious about you mentioned the sort of people were worried about both the aggression of a possible aggression of a proctor. And about the AI sort of noticing things that weren't real. Did you find that if they did have an interaction with a proctor who and sort of got to know that it's a normal person and see how they react if something does happen, that that they that their anxiety dissipated in that way?
So when we talk with the students in the focus group, there were a pretty substantial number of those students that had an interaction with a proctor. And a lot of the majority of what our Proctor's need to do are usually small corrections, right? Like, hey, we noticed there's a textbook behind you, can you please put that book away on a shelf or something along those lines, right, the students not actually actively trying to cheat when the student when the proctor intervenes. So in those scenarios, when a student had these interactions with the Proctor, they said that from that point forward immediately on that test, like their anxiety entirely dissipated, because now they say they saw, okay, this is our last year to help because now this would go out the window, right, if a student was truly trying to cheat, obviously, their interaction with a proctor would not be as positive in their mind. For those that are just having the the run of the mill small corrective actions to get them back on track, they reflect positively and felt that their anxiety was drastically decreased from that point forward. So what's that that's told us that we have to try to introduce the proctoring interaction earlier in the process, right, during your practice exam, or during some type of onboarding? Let the let the students sort of meet the proctor whether it's a true meeting, or some type of simulated meeting, let them see, alright, this is how the proctor is going to speak to me, this is the tone they're going to use. And the way that we handle that is so much different than what they were accustomed to, you know, they had referenced Proctor's in the past that were on video, or even over voice chat, or whether or whether it be through a type chat that had been felt like they were trying to meet a quota, one of the students had mentioned that it was a quote from this study, you know, felt like they were purely there to police, things of that nature. So us taking that different approach, and the students kind of seeing that, that went a long way, and having a positive impact on you know, anxiety reduction,
that's very, really, really interesting. You know, I mean, they have this term, which I'm sure you are very, very familiar with, but maybe some of our listeners may not be aware, some artificial intelligence models keep what they call a human in the loop. Meaning that, you know, it's not entirely run by artificial intelligence, much of it is but at some point, you need a person to come validate and that's obviously exactly what honor locks model is. But it sounds like what you're saying is that in a human in the loop model like this, if the human is actually upfront if they meet the human and see that there's a human there, and not only you know, not only very very deep in the process, but see me them relatively early in an innocuous way. There are some positive psychological benefits.
Absolutely. And and it's really about that transparency, right helping the student understand better the process, meet the human understand what the AI is doing. Our our numbers show that only about 8% of students that are using our lock take apart Have this test. And some of that is to do with maybe a student not choosing to take it or in other cases and instructor not making it available to them. So knowing that if there's 90 plus percent of the students walking into their first exam, pretty well blind to what this system is going to be like and how the experience is going to go, that obviously it's not a good thing. Right. And that's part of the improvements I was referring to you that we're trying to build. But yeah, that transparency and familiarity is certainly the key element that we've sort of discovered through that that research project,
really interesting user research. And, you know, as the VP of product, you are, you get to think about how to interpret those psychological findings and transform them into meaningful product features. As you mentioned, it could be onboarding, practice testing, transparency, about the process, I want to ask you a sort of future oriented question, which is, you know, as you mentioned, test anxiety is very, very common, whether online or in person. And it's been common for a very long time. There's been a lot of studies about it, they sometimes they show that female students have more testing anxiety, there's been a number of studies that show that some studies show that students from underrepresented backgrounds might have more testing anxiety, if they have what they call stereotype threat, if they're sort of here a negative stereotype, do you foresee a future where online proctoring could actually go all the way towards reducing testing site and evening the playing field? Even if right now, is it because it's so unfamiliar, it might enhance it? I would
like to think so that we're certainly trying to move in that direction and sort of have that impact ourselves. Right. So one of the things that we're intending to do is, after we make some of these product improvements that we're working on, is to rerun this study on somewhat of a larger scale and see if the new onboarding experience that we've built, has made the impact on anxiety we're expecting, and we do expect that it will reduce it from a baseline level, whether or not we can succeed. We'll we'll see. But that's certainly our objective is and I do think that across the board, right? In the scenarios you described, if what the what the students really want to feel is that they are on level ground, regardless of demographics, background income, any of those things, right. And if an online proctoring tool, or hopefully we're the ones to do it on your lock is able to give the students that confidence that their experience is exactly the same as this other student, regardless of any of those elements you added, then I think it is possible to you know, reduce that test anxiety and create that level playing field. And that's certainly what we're striving for.
Yeah, I think that's a that's an exciting vision that would be and understanding going and doing that qualitative research in the focus groups. And the the studies that you've already been doing is I think the first step towards setting up a baseline and understanding, where's the anxiety level now? What causes it, and how could we reduce it. And I'd love to see a world where people don't feel as anxious about tests, because because it does affect academic performance as well. And it's something that scares many, many, many students, especially high stakes exams. To me, one of the most interesting parts of the honor lock model is how it really does use this sort of two layer approach this human in the loop model. And, you know, I was looking into some of the history of this model, one of the really interesting uses of it was was one of the most famous AI cases, which is, you know, when Garry Kasparov, the chess player, lost to Deep Blue, the IBM AI algorithm machine. And basically, he wanted to dig deeper into it. And he ended up finding that when you put a human and an artificial intelligence model together, they would be the artificial intelligence model. So that even though AI could be the best human humans plus AI could be the best AI. And researchers have talked about this being like, because humans and machines excel in different dimensions of intelligence, they do different things. Well, so you found a really interesting use case for this, which is that AI is is very good at watching endless hours of people taking tests and noticing certain things that might indicate cheating, I imagine that's books, cell phones, various things. I mean, we won't we won't speculate here. But at the end of the day, you need a person to look in and see if there's something really happening there. I'm curious how you see this, this human in the loop model, playing out in the EdTech space, it could be in proctoring, or in general, because it's a really interesting idea to have AI and human intelligence is working together and in a complementary way, and I think it's also a lot less scary than than a fully AI future. What are your thoughts about it?
So I think that in our case, creating kind of this feedback loop right, this system of AI Feeding a proctor, a proctor recording what they've observed feeding that to an instructor and then eventually getting that instructor to feed back to us what their opinion was of what was discovered, right, this is sort of the ultimate loop that would then feed back into our AI and improve the accuracy there as well. And then this would be an ever evolving, improving process. So it's certainly a part of our business model that we don't intend to have that go anywhere, right. It's, it's vital to what we're doing. And I think that, I don't imagine that in education, people are going to fully want to move away from human interaction, right? Like this is the entire thing that education is built on is the instructor and the student relationship, having some form of connection, knowing your students so that you kind of understand the nuances of a situation, right. And that could be that that certainly can fit the mold from a testing perspective, if you know, as a student, and they have a couple kids at home, you know that there's going to be background noise, there might be a kid that runs in the room, like these are things that you know, based on your relationship with that person. So having that human as somewhere in the process it right, not just relying solely on a computer to make determinations is very important, I think in in the world of online proctoring and education as a whole. So I don't think it's going anywhere. I think that figuring out the right balance between the AI and the human is the key. And that's kind of the the thing that we're going to consistently strive to improve on ourselves and sort of what I see happening across the market as a whole to
Yeah, that's a terrific answer. I love I love your emphasis on personal relationships. I think every your your former teacher, I think anybody who has been involved in education, has has seen that personal relationship, that actually knowing students knowing who they are, what they're about why they are there, knowing what their life is like, is a huge part of our education. It's not It's it matters a lot. So that keeping that humanity in is incredibly important. And I think it's really exciting to hear that an online proctoring company, which you may think of as being very sort of a, as you mentioned, sort of a cold catching machine or policing system is really not thinking about it that way at all thinking about it much more as a system that can that can maintain some of the real positive relationships, even between something fleeting like a proctor, and a student take a you know, college student taking a test. It's a really interesting vision. I'm glad to hear that this humanity and this type of research is being done. So we're coming on time, Jordan, but this has been a really fascinating interview, I wanted to ask our final questions. The first is, what is the most exciting trend you see in the EdTech landscape right now from your vantage point that you think our listeners should probably keep an eye on?
So I'm gonna cheat a little bit on the answer. I actually don't think this is a trend that has necessarily caught on in ed tech. I think it is an educational trend that Ed Tech is maybe yet to catch up on. And that's the the focus on how to, and this is this has been a growing trend, I think year after year and COVID. And moving online accelerated this as well. Is the authentic assessment trend. How do we better assess the students? That's not just solely reliant on a multiple choice true false test? How can we make that scale in the online world? Right? If you have a class of 20 students, it's very easy to have students create 20 projects, and you can grade them. If you're talking about a massive online class with hundreds and hundreds of students. That's not so easy. How do you know that the students that are doing this work? Are the actual students enrolled in your class? How do you even go about grading assignments at that scale? Right? So I think there's a lot of really interesting innovations that can happen to help support that movement. And I think that educators all across the landscape, whether it be K 12 or higher ed, are slowly but surely buying more and more into the fact that okay, there is there, there must be a place for these alternative styles of exams for us to really understand if a student has learned what we've taught them. It cannot just be memorization or traditional testing. So that's kind of something that I personally am interested to see what type of new innovations might pop up to help facilitate that type of assessment at scale?
That's a terrific answer. 100% agree, I think that feeds into best practices of instructional design. And I think you're right i It's always been a little bit surprising to me that in many cases, the authentic task that students are being asked to master is something they would be doing on the same computer that they're doing to to access the LMS So to actually access an exam, so you would think that the idea of popping up a real software tool and having you having students perform within the software tool and not being an authentic assessment would be something we would have cracked by now, but I, I've seen small glimmers of it in certain places, but there's definitely no real breakthrough. I think that's a terrific answer. So And our last question is, what is one book or blog, it could be Twitter feed, and maybe a newsletter that you would recommend for somebody who wants to dive deeper into any of the topics we discussed today.
So we actually took our test anxiety study and sort of converted it into an e book that I would definitely recommend for anyone who wants to know the details of that study and outcomes. Easiest way to find it is you just search on our lock student test anxiety, and it'll be the top three results that you'll see in Google. But that really dives into the details of what questions we asked the percentage and decrease in anxiety that we saw from the first exam to the second, write all the nitty gritty details of this study. For those that want to dive a little bit deeper. They're
fantastic. And we will as always include that the link to the study in the show notes for this week's episode, so you can find it there or follow the link that Jordan just named Jordan Adair. This has been really, really interesting. I think you've given all of us a lot a me certainly me and I'm sure many of our listeners are really interesting insight into the mindset and ethos of the online proctoring world in 2022. And I really appreciate it. Thanks for being here. Absolutely, Alex, my pleasure. Thanks for listening to this episode of the EdTech insiders podcast. If you liked the episode, remember to subscribe on Spotify, Stitcher or wherever you get your podcasts. And if you're listening on Apple, please leave a rating and review so others can find the podcast. For more ed tech insiders content subscribe to the Ed Tech insiders newsletter at edtech insiders.substack.com