Welcome to Ed Tech insiders where we speak with founders operators investors and thought leaders in the education technology industry and report on cutting edge news in this fast evolving field from around the globe. From AI to xr to K 12 to l&d, you'll find everything you need here on edtech insiders. And if you liked the podcast, please give us a rating and a review so others can find it more easily.
Christina Ishmael is the Deputy Director at the US Office of Educational Technology, which will call the Office of Ed Tech or the OE T, part of the US Department of Education. She is an educator, a learner, an advocate and an agent of change. As the leader of the Office of edtech. She executes the office mission of developing national edtech policy that enables everywhere all the time learning, and supports digital
equity and opportunity. Before the OE T Christina was a Senior Research Fellow on new America's Teaching Learning and tech team, where she advised on the inclusive technology portfolio and incorporated digital equity and culturally responsive and sustaining education. Christina was also the Digital Learning Specialist for the Nebraska Department of Education, where she led professional learning and advocated for school
librarians. This experience granted her a unique perspective on leveraging edtech in urban, suburban and rural school systems. Christina started her education career as an early childhood and elementary teacher of emerging bilingual students in Omaha, Nebraska, Christina Ismail, welcome to Ed Tech insiders.
Thanks so much for having me. I'm happy to be here.
I'm really happy to be speaking to you today. I just saw you speak at the ASU GSB conference and I've been following your work for quite a while online. You've been an elementary school teacher and bilingual educator, digital learning specialist, consultant, Senior Research Fellow, all leading up to your current work at as the lead of the Office of Educational Technology at the US Department of Education deputy director I think is your official title.
Yeah, that's amazing. I think a lot of people listening to this probably don't know a whole lot about the Office of Educational Technology, even though they work in ad tech. Tell us a little bit about your journey from the classroom to the government in ed tech and how your vision for Ed Tech has guided your career. Why ed tech? Why the doe?
Yeah, wonderful. Well, thank you for highlighting all the things that I have done that have gotten me
here. So I did start out as an early childhood teacher teaching preschool and kindergarten in Omaha Public Schools, so Omaha, Nebraska, and then moved into a second grade bilingual position all second grade content in Spanish, and then went back into a regular secondary classroom moved districts and then I was in a smaller school district in Omaha called Ralston public schools where I was an ESL teacher or an English language learner teacher for grades
kindergarten through sixth. And I think it's important to note like throughout that whole process, while being in the classroom, I obviously saw how technology was being used. My first school was a magnet school. And so we had Spanish math and technology as far as our magnets were concerned. So not only did we have our regular general school budget, we also had our title one budget, because we were a title one school, then we had a magnet
budget on top of that. And so we had quite a bit of access to educational technology in our classroom, whether they were desktop computers at the time, because early 2000s. One of my first grants that I ever wrote for donors, twos was for iPod Touches, because everyone loved iPod Touches at the time, for small group instruction and all of that. And so and the majority of my students from the very beginning were multilingual
learners. And so I saw the power of technology in the hands of my multilingual learners, especially from the earliest ages, and how it not only helped for language acquisition, but for content acquisition. And so whether that was using, you know, an online based tool or the iPod Touches for various apps, while we would be doing literacy circles or guided math, then I saw that happening with my own students. And the same happened when I moved into that El position with multilingual
learners. And then I was fortunate enough to move up to the state level where I became the state ed tech director, the Digital Learning Specialist for the entire state of Nebraska, and had a unique perspective. I see I feel like moving from the classroom level. It's very myopic. I see what's right in
front of me. Then moving to the state level, I got up to more of like the 30,000 foot view and could see an entire state worth of infrastructure and professional learning and what was happening what was not happening kind of things and I did my my action research for my second master's on professional learning for Ed Tech, specifically across the state of
Nebraska. And there are between a state agency and a school district, there are these things called service agencies, there are 17 of them across the state of Nebraska. And the majority of them were designed to provide special education services over 50 years ago, and then they started adding on other pieces like professional learning, like, you know, kind of broadband and connectivity, things that they could do for the districts that they serve.
Anyway, they were the ones that helped execute the content standards that were set by the states. So they served as that intermediary down to the school district level. And were the ones that were mainly providing professional learning for Ed Tech. And so in my role at the state level, I was working internally to make sure that all of my colleagues understood the power of edtech in the hands of learners, especially as we worked with our various curriculum and instruction,
folks. But then the other half was going out and supporting
school districts. So 245 public school districts in Nebraska, 310,000 students, and urban, suburban and rural like, all three very, very easy to identify majority being rural, of course, with just a couple of them that would technically be considered an urban school system, and then the rest of them suburban, and rural, and then moved out to Washington, DC in 2016, where I was a fellow in this office, and got to see, again, another perspective at like the 100,000 foot view, and
be able to see how policy at that level kind of trickles down to the state, then down to the local and haven't really gone back. It's always been kind of at that national level, I moved on to a nonprofit here in DC called New America where I was doing work there. And then was part of the Biden Harris transition team and remembered having that like mission driven purpose build work, that I never thought that I would leave a classroom. And then here I am doing policy 15 years later,
it's It's wild. And I love what I do.
Well, I'm sure people will feel really excited to know that people who are making this type of federal policy know what it's like to be in the classroom have that. People don't expect that trajectory. And I think it's incredibly exciting. You know, we have listeners around the world, and some of them are not in K 12, or education at all, that some of them don't even work with higher ed, they might not be familiar how the US Department of Education works.
It's a really unusual and interesting place, or the Office of Educational Technology is within it. Give us a little bit of an overview of the of the Oh, et the Office of Educational Technology and how it fits into the broader mission. Yeah, that's great.
So we are somewhat unique in the sense for our Department of Education, there are kind of specific things that we can and cannot do at the federal level, which is different compared to other countries with ministries of education that have a much more centralized kind of approach. We believe in local control here
within the US. And so really, we act many times as almost kind of like the pass through or the conduit for policy, but also for funding, specifically when we're talking about education at that federal level that then goes to the state education agency or an SCA. And then the state are the ones that are determining state content standards and specifications around different programs. And they're also the ones that would then give the dollars to the school districts.
So you're thinking like, you know, the funnel all the way down, then to the actual school district from there, the Office of Ed Tech was actually established by Congress in 1996. So during the Clinton administration, and Congress thinking ahead, saying we know that technology is going to impact education, we should probably have some people that are addressing this. And so that's, that's when it came to be and we are congressionally mandated to exist, does not mean that I always have a budget or
staff. But it does mean that we are in charge of developing national ed tech policy for the entire country, and making sure that we are involved with the conversations around policy and practice, from the ground up. We have previously sat in a different kind of location within the organization. And now we sit in the office of planning, evaluation and policy
development. So all of the policies, whether it is in K 12, or p 12, I should say for early learning, as well as our higher education and Adult Education and kind of in the workforce. We're part of those conversations from the ground up so that we can make sure that all things edtech or digital learning are included or considered at the very least, as we move forward with executing on Secretary Cardona has raised
the bar initiative. So all of the kinds of things within that we want to make sure that Ed Tech and digital learning included.
Great explanation, I feel it's very clear. Okay, good. Oh, very much so and sometimes it's hard to follow the thread when it comes to government agencies. But that makes a lot of sense. Okay. So you mentioned that the OE T is from 1996. That's, you know, that's relatively recently, but at the same time, pretty, you know, a lot has changed in
edtech. Last year. So tell us a little bit about how the policy and the sort of philosophy that comes out of your office, the the fellows and some of the things you create, get from the federal level down to the state level, you mentioned, the SCA is and you know, some of the structure but I'm curious, almost on a on a conceptual level, how does it make its way from your office into the classroom? Yeah,
yep. So I joined the administration in October 2021. So, you know, good six, seven months after everyone had kind of come into the position within the new administration. I am a political or presidential appointee. So I serve at the pleasure of the President. I think that's also important to note, because there's a mix of folks in federal government that are career staff members. And then there are political appointees that are there for a specific
type of administration. And that's important to note, because we are constantly getting new ideas that come through the door as well. So I do like to start by saying that I have always believed in looking at our end users, whether I've never worked in the private sector, but I know enough about ed tech that they are trying to solve problems for
their end users. And always, you know, whenever we're thinking about design thinking, and always starting with the end user in mind, going through our empathy interviews, or whatever that case may be, how can we design more human centered policy. And so that is how I have always approached the work that I've done, but also coming from that very first hand experience as a classroom
teacher. The previous work that I did was more curriculum focused around open educational resources, and saying, I used to take the stuff in my classroom and customize it for the local context. Was I doing that? Fairly? Probably not like, did I know things about copyright and fair use at that time? No, I didn't, I was doing what was serving my students that were in my, you know, vicinity, and in my line of sight, to help them as far as their language and
content acquisition. And so I've always brought that to these policy conversations, and always remembering the former learners that I've had. And also thinking about the educators that I've worked with, and the range of abilities and skill sets and literacies. When it comes to using technology. There's always the folks that are like, Oh, I'll, I'll be an early adopter, I'll do it. That was me, I was the first one to like, sign up for anything. And then you've got the folks that are like, I'm
not touching that. And then we had a pandemic, and we didn't have a choice. And so I think that is certainly whether folks had a good or bad experience. But it certainly brought the whole thing around ed tech more to the forefront within education. And so we are seeing more people look to the Office of Ed Tech for more direction or more resources, or whatever the case may be. And so I feel like half of our work is more
internally focused. And so we are working actively, again, on all of the policy development, just working with other people in the agency. And then the other half is externally focused, where we get to work with learners and educators across the entire country, making sure that they see themselves in the policy that we are creating, and then hoping to push out as far as the practice is concerned.
That's where it's really interesting. You know, when we talk to some higher education, folks, they talk about how when the pandemic came down, suddenly the number of requests to the you know, IT department and the Centers for teaching and learning just shot through the roof and those those laggards, they call them, you know, late adopters, who didn't want to be using technology suddenly found themselves having to learn how to you know how to do a zoom classroom in a
breakout room. And I'm sure that you know, that happened throughout the entire country. One of the things that is the sort of epic, big question underlying all that tech, we talk about this a lot on the
podcast is, is it working? At the end of the day, and the OT recently released something really interesting, which is an edtech evidence Toolkit, which basically helps define and give really specific examples for the four tiers of evidence recognized by the Department of Education by Ed under the 1965, ESCA act, they have these ideas of what makes something effective, what sort of constitutes proof and to help schools and educators and I'm
sure SCA is guide decisions. So, really cool project we'll dig into a little bit but but just starting from the beginning, you know, what was the origin of this evidence toolkit project and what are you hoping that it'll do to support the education community throughout the country.
I'm so glad that I got to answer the question just before this and talk about human centered policy, because that's really how this came to be. And, and even recently, I mean, September of 2022, seems like a long time ago, but it's not, especially in government timelines. And so we had our national digital equity
Summit, September 28. And that was a culmination of many different things that we had been doing around digital equity to support all of the work that was going on, there still is going on, put forward a resource had a celebration, learn from others. That morning, before the actual event, we had some folks come together with my entire team, because everyone was there in the office. And it was focused on evidence, and how are schools making the EdTech
selections? What long before we even get into procurement? Because that is a whole other process? Where do we even start? And so because it is a local decision, we can certainly put forward ideas and questions and things like that to get folks
having the conversation. But what we heard from the people in the room, which they were representing membership organizations, like CCSSO, and Sita, as well as some folks that are working more specifically within the EdTech evidence work, they said, We could quite literally offer a webinar or professional development every single day of the week, on the
four tiers of evidence. And we would still have a ton of people within education that don't know what it is that don't know how to then apply that when they go to look for tools, or interventions or curriculum, because it applies to all of those things, too. It's not exclusive to EdTech. So I was very fortunate to have one of my team members, Dr. Jana Prado, who had just joined our team there in that meeting. And she was like, Yes, this is this is her background, she was a
practitioner. But then she went into research more specifically. And she was like, there is a gap. And we're like, No, it's not even a gap. It's a chasm, like, real is a chasm between researchers and practitioners. And I mean, I would put myself in that category. As a practitioner, I did not understand what it meant to be using evidence based strategies or tools, etc. So when we heard directly from these folks, we said, there, there's a missing
piece here. And this is where we can step in and say, hey, here are some additional resources that might help you understand that. And so that's, that's where the project started. And it took us quite a bit of time to get to where it was thinking of how to make the information that when you go to look at the definition, they take up an entire like half page of a
document. So how do you then make that more digestible and accessible to a classroom teacher, or to a school principal, that's not going to read a 40 page report, or to a superintendent that will also not be how, you know not have the time for that. So that's where the one pagers, which is very government friendly. For us, the idea of a one pager it's really two pages, but printed on
both sides. Who did that came in to be and then we're doing we have the two blog posts right now that talk about how to use a one pager is with a team from across your entire district to then have a conversation to then go into the procurement process. And then additionally, we will have some more blog posts that are coming out in the next couple of months.
It's fantastic. And I just want to unpack a couple of acronyms in there in case people don't know. So you said Sita, which is the State Educational Technology directors. Yes. CCSSO, which is the Council of Chief State School Officers, right?
You did mention a global audience and my apologies, I jumped right into my acronym.
That's that's your role. That's how you got to speak fast about lots of big groups. That is true. But so these one pagers are really trying to, you know, democratize basically make this usable, actionable, quick to digest, and actually guide decision making rather than being these long documents and white papers sort of on a website that people know about, obliquely, but never
really read. I think it's a terrific idea when something it really meets the moment right now and people are having to make, especially right now really often very quick decisions. There's so many tools, things move so fast. So we've reported on the show for quite a while about the rise over time and sort of the importance of evidence in decision making for edtech. It's it's kind of an amazing thing, how little evidence does often come into decision making.
Because of the chasm you just mentioned, the sort of researchers and practitioners are worlds apart and we don't hear often dig into the four tiers. These ESCA tears that you've mentioned, either and I think that the people listening that our listeners might as a little bit of a, an encouragement for them to dig into your one pagers Do you want to just give us a little crash course
happy to, since you mentioned like so many tools, I think I also want to like kind of set the context here too. While we cannot endorse this specific company by any means I we do find their data incredibly helpful in setting the stage for these conversations, which is learn platform has data that shows that on average, a school district has 1400 17 tools, 1400 17 tools, and that's for an entire district, I get that there are going to be some duplications, because that's
just inevitable. But are those all evidence based? Probably not. And so that's where when we started to see those numbers, we saw their data started in 2018, that showed like 500, and then this massive spike because of the pandemic, and then it's kind of leveled off a little bit. But it is still there for sure. And so we're at that 1400. And again, we know that not everyone is looking at the evidence
behind it. And so we actually don't even start with tier one, which is where you think as it's tiers one through four, we purposely have started saying like, we're starting with tier four, because that is where the actual entry point is for this. And as you mentioned, it's really hard to get evidence on ad tech tools. In general, you will not see a lot of randomized controlled trials on ad tech tools. There are some I don't want to say like just completely
wipe that out. But it's so hard to do traditional research methods, I don't want to get into all of the different types of methods because I am not a research, the quasi experimental versus, you know, like the gold standard of the randomized control trials. But the tool changes too fast when we have a traditional research approach. And so if I started something
today in May of 2023. And then even if I come back to it in August of 2023, for a new school year, the tool has changed quite significantly, whether it's version one, point 2.3, you know, whatever the case may be. So there's not a lot of evidence that already exists. And so that's where that tier four is where we can actually start. And so we're working our way towards tier one, if you want to think of it like coming down that funnel. But tier four is basically where you start with a
logic model. If you're not familiar with a logic model, that's totally okay. There are plenty of templates out there that talks through logic models, and you are identifying your goals, you're identifying short, medium, and long term kind of plans that will get you towards that. And so if you are going to look at a specific app or an intervention, what are the resources, the activities, the outputs? Again, that's short and medium term outcomes? And then what are the long term outcomes
that you're hoping for. And it starts with a problem statement. So it starts with, we have noticed that the numbers in second grade math have dipped, whatever the case may be, we are going to try this app for 20 minutes, are they in these two rooms, and that, you know, so you can get really researchy and kind of nerd out there. But the logic model is the easiest one for people to be able to get
started with. And we're seeing one of the reasons we purposely made sure that we launched this at ASU GSB was because we knew that the developers were all new DSP. And if they can come into the conversations with school district saying, here's our logic model, here's how we've seen this use in other schools, they are going to get so much further in the conversation than someone just doing a cold call or having a conversation without any sort of evidence behind
that. So that's tier four, then you work your way into tier three, that's getting into some more nerdiness around the types of research, it's more of a correlational study, then you get into tier two, which is the quasi experimental study. And then in tier one, which is the experimental study, and that is more of that gold standard, the randomized control test, we have seen that companies that have gotten to tier one, it is very challenging, again, because the
tool changes so quickly. Even if it is a relatively stable tool, there are still enough changes, it's hard to look at things kind of longitudinally. And so the goal is tier one. But at the same time tier four, if you can start with tier four and have that logic model ready to go, then that is going to certainly benefit you in the long run as a developer or someone going in to talk to school districts to make the case for using your tool over others.
Excellent, excellent. And just just to give the the official names that match each of the tiers that you're naming Christina, it's like tier four, the logic model is demonstrating a rationale. So it's there's no evidence necessarily in there, but you have a structured way of thinking about it that you can talk through. And it all makes sense. So you've thought through what the evidence would be, then promising evidence is that correlational level moderate evidence, is that quasi
experimental? Yeah. And strong evidence here. One is, they say at least one well designed experimental study exploring the causal relationship between variables. And of course, you know, the random assignment is part of that as well. So I actively didn't go into research and academia as well, I could have flirted with it. But this is not my natural language, either. But it is really important. This is the gap between researchers and practitioners? It is, yeah, but really excellent explanation.
And so this is how it tech developers, you know, at tech companies, people who are trying to sell into districts or schools should be thinking, okay, I can demonstrate a rationale. How can I or am I wrong? Yeah. Yeah,
no, I think that's exactly like, Yes, please.
Yes. And then on the other side, how are you recommending the the districts in the procurements use this information? Are you you're sort of trying to use the tiers as a like, access? It's like, Oh, if you're not at least a tier four, you shouldn't even be considered or is it a little more squishy than that?
I think it's gonna be a little bit squishier. And it also depends on the capacity of the school districts. I mean, I think at some of the school districts that I served in Nebraska that, you know, only had 100 kids across the entire K 12. So the superintendent was no joke superintendent was also the bus driver, and those types of communities and those types of situations within our size of
actual system. We don't have the capacity, we don't have the people that have an ed tech or a research background that can say, Oh, yes, I can write to this. So we need to make it as easy of an entry point as possible, which, again, I think if if the developers can come to the conversations with the information around the different types of or which tear they would fit into, that's going to
be incredibly helpful. But then it's also important, like the onus also has to be on the school district that is making the decision as well. Let's say it is one of those smaller schools. So it's let's say it's 500. Students, I don't think that we're going to be using the full 1400 edtech tools at that point. But let's say even if it's 500 tools, one tool per student, that's still a lot of tools that really need to be weighed and looked at for the evidence, as far as is it an
effective tool? Are we simply using it because the teacher wanted to the responsibilities need to be split up certainly amongst the developers and the vendors that are coming to the schools, but then the districts and the schools that are making the decision and should be looking for this actively looking for this, too?
It makes a lot of sense. Well, you know, you mentioned learn platform in passing. And I know you're not you're not endorsing them in any way. But we've had Karl Rove tennis from learning platform on the podcast a couple of times, once a year and a half ago. And we're just starting to think about evidence. And then right after his company was acquired
by Instructure. And it's just it's so interesting, because it reminds me a little bit of what you're doing with the toolkit in that by sort of making sense of this incredibly complex world, the 1400 tools, and I think there were numbers like any individual teacher uses, like 80 Something tools or even more, real, more, right? Oh, my god, yeah.
140 848 tools
per teacher, I mean, can you even imagine? So by starting to make sense of it, and starting to sort of just help people see sort of how complex this is, and that it needs simplification, but that if you have these sort of heuristics, like these tears are these examples in the toolkit, it's not as scary as it might sound you hear 500 tools or 148 tools, you're like, I'm not even gonna, I'm just gonna ignore that I ever heard that and keep
doing my job. But when it's like, I'm gonna read a one pager and know what tier four really is, and then I'll be able to ask the next vendor, do you do anything ESCA, tears? You do tier four? Hey, that's, that's not so bad, right? That's something you can actually incorporate into your, into your decision making process is that sort of a part of how you're thinking about
a percent. And that's where I think right now blog post number two, that we had the the first one, which is just building evidence to guide edtech adoption, introducing the one pagers and then the second blog post that we have in this series is using professional development to support the evidence building in schools and so it's getting to the point where like, Okay, now let's, let's figure out how we can use it and make those better decisions together.
One of the things I find so interesting, you mentioned that gap between researchers and practitioners and another sort of complexity of of all education systems, I think is the is the relationship between government and the private sector and comes up in most countries and all sorts of ways. Right now one of the hot topics of course in edtech and
in all of tech is AI. And the Biden White House recently hosted a summit with AI leaders including Sam Altman from open AI Dario Amadei from anthropic Satya Nadella from Microsoft and Sundar Pichai, from Google from alphabet. I mean, these are like the biggest names in the world in this to discuss responsible AI policy. And you know, we haven't been talking about AI
much yet. But everybody is really curious about how the pieces are going to play together, when it comes to AI, being used in all sorts of sectors of the government is, especially of course, at Tech in our in our from our perspective, and you know, what regulations might look like? Or what it would mean to have a safe AI for for schools? You know, I know we're in the very, very beginning of this world.
Anything you've been thinking about from the Office of Education Technology perspective, about the future of AI in edtech? I have to ask,
Oh, yeah, absolutely. I'm glad that you did. I don't know how long it's gonna take for this episode to come out. But hopefully, by the time this comes out, he will have a new report that will be
out as well. And we've been in this space, since pre pandemic, I do have to say that it does not feel like we've had a lot to publicly show that we've been actively engaged within the developer community, but as well as the policy community who is concerned about general student privacy and cybersecurity and all the other risks that we that we certainly have to navigate, let alone the actual algorithms that could potentially produce bias results and all of that
separately. So we are actively working with the White House Office of Science and Technology Policy, as they are working on a national AI strategy. And then to come in and say, here's how we're thinking about it within education. I have been told many times that I am very optimistic about AI and education. It is just my nature. I'm so sorry. I am an optimistic person. That is why I thought early childhood.
But I do see the potential here am I going to throw everything out in the meantime and wait for AI to like, get there? No. But I do see so many opportunities here with the acknowledgement that there are also risks and risks that we don't yet have identified. So I think that that's also important to note. But we did start this work. And I will I want to call out my my team member, Bernadette Adams, who has actually been with the Office of Ed Tech since 1996. And it started. And so her
expertise is just so deep. And when she dives into like one topic that she's leading, it goes so vast and so deep. And so Bernadette has been in this and convened researchers and practitioners back in 2020, right before the pandemic, to think about how teachers are going to use things that have AI in them in safe and responsible manner. And then also looking at kind of like the larger research agenda questions that come up for those that are in the
research space. And while AI sounds like it might be new, as far as what we are seeing culturally and socially, we've been using AI in education, more specifically with intelligent tutoring systems since the 70s. And so it's also important to call that out and say, Hi, this has already been in schools. It's looking different now, especially when you look at generative AI, using large language models that create these types of resources. And you're like, where did this come
from. But it's important for us to really look at the opportunities and the capabilities of what it's going to present. And also looking at the risks and making sure that we are always talking about humans in the loop. We want the electric bike we want the humans still driving the bike, but then
being amplified. We don't want the robot vacuum that can go off on its own and do something like we never want to replace teachers, Secretary Cardona has said that so many times, but we know that AI is going to be used in classrooms. And we have to prepare learners to go out into the workforce to work with AI as well. We don't want to get rid of the humans in that whole process. So we want humans in the loop at all times when we're using these types of
technologies. And then just making sure that we are having open and honest conversations about it in the process. And that's why we are going to put this report out and then it will lead to kind of like the next phase of the work that we're
doing around convenings. And bringing folks together to have these conversations and let everyone know that we are having these conversations, some additional guidance, as well as communications and kind of outreach with district leaders to hear how it's working for them or it's not working for them. Did they do a total ban on tat DPT or not like we we want to hear all of the things for us to consider as far as policy concern for sure. Yeah, that's
a really thorough and interesting explanation of where things are at right now and makes a lot of sense, you're in the right place in terms of being bullish and positive about the future of AI. Any listeners will know, I am extremely bullish and very excited about what AI can do for education. But yes, it would be naive to think that this will just unfold without any incredibly bizarre and surprising problems and confusions. I mean, just it's
going to happen. I want to dig in, if you don't mind on this human in the loop idea, I love this metaphor of the electric pipe. And it's something that I've been wondering about in just looking at, we've been compiling a list of some of the AI ed tech companies that are out there, there's already you know, 150 Plus, it's coming
really quickly. And one of the things that I'm trying to make sense of is, does human in the loop in this context mean that students will not be able to access AI, without a teacher being sort of in between to check it or see it? Or verify? Or does it mean that there's going to have to be a really thorough and education specific LLM that has been vetted by humans so that anything a child sees, or students sees has gone through that kind of human
intelligence? Like, I think it's hard to hard slash can be misleading, I think, to try to define what a human in the loop means, in this new context. I'm curious what you think about that,
I definitely hear that. I'm glad that you brought that up. We've always again, if I go way back to what I mentioned at the beginning, where it's like, at that end user, and that's the human centered policy, it's the human centered tools that come along, that we think are going to go farther. And so that's how I've always been picturing it in my head is like when we talk about the educator at the center of things, or the student or the learner at the center of this, how is that helpful for the
learner? Is it going to be an adaptive platform that based off of certain number of things that they have gotten right or wrong? They go back to reteaching? Or they go on to accelerate, like, whatever that may be? That is, where the human isn't that loop?
For educators? We've been also thinking about it because the conversations have quickly switched to substitute teachers get rid of teachers like No, no, we don't want that, we can think about how this would come alongside and actually reduce the cognitive load on teachers. Because we have asked so much of teachers, especially during the pandemic, to add this, this, this and this, and this on plates that are very, very, very
small and are already full. And so how can we think about the use of these technologies for reducing cognitive load, the two examples that I go back to most often are taking attendance and lunch count in the morning. Like, if I could use an AI tool to just do that for me, more
instructional time. And then for for educators that did teach or know or familiar with early childhood and teaching reading, there are tools now that you can have children go on and read passages, and it will check for fluency, it will check for all of all the things accuracy, comprehension, all these things, do you know how many hours of lost instructional time that my former students had, because of that, it's so important that they're tracking the progress for reading acquisition, but it
is a significant amount of time, and it just requires so much time. So then I think about the overall kind of reduction in that cognitive load is quite significant. And then maybe I'm gonna like my job better, and then maybe, you know, like, all of these things that can potentially kind of add up. And we certainly want these things, these tools come alongside and help and support as much as possible.
Yeah, so the, the phrase, you know, AI teaching assistant, has become one I've heard quite a lot in the last few months. And it sounds like some of the functions that educators perform are not fully educational, or they are extremely time consuming and don't actually add as much value as they as they would if they're playing the true human intelligence, like, you know, counting heads in a, in a class or on a field trip. So that that makes a lot of sense. I think those are good
examples. It's really, really interesting to see how this is going to evolve because I see you know, you definitely see AI companies that are exactly in that mold, you know, AI teaching assistant AI to take this off the teachers played AI to support them. And then you see others that are like AI to help students take notes or AI to summarize this are AI tutor that students can access while
they're in class? And those are the ones that I'm like, I don't I'm just curious about what the future of those is because is there a human in the loop in those I mean, it would be silly I think personally to deny children the access to this incredible new technology at the same time do we trust them to have access with a deep trust the tech that feels like such a pressing question you know, as of this moment, masa
such a pressing question When I especially if you look at some of the research that has been done, where it talks about bias, and within the data that then informs the AI itself. And yeah, so we are, again, actively having conversations, not only with the White House, but also with our stakeholders, it's important for us to understand how AI technologies for safety and security purposes, are using facial recognition, which has historically, obviously disadvantaged people of color.
So like, we have to have those conversations because we serve everyone. And so we want to be able to address the, again, the positives and the opportunities, while also recognizing and acknowledging the risks.
Yeah, it's I mean, it's such an amazing moment. It's one of these topics where, you know, if we have the same conversation, in August, it's going to sound totally different. living through history, I have a sort of a wacky question. I'm curious if this if this resonates with you just combining some of the things we've talked about? Yeah. Do you see the role of AI in providing evidence, evidence for efficacy? You mentioned that, you know, RCTs are so hard to
do. It's slow. But we're starting to see some interesting new methods coming up. You know, we've talked to teach effects on the podcast, where they record and analyze class speech, and then give reports out about who was talking and how much and what the teacher was doing all aggregated and anonymized. But it's like, that's data nobody's ever had before. Yeah, I'm curious how you see this, this interesting relationship between AI and efficacy evidence? Yeah,
I think we will start to see some changes in this. One of the most simple applications right now that I can see, is asking something like chat GPT, or generous AI tool to create a logic model. Yeah. That makes sense that would make perfect sense put in all of your criteria as your prompts, and how to create a logic model. There's, I mean, like, that's the first step.
That's tier four right there. I also, I know that it will certainly come up on the data analytic side, once we have data sets that are pulling from various things, whether it's assessment data, or kind of formative assessment and trying to understand that, I would say that educators don't always have a background in data analytics either. And yet, we are charged with saying making data driven decisions or data based or data informed decisions. But we are not taught that and teacher prep
programs. And you can go out and look at colleges of education today. And they're still gonna have pretty similar courses that I took in, like the early 2000s. And we had the one ed tech class that we took, and that was cool. But like, we didn't talk about edtech and reading methods or math methods, let alone get into, here's how you would look at your formative assessments to then drive what is going to happen next in your curriculum,
things like that. So I think the data analytics piece is going to be really interesting to follow. And maybe we get some more targeted interventions sooner for learners. I just don't know yet.
Yeah, no, I'm totally with you. I don't know either. It's such a it's such an interesting moment. And one of the things that's always struck me as really interesting about the the tears of evidence, historically is that some of the, the models or the tools, especially the EdTech, tools that reached tier one that reached the highest standard of evidence are developed by
researchers. There, you know, the Carnegie Carnegie tutoring system developed by Carnegie Mellon, or assessments developed and that's very rare, by the way, not that many edtech tools are developed by, you know, academics and researchers, but the ones that do because they're no research incredibly well. They're all about RCTs. They're all about figuring out if it really works. And they get this amazing tier one evidence. But they are a very specific flavor
of ed tech. So I'm curious if you see that changing in the future, like Will people be able to reach tier one with without having a PhD next today?
Oh, that's so interesting. I hadn't thought about that. And having visited Carnegie Mellon's campus, I also think it could be I mean, like, I don't want to call CMU out here, but I'm going to call them out for a second. Because I do also think that it's the way that they've structured their campus, and their actual kind of integration of studies because they were very intentional. Have you ever been to their campus?
Okay, so beautiful campus in Pittsburgh, but they were very intentional with the computer science building, having a literal bridge to the humanities building. Wow. And so if you also think about design thinking and thinking about our end users in the end, like how has that helped inform some of the stuff that's coming out of Carnegie?
Okay, all of that to say, I think it will be very interesting to see where this goes as far as kind of the the application of AI within the different tiers, perhaps we can get to a place where you don't have to have a research background to be able to look at tier one, which is, again, back to the gold standard of things. Who knows?
Who knows, maybe AI will play a role in helping procurement departments evaluate whether tools are have evidence for their particular populations or age group. So yeah, I mean, there's just so many options. It's it's exciting time, and I'm the bomb start falling, let's be optimistic. And I'm really, really excited about where AI is gonna go. But I think we have to be smart, you
can't not see things coming. And that's that's, you know, that's all of our, all of our across the bear right now, we have to figure that all out. So we are coming on time here. We end every episode with two questions. I'm so curious, what is the most exciting trend that you see from your unique perspective in the Office of Educational Technology, that, you know, what's a trend in edtech that you think our listeners might want to keep an eye on?
Well, I feel like we have certainly talked about AI. So I'm not going to go there. The one thing that has come up, and it's primarily because of the pandemic. And to be able to see it from this kind of level, I should say, I'm really excited about the way we are talking about mental health in education now. And it's so important for a whole host of reasons, not only for learners, but also for the educators themselves, because educators are also struggling or also
burnt out. I talk very openly about this on social media, my own personal social media, but I manage my mental health of depression and anxiety with medication and therapy, and I talk about it to D stigmatize it. And I've had many educators that have approached me because they see that I'm openly talking about it. I've had many conversations with students in
the process. And so I think the fact that we are seeing more of a shift within society, probably larger, but then within education, where we're not afraid to talk about emotional health, well being like in general is really great to see. And then we are seeing the options when it comes to the intersection there of mental health and technology because we are seeing more things like telehealth services and tele
mental health services. And knowing that we can use actual federal dollars towards those services to provide for students is really important to me. And it is that it's that side of the humanity piece. We want to make sure that kids feel safe and seen and heard, but also like recognizing that technology can help us when we have limited capacity.
That's a terrific, that's a terrific answer. And I'm really, yeah, I I don't have much to add to that. I'm really excited. I think the younger generation is also I think more in my limited experience is a little bit more likely to destigmatize some of these things. Gen Z already for you. Yeah. And the pandemic also helped us recognize that mental health and education are not actually different things. I think that you know, I think
yeah. 100%. So last question, what is one resource that can be book blog, newsletter, Twitter, feed, anything a person that you would recommend for somebody who wants to dive deeper into the topics we discussed today? If somebody wants to understand more, where would you point them, I'm going to be super selfish. And
say, please visit the OVC website. That ed.gov te ch.ed dot Gob tech.ed.gov because not only will you see the EdTech evidence toolkit that we that we talked about, but you will see the forthcoming AI report on there, or work on the national edtech plan, which is the flagship document that we put out as an office, any of these types of things that we put out there always gonna be found on the the Office of Ed Tech website,
that's great. I usually don't allow people to put themselves as a resource. Because the Office of Educational Technology is such an interesting place. And people I don't think visit it enough. I'm gonna I'm gonna let it slide this time. Thank you so much. It's really been a pleasure. It's really nice to know that the federal government has people with true teaching experience, and who really understand education from many different lenses in the driver's
seat. And you know, we're all I'm excited to see that AI report. Maybe we can have you back on to talk about it when it comes out. It's really it's really an amazing time. Thanks for being here with us. Christina Ismail, Deputy Director of the Office of Educational Technology at the US Department of Education. Thanks for being with us here on Ed Tech insiders. Thank you. Thanks for listening to this episode of
Ed Tech insiders. If you liked the pod Cast remember to rate it and share it with others in the EdTech community for those who want even more and Tech Insider subscribe to the free ed tech insiders newsletter on substack