Welcome to Season Two of edtech insiders, where we talk to the most interesting thought leaders, founders, entrepreneurs, educators and investors driving the future of education technology. I'm your host, Alex Sarlin, an edtech veteran with over 10 years of experience at the top edtech company.
Hello, Ed Tech Insider listeners. It is April. Welcome. We are your April Fool's is Ben. And Alex, your co hosts for another week in ed tech. We have so much going on in our space. We're also in the middle of conference season. Lots to report. Lots of announcements coming out. But first what's going on with the pod and Ed Tech insiders? Alex? Yeah.
So two things to announce. One is that on the pod next week, we have an amazing interview with a mathematical intelligence expert. He's a artificial intelligence expert and mathematician was an edtech exec for many years. And it's really interesting breakdown of what artificial intelligence can and can't do versus human intelligence. Really fascinating skies. Brilliant. And then coming up right after that interview with the two past ASU GSV cup Grand Prize winners, Kibo school and Simba, we have the founders of both of those companies talking about that, you know, that experience as we get into ASU GSB CS, and it's really exciting. We will be at the conference, I'm going to be moderating a panel on higher education and AI, I can't wait to do that. The other thing we want to announce is that my edtech product management class that I did last year, we're actually running it again in just a few weeks. So if your current or aspiring Product Manager, please go to Koh leap.com. That is the platform where I am hosting this ed tech product management class. It should be a really fun community, really awesome experience. And it's all updated to include all this AI stuff that we talk about every week. Ben, what is going on in the SEO world in ad tech insiders event land?
Well, yesterday, we had an incredible event in San Francisco with 50 Plus folks coming out to driftwood pub, thank you, for all of you who turned out that's following our events in Austin as well as Austin. So we're taking it to the Road Again, April 17, in San Diego at ASU GSB 6pm to 830. We've got a cool spot rooftop bar at right near the aircraft carrier. So there's this big aircraft carrier party every year at ASU GSB. And so you can pregame at our event. What's unique about our thing is we're really focusing on it as an AI social. So we've invited some really cool guests who are AI experts to have small group Convos about different elements of AI. So you can network and drink and have fun and see your friends. And or you can sit down with some of the leading cutting edge ad tech folks out there who are thinking about applied AI. I will also just say for those of you who haven't taken out this course, it's for veterans. It's for rookies, there's something everyone will get out of it. But what I love about how you run that course is there's also a lot of community peer to peer learning. And so it's just a great group of people that you had last time. So encourage folks to check that out. And then we're also going to be doing our postcards from ASU GSB session. So if you have folks that you think we should be interviewing, please let us know. So lots going on with the events. And speaking of AI
experts, we also have special guests on this episode, Qian cuttin, feroce, CEO of work era, they just raised a $23.5 million Series B round Qian is hopefully joining us at the event and ASU as well. It's just it's they're doing amazing work. So that's going to be a really, really interesting interview as well. But with that, I think it's time to jump into the news. That's a lot of a lot of precursor. Should we jump in first week
of April we're starting to get the AI backlash and we knew it was coming. You know, I think the headline was really a letter from Elon Musk at all. You know, it's you've all Harare is on the list. Professors from MIT and Stanford are on the list. It's a conglomerate of industry leaders, tech folks, academics who are worried specifically about AGI or this point where you know, the computer super intelligent surpasses human beings, but they're also worried about the kind of social ramifications impacts and so The letter itself calls for a regulatory body. And most importantly, a six month pause on large language model development paths. GPT four, there's a bunch of other news that we'll cover. But let's kick off with this one. You know, when you read the letter what came up for you, Alex?
Yeah, it was really an interesting moment. So, you know, we've talked on this podcast about how AI is moving really fast and how the government's especially I would say, the US government, being a gerontocracy is not quite ready to understand it and where it's going, and what it's going to mean, I admired very much the part of this letter that said, we have to think about regulation, we have to think about watermarks, we have to think about, you know, policy that will make sure that this doesn't just drive our entire, you know, society off off a cliff. And they didn't specifically mean to us, but I think that the US specifically is behind in this, I really appreciated that. But that said, the idea of a bunch of tech leaders, especially calling for a just pause in development of technology. That's literally what the letter is calling for. So let's take a six month breath and figure out what we're doing here. It's hard to even imagine that tech community could ever do that. I mean, this is a this is a game theory problem, right? Because if, if open AI just pauses for six months, then Google bar can just take it over. If Google barred pauses for six months, then Bing is going to take over like they would have to literally all decide, and transparently, it's just impossible to imagine that in this hyper capitalist world, with all this investment money going in that the entire field can just stop on a dime. So it feels a little naive to me, frankly, I'm like, of all the things they could be asking for getting these huge names to get together and think about just saying, Stop put on the brakes feels like, I don't know, it feels like a very weird, a weird ask. I mean, since when does anybody asked to just put on the brakes on anything related to tech or innovation or investment? It's so strange. What do you think? Yeah, I really
don't take it at face value. You know, I think maybe I'm a little bit more cynical here. But they're playing a political game here. And it's unfortunate that some of the people who are genuinely concerned about AGI are getting wrapped up in it. But this is classic big tech versus big tech battling. And the way you big tech battle is you use public opinion, legislation, courts, legal, all that stuff to slow your competitors down while you catch up. And I mean, I think it's naive on the part of some of the people who signed on to it, because I think they probably have a genuine fear of some of the consequences. There's some other elements to that show a lack of self awareness. In the letter, they say, you know, are we going to let AI take away even our fulfilling jobs, and I found that really condescending, it's like, Oh, hey, I was fine, when it was taking blue collar jobs. Those aren't fulfilling jobs. But now it's coming out from my job. Oh, now, what are we going to do? So just even that part of it made me feel like this is coming from a place of being out of touch of privilege. And, you know, both the kind of recommendations are not only like, improbable, they're impractical. And, you know, this is also where I think listeners to the pod understand government plays an important role in balancing things, especially when it comes to kids and learners and so on. But we have not seen very many good examples of functional government oversight in basically y number of industries. So why would we expect a governmental agency to be a solve here? The story that I think is getting buried in by this lead, is actually there's a lot of people doing really unique, interesting things with data at a sub scale. And I feel like the LLM aims and you know, that's the core engine of generative AI. And, you know, the theory goes bigger is better on the LLM. They've been sucking the oxygen out of the generative AI discussion. But some really important studies this week coming out, one from deep brain, which shows that very small LLM 's very small datasets can actually do some pretty powerful generative AI using the transformer technique of inferring what's coming. So two examples one is they taught an AI on a much smaller data set, how to play Atari video games and They had 100 different games. And they use the data, you know, playing for two hours for each game, they had not only the Atari predict what was coming next, but they also had to go backwards, and then create artificial, almost synthetic data around other hypothetical games that had been played. And within just two hours of tuning training, the computer on like 90% of the games could be a human being only when the context radically shifted, did the human outperform like so if you went to a whole new level, and all of a sudden, the rules are a little bit different. What this shows, though, is that a profound use of generative AI is actually it's almost like derivative here. But it's basically creates new data, that synthetic data that then can train your AI. And so you actually need way smaller datasets in the first place. And another story that came out this week, there's an organization that is using open source LLM from two years ago, and then training them up. And so it's totally open source. It's a free LLM. And then they're, you know, basically adding, they basically added, like three months, four months of tuning on top of it, and their performance was right at like GPT 3.5. So a question for our space would be, do we actually see a world where it's these mega LLM that continue to grow and be dominant, they're the electricity provider across the space, knowing that they're basically perpetually going to burn money, because you have to just keep plugging in more money to build the larger LM larger LM and they will get more and more powerful to do the tasks that generative AI needs to do in the world to add value, are we actually at the point where it's close enough to like, you know, Assam total value add that we're gonna see a raft of smaller elements that are specialized for different use cases. And so third headline that I would just show, you know, talk to the listeners about is Italy, just ban check GPT. And there's also an article about Ed Tech and GPT. At cosin. A bunch of CTOs from school districts basically said, there's no data privacy period and tragic beauty. Imagine that in edtech, we actually have our own MLMs, that are trained specifically on kid safe data, and purpose built for anonymity, I actually think that that could be a huge differentiator for a single player or a set of players. So while we're all talking about, you know, super intelligence, and when does that come and how we're speeding towards that the kind of risks and opportunities are super real right now. And what we're seeing is actually subscale MLMs, and subscale AI companies doing pretty incredible stuff with limited data by use of synthetic data creation. So that's what I'm geeking out about this week, Alex, as you put all those pieces together, what comes to mind,
I'm going to take them one by one, because there's so many interesting things here. So first, just about the Italy piece. And the privacy piece, Ed Tech, we know has a whole different set of privacy concerns than other tech fields, health tech, maybe being excluded there, that one also has a lot of privacy concerns. And it's really interesting to see a whole European country, and maybe the first of several saying, This is not GDPR compliant, we have done all this work to think about how privacy works in Europe, and where the data lives and all of these things. And this, it's may be exciting, but it just doesn't meet the bar. So we're going to ban it for the whole country. That is, I don't know if I've heard of something quite that drastic, I would say from a European country in a long time. We've definitely seen, you know, China ban a lot of American tech companies, but it regulates it regulates hard, but the idea of just, you know, banning it, something that new that quickly, really interesting to see. And I think we're gonna there may be a domino effect there, as people start to understand why they did that. I think other European Union countries and this is breaking news that this happened, other European Union countries will have to say, Okay, well, if indeed it is true that, you know, Chuck GPT is not GDPR compliant, and Italy's just write about this, then we have to decide whether we're going to value our privacy issues over sort of innovation, like are we going to? Are we going to put our own tech industry, you know, a year behind in this fast moving world by blocking chat JpT or are we going to embrace it and try to and have to figure out the privacy issues. It's just going to be an interesting discussion with a lot, especially in Europe. As for the sparse data sets, To the idea of training with synthetic data, I had a sort of firsthand mind blowing experience with this recently, I was talking to some AI experts about a use case in which a speech recognition use case, which is basically, you know, how can you train the AI to recognize speech with a new group of speakers like one that is not typical of the large speech models that are already out there. And I've already been trained. And what the experts recommended was, Okay, have a few users say, you know, a little bit just, they don't have to say very much, you just didn't need to 10 seconds of their voice, then put that into Google's tool, which can extrapolate out and basically continue talking in that same voice, and say, almost anything you'd like it to say, or at least extrapolate out and just continue to talk, build a dataset that way, and then use that dataset to train a model to recognize people with voices like that. And by doing that, you can drastically reduce the amount of incoming data you need, because you're basically turning every little chunk of data into a big chunk of synthetic data, which can then train a model. And that is not what I saw coming in that conversation. And it speaks very directly to what you're saying about sparse data, or you know about creating intelligent AI with with small models that Atari paper, I have not read it, that sounds incredibly interesting, especially because the the Atari use case is one of the first things that AI has done really, really well as mastered these game environments. You know, we all know deep blue and Chess and Go and all of those things. But video games have been one they've done for a while, one of the landmark arguments of people who say that human intelligence is still way ahead of artificial intelligence is that look, yeah, a computer can learn to play video games really, really well. But it takes them hundreds and hundreds of hours of playing, you know, sometimes against themselves or playing over and over and over again. And you know, a normal person, you could give the same game to a six year old, and they'll learn how to do it in an hour. And people say, there you go, that shows that AI is not actually faster or better. But if you start looking, you know, going down the route you just said been where it's like in two hours, not that long. Already, you can train, you know, the AI can actually extrapolate out and start to understand the rules of the game or start to imagine different circumstances than it really changes the equation. So I think you're right, that this sort of sparse data use case is something that is flown very much under the radar. And I know you've mentioned on the pod in the past that the Duolingo has been thinking a lot about this, and how, you know, they do a lot of voice recognition, they do a lot of different things with AI. And it's sort of like, yeah, assessment, you know, there's a lot of new things that can happen with AI, if you don't need a massive data set for training, which has been the blocker for many different AI use cases. So that's huge. And then the last piece is this idea of LLM 's, you know, electricity, is it going to come from the Googles and the open API's and the, you know, metes and the Microsoft's or is it going to be custom made LLM? And I don't know the answer to that. But I'd be it. You know, we've talked about AI as the new electricity for a while. And I think that that is an interesting metaphor, I always go back to it because electricity was a Have you ever seen those those great movies about Edison and Tesla, it's like electricity, when it was launched, was something that had the same possible outcome. It could have been owned by a few monopolies that sort of ran all the all the plants, or it could have been something that was sort of democratized and people could use in all sorts of different ways and different amounts of voltage and all of that. And it was a it was a political battle of why that why it came out the way it did. So I think the political battle is probably just started. I mean, if you're open AI, do you they're racing to make API's, they're racing to get everybody to use their their stuff. But probably one of the major reasons they're doing that is because they want to be embedded in every system so that if somebody else comes along, whether they're big or small, it's already taken. So I think it's a it's really Yeah, economic question more than technical well, and maybe
the electricity analogy extends in that we now can have distributed production of electricity through solar and wind and so on, were centralized through large power plants. And so which model will be dominant? Or will there be a network of them? One thing that I think is a tell is open AI just opened up its marketplace, this app marketplace, not just for our listeners, it's not live to everyone you have to join a waitlist. But there's a scramble for companies to be listed on the open AI marketplace. A marketplace is just a classic business strategy of you know, basically creating this incredible surge in demand and then matching it with supply. And so you can see that Chad GBT is not resting on its laurels thinking, We're the new electricity. Everyone, you know, pays the toll. They're thinking about, Okay, we've got 100 million users coming to the site all the time. How do we monetize that and solidify our position in the ecosystem. So you know, if you think about the iPhone, and we've also analogize, like our first experience with chat, DPT was very similar to the first iPhone experience where, you know, blackberries existed and all kinds of mobile phones existed. But when you had the iPhone and could access the web, and, you know, manipulate it with your fingers, it was like, Whoa, this is a big deal. But really, the App Store became like a key part of Apple's defensibility. And leverage, you know, get everybody wanted to be on the App Store. And by everyone being on the app, sort of you wanted to be on Apple to access the app store. So I think this is playing out so fast. It's like things that would normally happen in four to five year increments are happening in four to five month increments. But fear not interrupted at tech entrepreneur, the oxygen or the electricity is not being sucked up totally by jet jet. TBT. This week has really shown that there's lots of room for entrepreneurship in our space.
And you know, in the EdTech space, you know, we talked just recently about how Khan and and Duolingo got early access to these amazing API's. But we saw upgrade today, the Indian Ed Tech company introduced chat GBT fundamentals into course curriculums, we've seen edX launch a chat GPT class we're seeing turn it in, is very close to launching their AI detection tool. But what's also interesting about what Turnitin is doing is that even though they're plagiarism detector, you know, at heart, or that's a lot of what they do, they're not coming at this from a pure plagiarism detection standpoint, they're saying, you know, these tools are going to change how we write, we should learn this, if there's a lot of amazing stuff. But at this exact moment, one of the immediate use cases is making sure that it's detectable. So we're going to start there, but they're, they're even coming from that integrity standpoint, are still being very, very open minded. It's been really interesting to see, we also see presto, which is a cool AI ad tech company starting to really blow up and get a lot of growth. And, you know, I'd love to bring that CEO on the podcast as well, because they're doing some really interesting stuff. That's Steadman.
You know, one other thing on that. So if you have these specialized use cases, remember we've been talking about, like defensibility is challenging here when Chuck GPT wanted some new feature set every week, or Microsoft, Bing or so and so it makes it really hard to protect your value prop if it's solely resting on the AI capability. That said, the user interface and the kind of specificity to the use case. That's where all the defensibility is coming from. So like another example of the company, I was at the happy hour talking with Dan Whaley, who's the founder of hypothesis, which is an annotation platform for colleges and universities. So basically, colleges and universities, you know, assigned digital books or whatever, and they can have classroom discussions literally on top of the digital texts, almost like comments in a Google sheet. So all of a sudden, they've implemented chat TPT into that. So not only can you have a dialogue with your classmates, but you can instantly ask questions right on top of your text. And they just announced a deal with Atlassian, where they're going to be across all of that Atlassian and Confluence platform, which basically is the segue from edtech, specific to general market. So there's also ways in which ad tech use cases now, if, for example, it's something like, I want to embed learning on top of a digital text, which here two, four has been a university use case. Now this is actually a general use use case. And people are jumping the borders. And because they built it for edtech. It's also safe, compliant, shareable, controllable has like administrator capabilities, all the things that we lament having to build an edtech. So I just, I keep coming back to the AI movement is not only exciting, this age of AI is not only exciting because of what AI can do, but also learning is one of the like top three use cases for generative AI. So anyways, that we this is just an example of how are on the pod. There's so much to cover each week, and so we appreciate everyone going into a minute 20 here with us on the AI. So let's zoom out. And let's you know, one of the topics that we were just alluding to is colleges and universities, lots of data coming around the ROI of education. Now, colleges and universities are starting to look forward at enrollments for the fall. We're seeing some of the shakeup happening in the education space. What are you paying attention to? What are you watching out you are lead on the university be? Tell us what you're seeing.
One thing that's been covered a lot right now in the in the press, for obvious reasons, is the enormous tuition issue in higher ed, which somehow, despite every year, it being an issue of despite it going up at these accelerated nuts rates, it continues to go up at these accelerated Netsy rates, no matter you know, we've seen a somewhat of a growth of microcredentials, we've seen a growth of students trained through alternative routes, they're called who are doing alternatives to college. And even so you're seeing Ivy League tuition costing over $80,000 a year. That is unbelievable. And you're seeing rising tuition, not just about, you know, there's a great article from the Harvard Crimson basically saying, rising tuition is not just about sticker prices rising and the actual amount of money people pay staying the same, which is what a lot of people say, as a sort of apologist stance on this stuff. It's just not, there's really, really complex issues between the amount of money that students are paying, and families are paying compared to the sticker price. But the big headline is basically prices just keep going up. There's a great article about that, in Hechinger, about prices are rising more for lower income students than their higher income peers. So basically, you know, all in all, the headline is, prices in higher ed are still insane, they're still growing way faster. And no matter what all of us do in the EdTech world, to try to cap those costs to try to create alternative routes and different ways to find scholarships. And you know, all the different things we do. Higher Ed just is, is addicted to money. I don't know how else to put it, they just will not figure out and we've covered on the podcast that there are a handful of colleges have been experimenting with lowering their sticker price as a way to appeal to more students, given that so few students actually pay the full sticker price. But this is the counter trend to that and you're just seeing unbelievably high tuitions. So that's one big one. Another is, as mentioned, you know, the power of microcredentials a really neat article in EdSurge, this week about trying to sort of go through the history of higher education in America talking about community colleges, which did just see a an uptick in enrollments, which is they had been seeing major decreases during the pandemic. And just talking about how, you know, it's starting to be more and more obvious that microcredentials have to have to happen. And there has to be more fast cycle stackable, you know, stackable options for students, and that that might happen, either outside of universities with, you know, boot camps, or the MOOCs. But it also might happen inside the universities where universities themselves can start to offer micro credentials that are workplace relevant. You know, after the first six months of school after the first year of school after the first two years of school, and instead of it being this all or nothing proposition, we can actually move more towards a micro credential world. So it's sort of this like, give with one hand takeaway with the other hand, you know, they're starting to be innovation, people are trying to think, you know, OPM is are under the radar, people are starting to think about how to make this a fairer, fairer and better world for higher education, especially in the US. And at the same time. The colleges just keep pumping up those tuition costs. I don't know what to make of it. I I don't know. Like, then you think about, you know, the cost of delivering education, the price and cost of this? Why are Ivy League schools? And why are so many schools continuing to outpace inflation, even as kids are saying more and more and parents, the ROI just isn't there? Well, I
think we've got three factors going on, you know, in the business model, one is a bifurcation of market. So you're basically seeing elite universities continuing to be a luxury good, it's your higher ed and Ivy League school is now the Louis Vuitton of education. And then you see, you know, a vast growth in the more affordable free or more affordable access. And so the middle layer is the part that's struggling and suffering the most because it has neither the brand cachet of the Louis Vuitton, nor does it have the practical applicability of you know, your off the shelf, no brand backpack. The second thing that is going on, though, is actually a phenomenon that's been studied for a long time. And there's actually a great book that I'll see if I can get the link for It's about World War Two. And these Departments of Defense across the world that geared up for World War Two. And it's a massive story of triumph of how people galvanized for World War Two. And then we entered the peace era. And this industrial complex and and where these war entities just kept growing and kept growing. And there's something around large, massive organizations that they continue to self perpetuate, you hire somebody who's smarter, they make new work, and new jobs and so on. And I think the university bloat and expansion here is really a product of organizational behavior where it's very hard for them to make cuts, it's very hard for them to take things away that they've either given students or faculty and so on. And the kind of market pressures are so long and drawn out, there's not the shock moment. And so basically, you see universities kind of traveling long as normal, and then boom, they're closing. And that's, you know, in startup land, we see things fluctuate up and down, and cuts and ads. That's just not the university landscape. And then I'd say the third thing is, I think we also have to be realistic that price variation is quite common now. So the idea that the full sticker price is not what people pay, that's the article about low income people's their tuition costs going up, whereas high income, people's tuition is staying relatively the same. That to me was the most insightful article, because the theory goes, you know, we charge 100k, in tuition and room and board and all that stuff. But it's a graduated scale based on your ability to pay, and therefore only a small subset paying that much. What we're actually seeing is it's not functioning that way. And so as much as we want these institutions to be about academics, and integrity, fundamentally, it's the business model that is driving some of these, you know, what, in aggregate turn out to be irrational actions. And this is why entrepreneurship is so critical to our society, because inevitably, companies will fill the gaps. Now, we're going to come in a couple of segments to some new data about how Coursera and to you and stock prices are trading low, and they're not having as much of an impact. So I think this idea of MOOCs or of these large scale players, that also is showing some weakness. So I don't know that we've actually hit on the kind of uniform solution, and it may end up being, like we've talked about a million times in the show, you know, modular, a little, you know, more fragmented and new stepping stones, where we just need to make sure that we have navigator support for learners and families. Because, frankly, the data shows that higher ed is just not going to serve the masses anymore, like it did in the post GI world. So
your point about the sort of bifurcation is really powerful. And you know, this Hechinger Report article really does jump out. I agree with you. It's basically saying that for almost 700 universities, students whose household income is under $30,000, saw a much larger percent change increase than students whose household income is over $110,000. That might sound complicated, but basically, they're raising prices more for lower income students at almost 700 universities and at universities. The net price doubled for the lower income students at 19. It tripled and it 10 it quintupled quintupled, you have five times as much cost, it sort of just boggles the mind. Another thing that caught our eye this week in the higher ed landscape is a you know, higher ed dive keeps cropping up, you know, a list of the schools that are consolidating are closing and to your point, Ben about things cruising and then just falling off the cliff, you know, this more than 80 colleges that have closed since 2016, or closed or merged or sort of come together. And I think there's just this inertia is momentum slash inertia of schools just keep doing things the way they're doing it. They make these minor adjustments. The Ivy League schools have so much selectivity, you know, they've so many people waiting for them, they can do whatever they want with price. Everyone else sort of trying to figure out the business model that makes sense for my money, the people who have been best at disrupting the space or the mega universities. I mean, I'd like it to be the alternative credential folks. I think that's amazing and very widespread, but I think that you know, WG EU and Southern New Hampshire and Arizona State and Purdue in the places that have been doing, they've grown at these extraordinary rates because they're going after exactly the students, you know, who would want to go to Adrian College in Michigan, but it raised their their tuition went up 60% If they're making You know, less than $30,000 a year. So they're going to turn to a Southern New Hampshire where it's competency based. And it's a fixed price per semester. And it's just a much better deal. So it's a crazy time. And I hope that we as a as an ad tech ecosystem, keep pounding on those different possible solutions, because obviously higher ed by itself is just continuing to hit its head against this money wall, it just can't figure it out. I think you're more forgiving than I am been for over a year saying that it's sort of baked into the business model. But I feel like some of these folks know that there is a possibility to reduce the price and they just aren't taking it.
Speaking of lack of goodwill, we also should cover the Tick Tock debacle in front of the US government. I think the reason why tick tock overlaps so much with Ed Tech is they've actually made some really big strides into the education and kids space, they've teamed up with Quizlet, they've announced a new stem feed that is really around science learning. In China, there's actually quite a bit of learning that happens on tick tock, but co shozy Chu testified before US Congress, and was basically raked over the coals. I think, you know, one thing that stood out is finally Democrats and Republicans agree about something. So it was actually like one of those moments. But I also think it really raises a bunch of questions around what does this mean for Tiktok education? Because obviously, whether there is that, you know, legislative action, or for sale or things like that, this is going to put a big chill on tick tock in school and higher ed use cases, in part because of this fear of spying, data, privacy, etc. And yet, there's an incredibly strong, tick tock learner community, tick tock, educator community and these partnerships, how is all of this political pressure on the big tech side going to affect the ed tech side? What's your prediction? No.
I mean, my two thoughts on this first, I do think there's going to be I won't call it a ban, I think they will force tick tock to sell its American, you know, portion to an American company, I think that's going to be the what they say I don't think it's just going to be a pure ban, I think they're going to say, You can't do this, as long as this is owned in China and Europe have the Chinese government as a stakeholder, that's the problem. And if that happens, then maybe it won't be as disruptive maybe it'll be that the US version of Tiktok, whatever that looks like. And maybe there's also a European version of Tik Tok continues to have its relationships with Quizlet continues to have, you know, hashtag Tiktok education and all these amazing teachers. And, you know, maybe that would not be as problematic as, as we might expect. The other thing that I'm thinking and it's maybe the opposite, in some ways is that this is yet another plank in the social media, you know, boat that is falling out, I think, you know, if you zoom back a few years, social media was considered, I remember the Arab Spring, you know, social media was considered this incredible innovation. It was like, Oh, everybody was so excited about how was building community and the sort of Zuckerberg rhetoric about you know, it brings people together, it allows people to stay connected. People were really feeling that. And at this point, you know, social media is get moving up the list and public enemy number one, and not undeservedly. So. But this tick tock hearing on the US, we saw Zuckerberg being pulled in front of Congress, we've seen a lot of, you know, we've seen a lot of backlash, and this is an Tiktok, if not the biggest, I think it is already the biggest social network, especially for younger people, you know, it's in the line of fire as well, it just, I think it's the whole concept of what social media is, and what role it has in society, I think is under extreme fire. Something's gotta give, I don't know exactly what it is. I mean, tick tock, either being banned from the US or Europe or being sold would be one piece of it. But I just wonder if we're sort of coming to the end of the age of social media and the beginning of the age of AI. That's sort of how I would read it.
My read on this is that it's an opportunity for the big tech players that have been on the sidelines of social media to get in. So I actually think it's the most likely buyers for tick tock us are a place like Microsoft. If you look at Microsoft, and its big moves. They've been brilliant. I mean, like they were way behind Google. And they are now really surging forward. By the way, they also have incredible experience with social through LinkedIn, which is probably financially the most successful platform social media platform. At this point, given that advertising has really taken a big hit at Facebook and they've kind of pivoted to Metaverse, the other player that I would watch in the space is Amazon, they don't really have a social media property, they do a good job of creating, you know, engagement and then serving up products through their own marketplace. So that would be another potential aspect here. In all of those scenarios, though, I think learning becomes the rounding error yet again, and we ended up getting whatever acquires deign to provide for the EdTech space. It's frustrating when you're in edtech, where you feel like these externalities are constantly, you know, creating distortions in the marketplace. And if you know Tik Tok is gonna come and compete with your stem video platform, you're gonna get crushed, and investors won't invest in you. So I think it just the faster they can get clarity on where tick tock is gonna land I think the better for the education use cases. You know, I hope that combining your point about the age of AI, combining some of these video platforms with AI are both incredibly powerful and incredibly scary. By the way mid journey. Also, today, just this morning, shut down their free trials, because of the amount of abuse that's going on with both photo and video generation. I don't know if you've seen all the pictures of the Pope, when balanced Yaga jackets and writing in Farrar isn't stuff, which I find really funny and hilarious and awesome. But now there's people who are like faking historic events like natural disasters and things like that that happened in the past. I think that there's these platforms are incredibly powerful in their reach. So when you start layering on the AI, I actually think it's like, when AI goes social, that's really when it's going to be, you know, powerful for both good and bad. All right, let's move on to our next topic. What do we have next doubts.
So, you know, we follow GSV investor, Lubin Panfilov pretty closely, he has a really good medium, he has a very thoughtful ideas about the business of edtech. And he put out a new one this week that I really found very interesting, basically talking about, you know, how it had a couple of different ideas, but but one of them is that the public into edtech companies that are out of which there aren't as many as you might expect, are really now trending into sort of two buckets based on their their earnings based on the what they call the rule of x. And basically, you're seeing things like companies like Duolingo and Roblox and Kahoot and Instructure getting very high multiples for their valuations, because they have very high rules of X and rule of x is combining revenue growth rate with EBIT, EBITDA margin. So you sort of put these two business metrics together, and you get a number out of it. And that number is hyper correlated to the valuation. So you have the companies I just named the Duolingo as the Roblox Cahoots having very high high valuations. Duolingo is at a 9x. And then you have companies like Chegg, and Udemy, and Coursera, and to you trending at much lower multiples, because they also have much lower rules of, you know, x components. So it's sort of cracking the code in a really interesting way about what business metrics inside these tech companies, at least the public ones where we know all the numbers are leading to its, you know, its understanding of its future valuation, that I think it's pretty powerful that he also, of course, talks about, you know, co pilots and AI and you know, nobody's writing anything right now without talking that as well. But I thought that this really stood out to me this idea of, of the rule of x being interesting predictor of valuation, at least at this moment, Ben, this is definitely your area more than mine, these sort of business metrics, what did you make of that correlation?
You know, the reality is we were in a top line growth market before. And when we went into this kind of Ed Tech winter or this overall downturn, it was profitability at all costs. And so this analysis by Lubin is really helpful because it actually shows a relationship between both, which is you need to both be profitable and need to be growing. And Duolingo, I think, continues to be the Darling for Lubin and many others. They in 2019, they were doing 71 million in revenue. They are now doing 370 million in revenue as of end of 2022. That's insane growth, and yet, they're also showing incredible profitability. And part of that has to do a little bit under the hood here with product lead growth. And so one of the things that Lubin analyzes too is sales and marketing as a percent of rap. New and dueling goes actually cut their sales and marketing as a percentage of revenue. So they're generating a higher ratio of revenue to sales and marketing spend, then they were in 2019 and 2020. So only 18% of their sales and marketing represented only 18%. So that's really what drives these ad tech valuations. If I am growing very fast, and I'm growing profitably, I'm going to be in the upper right quadrant. And that's where we see Duolingo is the outlier. We also see Instructure, Roblox Kahoot. So these are examples of product led growth. But let's know constructure is a b2b sales platform. So these are not all b2c, you know, product lead, and Kahoot actually has quite a great dual, b2b, b2c, where we really see the challenge is in businesses that have a really high sales spend, which basically means your customer acquisition cost is very high. So Udemy, Coursera, to you check, some of these are direct to consumer. So like Udemy, and Coursera, I would categorize them as direct consumers, although Coursera very large corporate business now and to you, primarily has been selling to schools and universities. So it just goes to show any one business model it that's not really the challenge. It's how much are you paying to acquire customers? And what is the ultimate growth? And I would just say the last thing in this list is really how does generative AI play on all of this? And the hopeful sound for Lubin, you know, kind of, he's talking about there's a new bull market that's coming, it's around AI? Can generative AI actually drive down those customer acquisition costs? Can generative AI help improve both the growth rate and the margin? That's what's getting people really, really excited. I think the downside of that may be these, you know, $370 million, or billion dollar in revenue companies are going to need fewer employees to achieve those magnitudes. So from a employment standpoint, I think we're about to head into the bearish part of the market with all of the the layoffs. So again, the article is called learning with copilots, published by Lubin on March 29, you can read it on his medium site or through LinkedIn. One other thing I'll say is there's an article in the information called don't build the wrong kind of AI business, it has its great picture of somebody putting on AI sandwiches, you know, basically everyone slapping AI onto everything, I think lupins point of view around driving growth, and driving costs down through AI and our earlier point around, you know, particular use cases and user experiences. That's really where the defensibility is in those areas, not in just the pure, you know, generative capability itself,
as well as the safety data compliance privacy, I would add that as one of the things that in an edtech also going to be differentiated the UX, and the interfaces make it actually usable, but also, you know, keep it safe, whether that's a, you know, a private LLM, or a way to take the open API's of the world and make them safe. That is definitely a space as well.
All right, in our last headline before we head to our interview with Qian, my Jews juice in the news again, we've always said when there's smoke, there's fire. Now we're starting to see the fire. But Blackrock this morning announced that they are cutting their valuation of buy juice by half, the billion dollar multibillion dollar edtech company still one of the most unpaper valued edtech companies of all time, has really hit the skids. They're doing a third wave of layoffs across all of their properties. We're seeing folks from multiple us extensions, leaving in droves. And ultimately, it all comes back to not a failed core business strategy, but a questionable m&a strategy that really extended the buy juice team into tons of different areas, including Osmo epic books, a number of tutoring platforms, and their m&a spree also occurred at the height of the market. So they were buying at premium prices. And now with the market gone cold, and with their growth stalling, we're seeing massive, massive cuts. Alex, you know, you've been deep on this. We've been talking to people both in and outside. What do you think the future holds for by Jews? Yeah,
I was, I remember this question coming up maybe six months ago. And I think at the time, my take was, hey, I think by juice is going to find a way to stick around. This is a big news for them, it feels like, you know, we've seen these strange reports of various things happening of, you know, these incremental layoffs of things just not going their way for quite a while. But this feels like a big deal. And you know, buy juice is India's highest valued startup of any kind. It's not just an edtech company. So this could have a real scary chilling effect on the entire Indian startup market, let alone the Indian edtech startup market. And IBM, we've seen GSB, you know, double down and triple down on their aspirations for the Indian tech market. But we've said, you know, I've said this for a long time, if by Jews is the poster child, for Indian Ed Tech, or for for edtech, it's, you know, it's one of the biggest ad texts in the world, the biggest, and it just goes down in flames, which is looks like it's possible. Now, our whole field is going to have to figure out how to make sense of that,
it's a little scary, just to give it a little bit of color on that, they dropped the valuation from basically 22 billion to 11 billion. If you think of all of the other edtech companies and add them all up with the exception of the publicly traded ones. The total valuation is somewhere around 22 billion. I mean, there's, there's like, the big circle. And then there's the all of that tech circle, and they had been about the same size and now having by Jews massively cut down, I think it's a reflection of the overall market and valuations, but it's amplified by some of the m&a pieces. I also would just say the chilling effect, is also going to impact existing VCs, there are a number of ad tech VCs, like you said, GSB, one of them who have basically their funds riding on a return from budgets. And you know, the VC model typically is you invest in 100 companies, and one or two of them return the entire fund that was by Jews for many of these people. And so it's going to be a really tough news. I'm also kind of surprised that Blackrock is doing this so publicly, a really means that Blackrock is not happy are not satisfied with where things are going, because there's no reason that they have to mark it down. 50%. And, you know, the most cynical view would be, it's probably even less valuable than the 50%. If they're marking it down 50. You know, where's the actual IPO price? It might be at 25%. So, you know, going back to loovens Math, it'd be interesting to actually look at the numbers, what is the profitability rate? What's the growth rate? Because it does seem like they're from a gross margin standpoint, they're on the negative side of that. And all of this also, I think, builds on some of the accounting scandals. So there was a lot of scandalous reporting. And you know, Alex and I were not Reporter So we can't validate that. But, you know, six to 12 months ago, there were questionable practices that really raise questions around, were they accurately sharing information about what their companies were doing. And then in one of the rounds, by juice himself, committed 400 million of the dollars of the 100 million dollar round, which is also like a very strange move. And so that kind of smoke combined with this fire is really going to be a big story to watch. And, and by the way, we should be getting some sort of press release from them in the next few days. So listen in next week, and we'll hear about it
one more note on this only a week ago, there was news out of the Asian press saying that bidros was looking to raise 250 million more dollars at the $22 billion valuation and at a flat valuation because that 22 billion was from last March. So that was a week ago and they have all they have you know, $101.2 billion loan and you know, not to call anybody out but Baidu is, is you mentioned they have some big investors. Gonna go ahead and read some of their investors because they have some of the biggest most powerful investment firms in the world behind them. This is Chan Zuckerberg Naspers general Atlantic Tencent out of China, Sequoia, IFC Lightspeed ventures, our Ventures we know from the US, Tiger global, and many more. This is a huge deal for the investment community. So yeah, we're all gonna we'll probably be talking about this a lot in next week as we start to see the fallout on that
sad news. We're gonna wrap the stories for today. If it happens in ed tech, you're gonna hear about it on Ed Tech insiders. And so coming up, we've got an exclusive interview with another newsmaker, a friend of the pod, Qian from work era. Alex, why don't you set the stage
for our special guest on this week in ed tech, we have Qian Catan, feroce, CEO of work era, which just announced a $23.5 million Series B round to grow their company even further and get even deeper into education, you know, skills, education, including AI, welcome to the podcast, Ken, thank you for having me. I'm very happy to be here, Alex. And it's always good to see you.
It's always good to see you too. I always enjoy talking to you. And even more so in this moment where AI is on everybody's mind. So congratulations, he that's a really big series B round, especially at this moment. Tell us a little bit about you know, what you're planning to do with some of this runway,
we're very happy at work here, we, I think we come out of a year and a half of a lot of uncertainty on all fronts. And our mindset was always keep working hard. People need education more than ever, as they're reflecting on their career thinking what to do next. And we wanted to provide that. So the mindset of our team was to keep going. So now with the new funding, we are going to double down on a few product areas. As you know, the product that work here started focused on measuring and mentoring people on AI skills, AI and data skills. First for technical individuals, machine learning engineers, data scientists, data engineers, software engineers, also for any engineer and analysts, you know, and I think I told you once that two thirds of my students at Stanford are not from the computer science major, even if the class is a graduate class in computer science, they're from material science, mechanical, electrical, law, business, medical school. And so those people are also the focus of work, your early days, upskilling in AI, and then everybody else in the workforce, but all focused on data and AI. Now, as we get this new funding, we are expanding into any innovation area, that is top of mind for people out there. So obviously, AI and data remains a big one, you know, software, DevOps, cybersecurity, other innovation topic, I'm thinking sustainability, green technologies, but also power skills or soft skills, behavioral skills, management leadership, which are going to make their way on work care very soon with granular assessments. And we are also working at measuring any skill, I think, this year will be the year where we're Kara really goes from being deep down skills, measurements on AI and data skills to the ability for enterprises to deploy it across the entire workforce. And so that's going to be a very, very cool year, and we will increase our impact.
It's so interesting, because we're Kara was way out in front, of course, when it came to AI and machine learning for these technical individuals. Because just, you know, there's not that many people in the world who know how to assess that kind of skill. Now that you're moving into a whole variety of fields, it's a little more of a red ocean, right? There's other companies that are trying to do data analysis, other companies certainly trying to do power skills. But that said, I bet you still have a lot of advantages, because of your technical expertise and the expertise of your team in building assessments and, you know, validating them at scale quickly, we're using all of these advanced computer science techniques. You want to talk about that a little bit, how are you going to, you know, catch up with and leapfrog other people who have been trying to figure out how to assess things like management skills,
I genuinely think assessments is a is a field that is going to change drastically with generative AI and just with the new language models that are coming up, because skills can be understood both in semantics and with data. So let me break it down for you. If, if I know that you can do two times two equal four. And as a human, I want to interpret what can Alex do beyond that? I will first look at it semantically, I would I would think, what are the things that you may need in order to do that, for example, you may need two plus two equal four. So I'm going to infer semantically that you can also do two plus two equal four. But at the same time, we're Kara has also collected so much data on these two skills, that we understand the cross correlation between those two skills and we can infer whether you have one skill based on the other one. So what's the impact of such technology? It's that we can measure 1000s of skills indirectly by measuring 10s of skills directly. We can accelerate the amount of feedback we can provide to people. And that technology is when you think about it applicable to any skill, you know, it does not need to be only for data and AI skills measurements. The one difference as we expand to new skills, is that the front end, so the signal that comes through when you measure math questions is different than measuring communication questions. You cannot measure a communication only with multiple choice question or recoding questions, right. And so there will need to be a tweak on the front end in order to support all sorts of cognitions. But we are well on their way in in deploying that. So. So that's also very interesting to us,
I can imagine there's just a whole different set of formats may be video submissions may be audio or even just open text writing, but it has to be something that feels quite different and can be analyzed, you know, by the machine learning models in different ways. So I if I'm understanding you correctly, you're saying that you basically have clusters of skills that correlate very highly with each other, and so much data that you can truly see them together. So if you tested an engineer on one particular skill, and they did very well, you can infer through your data, that they probably have the whole cluster of skills, or you know, the adjacent skills, and then sort of narrow down very quickly into what they're doing. Is that what I'm hearing?
Yeah, and what's crazy is there are certain skills that would be logical to you, like if you know, someone can do two times two, you know, they can do two plus two, you can you can infer that yourself, there are certain things that our human brains cannot see, like the knowing two times two equals four actually means so much other things that the person may know, that have nothing to do really with two times two equals four. But with the power of data, you can get that and you can really 10x the amount of feedback you can give someone, and sometimes their mind blown by, well, you measured me on these three things, but you're giving me feedback on these 50 Other things, it's going to change the way we learn, I think.
Sure. It's also interesting, you're mentioning the sort of correlations between skills that might be unexpected. I'm curious about the ones that sort of jumped category like you've been assessing data engineers, and there may be technical skills for data engineers that correlate with being a good manager or communicating clearly. And I'm like, That's blows my mind a little bit. Do you see any of that sort of cross sector, like clusters between skills that are in what we think of as sort of different categories,
we'll be talking soon, a little more about D skills that are sort of unexpectedly correlated with each other, but just you that are things that you can sort of expect, even if it's not immediate, you know, being strong in data science, or statistics actually also makes you typically a more better decision maker. Even if decision making skills are cross functional, like they they may be way beyond the data science function, there is a relationship between those types of skills, coding skills also can translate into some level of structuring your thought process. And there are things that you can see a person who's good at algorithmic coding translate into their debating skills, for example, it's too early to really find all of these cross correlations. But there are skills that infer other skills that are not logically adjacent, let's say or immediate.
It's incredibly interesting. And it has multiple benefits, including the one that you just mentioned, which is that you can assess people on a handful of skills with relatively short time, and then get a wide variety of outputs and predictions and things on all sorts of different skills that that person might have. The last time we spoke was right after Chad GPT sort of came out. And you had a really interesting prediction that came already came true. About two weeks after we talk, which was that open AI was going to start putting out API's, they were going to try to become sort of the embedded AI and all sorts of other functions. And that has very much happened. Ben and I have been talking a little bit about how there's sort of two potential ways this could evolve, or at least to open AI, or some of these big, you know, models could become sort of flowing through everyone, or people can start developing, you know, custom large language models, for particular use cases using open source data or, you know, for particular use cases, and maybe even with relatively small datasets, which is another thing that's incredibly exciting right now, I don't know if I'm even asking this question in a clear way. I'm an AI novice. But I'm curious how you see this moment now that open AI is really sort of finding its way into so many use cases across different industries.
And these things are always hard to predict. But if I had to make a prediction, I would say there is going to be stuff changing in the language models. Space and things that language models are going to enable outside of their space. Within language models, I think the scrutiny on the training of those are going to increase crazily, I feel that there is a lot of interesting work that's going to come out around the royalties of data, copyrighting have data, and in the benefit of the creators whose hard work is going to be compensated in the future, I do believe that's going to happen. And I think a huge category will be created, a lot of people are going to delve deeper into responsible AI even more than before, but in a different way, like you've seen, probably the layoffs that happened in responsible AI groups, I think those are just change of strategies, they don't mean that the companies will stop investing in those areas, they will actually double down on it. It's just that it is now in the hands of every AI creator, to be responsible. And so that function becomes also an enablement function for all engineers, you're also having engineers that are not AI engineers able to do a lot more. And so they're facing a new technology that they oftentimes don't know as much as data scientists know. And so this enablement will be important. And I think that will that will increase. And then the other thing is around benchmarking. One of the the aspects of measuring chat GPT GPT, for GPT, 3.5 53. And other language models like Bart, is that they are now benchmarks on human benchmark. So they take this language model and they make it take the TOEFL, the GRE, the GMAT, the PSAT which are not assessments that have been developed for language models, right, they've been developed for humans. And so we will have to find very interesting ways to benchmark language models, because their capabilities are now very akin to humans. I think there's also a lot of work to be done in that front. And it's going to come up where, you know, today, many companies are going to claim they have the best language models. But who knows, you know, we don't have the measurement capabilities to answer that question at the granular level quite yet. There's great work done at Stanford actually,
always. Bill Gates put out that essay, I think, just this week about how he had challenged the open AI people to beat the AP Bio exam and to ace it, because he said, Look, this is an exam that really takes some real, you know, sensical thinking, you have to go deep and understand. And he thought that was going to take a long time. And they came back and did it in like four months. And he's like, okay, Age of AI has officially begun. And it's such an interesting idea to say, Okay, once they're surpassing standardized tests that humans have created for ourselves, then how do we assess them? How do we know they're getting better and better, if they're already beyond the kind of assessment that we know how to create and validate, I can imagine a world in which they're creating assessments for each other the same way that you know, they play each other in, in games to get better and better and better. And, you know, sky's the limit there. So last question about this round. I love I mean, it's always so much fun to talk to you. You raise this $23 million round at a moment, as you mentioned, very tricky and complicated for edtech. globally. I'd love you to not predict exactly, but tell us a little bit about what you think, was so reassuring about work era for investors that they were willing to jump in at a moment that they're sort of very reticent to jump into a lot of other investments, because it's such an unstable environment. Does that make sense as a question?
Yeah, I feel investors, we look at the team. And I think we have an outstanding team, very multicultural across 22 countries. And we've been able to hire the best people we could find, regardless of where they were. And they would look at the product, and we are the first mover in what we do. So we were very early in betting on skills measurements will be the future of skills, intelligence. And then they talk to customers, we have very large fortune 500 customers that give great feedback on what we do with them. And finally, the market, I think that there is no going back employers are going to double down on their employees. I think today, if you're an employee in a fortune 500, global 2000 company, you're seeing all these hypes and technologies coming up, you're asked you know, you're smart, and you know that your job is going to change to a certain extent you don't know if the exposure is 10% 5% or if it's 50% 60%. But you know, there's going to be an exposure. So you're asking yourself, Is my employer going to invest in me? And in most cases, the answer is going to be yes, the employers are going to invest in their employees, retain their best employees, identify the skilling opportunities and double down on them. So it is possible that we will see less hiring, but more investments in existing employees. And this is all why we're here for we're here to To make employees more productive, make them more skilled at what they do. And as skills are shifting faster and faster, we will need more and more tools to be able to enable those changes. So where the market was going was also aligned with what we were doing. And I think we benefited from that
great answer makes a lot of sense. Huge and very quickly growing market, you have traction, you have great customers and a great team and an idea that was ahead of its time when it started a couple years ago, it still feels ahead of its time, it still feels like this. Very few people who are, you know, have the have the ability to create this type of super validated assessment on these incredibly fast moving skills. It's so exciting. It's always really fun to talk to you here. And I hope we can see each other at ASU GSB in the next couple of weeks. And you have congratulations. Thanks for being with us here on edtech insiders, I'm sure people are incredibly excited about what's next for work area.
Yeah, no, thank you, Alex. I'll see you soon. Couple of weeks. And you know, thank you for having me again. Thanks for listening to this episode of Ed Tech insiders. If you like the podcast, remember to rate it and share it with others in the tech community. For those who want even more Ed Tech Insider subscribe to the free ed tech insiders newsletter on substack.