Chatting with the Machine

as medicines. Hedge funds use it to predict which stocks are going to go up or down. Instagram uses it to predict which adds I'm most likely to click on For the record, the machine has learned that I will often click on ads for overpriced workout clothes. Anyway, if you want to understand what's happening with business and technology today, you really have to understand machine learning. I'm Jacob Goldstein.

This is What's Your Problem, the show where entrepreneurs and engineers talk about how they're going to change the world once they've solved a few problems. My guest today is Luis Vonon, the founder and CEO of Duolingo. Duolingo is both a wildly popular language app and also a hardcore tech company built on machine learning. Luis used to be a professor of computer science at Carnegie Mellon, and in our conversation he was really candid about the technical limits

of what Duolingo can do today. The app is good at teaching people to read and to understand, he said, but Duolingo is not as good at teaching people to speak a new language. And solving that problem turns out to be part of this great, big, interesting frontier problem that is relevant not just for Duolingo, but for the

whole field of artificial intelligence. We started out talking about the origins of Duolingo, which go back to a different problem, one that Louis discovered before he'd ever heard of machine learning. It was a problem he saw all around him when he was growing up in Guatemala. I was fortunate that my mother basically spent essentially her entire net worth on my education, and so I was fortunately that I got

a good education. But then I could see the people who would get public education barely learn how to read and write. This is just what would happen, And you cannot expect that these people are going to become, you know, the CEO of a public company or anything like that,

because they kind of wont. Often people talk about education as an engine for reducing inequality, but what you're describing is the exact opposite, and it's education, when you have to pay for it, is a mechanism for perpetuating inequality. And I really believe that, and I believe that's true

in most countries in the world. There may be some countries, you know, like the Scandinavian countries, where pretty much everybody gets the same education everything, right, Yeah, there may be some countries like that, But in the vast majority of countries, if you have money, you can get a much better education. So I wanted to do something that would give equal access to education to everybody, and so we started with that. But then we started thinking, Okay, if education is pretty general,

let's start by teaching one thing. Eventually we settled on teaching languages for a number of reasons, the biggest one of which is that learning English in particular can completely change people's lives. If you know English, you can double your incompotential in most countries. Wow, it's just as simple as that. And so it's it's why why is that. Basically it opens up for almost any job. You can get a better version of that job. For example, you could be a waiter, or you could be a waiter

at the five star hotel. You could be an executive assistant, or you could be an executive assistant for a multinational ceoyeah yeah, okay, So really teaching English the core, the core sort of reducing inequality dream of a language is really teaching English to people in largely in poorer countries. Yes, so what we wanted to do was teach teach English. But you know, if you're going to teach English, we may as well teach other languages, So teach those and

do so for free. So that's what Luis did and it worked. Today, tens of millions of people use Dual Lingo every month to learn English and dozens of other languages for free. The company makes money by selling ads and premium subscriptions. It went public in twenty twenty one and is currently worth billions of dollars. And the company really is built on machine learning. Luis gave me a few key examples of the way the company uses the technology. So let me tell you a few of the things

that we do. One of the things that we do we like very much is we have data on whenever people use dual Lingo. We record every exercise that they do and whether they got it right or wrong, and if they got it wrong, why they got it wrong. With all of this data, we're able to do certain things with artificial intelligence. For examples, for every exercise that we're about to give you able to predict what is the probability that you're going to get this exercise right

or wrong. So, in a sense, that is a thing that a teacher in a classroom could do fairly easily, right a teacher with twenty students, But you're able to do it with whatever how many people use your app actively forty two million per month, so the machine can do that for all forty two million people at the

same time. More or less, yes, and very accurately. Part of the secret source of Duelingo is that we realized if we were to only give you things that you're not very good at, we'd basically be giving you lessons from hell every time. So we can't do that because that frustrates users. So what we do is, whenever you start a listening and doing we're actually trying to optimize

for two things at the same time. We're trying to teach you things you know that you don't not very good at, but also we're trying to keep you motivated and engaged. Yea. And the way we do that is we try to give you exercises for which we know you have about an eighty percent chance of getting them right. Huh. And have you found that to be the sweet spot? I mean, have you done like experiments and sort of turn the dial. We've done that, and we're you know,

we're not the first to figure this out. I mean, there's a lot of literature and psychology, etc. Just and the number is not exactly eighty percent. It's a little higher than that. It's like eighty three percent or something. But there's there's a number, and it really is the case that if that number is higher, that means these things are a little easier for you. Then you get a little bored. You feel like you're not learning right. If I'm getting ninety five percent right, I'm like, what,

I'm just wasting my time? And you feel bored because it's like it's like a game that you always win. I mean, that's that's nice. At the very beginning, but then you're just not going to play it, and then if it's lower than that, that means that things are too hard for you. You get very frustrated and you go away. And there's a lot of tricks that you know, certainly app developers play, and you know we play as well. So I'll tell you another kind of similar trick. You know,

we end up applying it to language. But the easiest way to understand this trick is with a slot machine. When you get two out of three, it's you almost got it, you gotta do one more, you gotta do one more. You just gotta do one more because you got it. So there's this, there's this you're so close to psychological trick that we played. It's like, oh, there's two out of three, almost got it, But you knew I was going to get two out of three. Yes, sure, you gave me two easy ones and when there was

super hard that's exactly right. So so we we played this type of trick where just people are like, almost got it, and that gets them to do another one. So you know, in our case, we just we basically spend a lot of time training computers to figure out what it is that makes people use Duolingo for longer, and also that we teach them more so that that's a major use for artificial intelligence. Main use is just

in teaching better. After the break a big problem, Louise and dual Lingo are still trying to solve a problem that turns out to be a big frontier problem for all of artificial intelligence. That's the end of the ads. Now we're going back to the show. So let's talk now about problems you haven't solved yet. You know, like, what are you what are you trying to figure out? What are you working on that that isn't quite working yet. So dual linguals is very effective at teaching you all

kinds of things. But if you go look under the hood or you know, what is it that you're learning. You're learning reading really well. You're learning writing pretty well, but not as well as reading. You're learning listening pretty well, but you're not learning spontaneous speaking very well. This, by the way, is also you're not something you're not learning very well in university semesters, Like you're basically not learning

that well either in due lingo or in university semesters. Okay, it's just harder to teach in a sort of classroom. What you need to do to teach that is basically, have you really interact with wealth? For now another human and just you just practice that a lot. Now, here's the here's the thing about that. I know how to get you to interact with another human. Just put another human there. The problem is about eighty percent of our users just does not want to talk to a stranger

in a language that they're not very good. So the problem that we're trying to solve here is how do we practice kind of spontaneous conversation but without having a human on the other side. And we've been working on that, and you know we're not there yet. Can I just interrupt? Because I mean we were talking about artificial intelligence? Right? The most famous test that I know of, the most famous idea I know of of artificial intelligence in a

computer is can a computer hold one end of a conversation? Right? Like? That's the classic touring test is like you're going to have this like chat conversation and can you tell if the person on the other end is a person or a machine? Like? That's the og artificial intelligence idea, right, I mean, are you telling me, that's what you're trying to solve. Not quite. I mean, it would be awesome if we would solve that. I mean, but that's the dream,

right solution to that, that is the dream. But notice in our case, we don't actually care if the human can tell that there's a computer on the other side. Okay, it's okay. As long as it practices thing and as long as you're able to carry on a conversation in a way that seems a little natural or something, it's okay. If if it, you know, goes off the rails every now and then, So tell me, what is it that you're trying to build. This is exciting, Like what are

you trying to do? We're starting with text, by the way, so either just basically a texting conversation. So think of it as like a chat bot in uh, you know, in Spanish, where it just you're just having a real a little conversation. You've had lots of people have had experience with chat bots. Yes, they're right, Like you go to whatever, cancel your cable and they want you to text, and then you realize you're texting with the machine. So like that's the that's step one. So that's the idea.

That's step one. We of course, I mean a lot of those experiences with chatbots are are very um, they're just very geared at whatever it is you're trying to do. So, for example, that chapel maybe very good at at canceling your cable, but only that in my experience, they're not even good at that. No, they're not that great. So we're trying to do that, and you know, we're not there yet. I don't think so you're like out on the frontier. We are. We are like you're trying to, yes,

and we're not there yet. I mean, this is something that's going to take us, not just us, I mean the whole academic community and technology just a few more years. But so let me ask you this. Can we talk about that in a way that would be like, can we try and just go one level into sort of what you're trying to do and like what works and

what doesn't work, and like why it's hard? Yeah, I mean, you know, the first, by the ways, the first way you think of if you're trying to make a chap the first thing you think of is, okay, I'm just going to program the computer. Forget about artificial intelligence. I'm just going to program the computer to respond to specific questions, and how many possible questions could there be? You start thinking, okay, well, when the person says high, we're going to program a

think to say hi back. When the person says, how are you doing, We're going to program I think to say I'm doing pretty well? How about you? Yeah? V zero of a chatbot. And this, you know, this comes from you know, fifty years ago. This is what you start doing. The problem is there's billions of things that people can say, and so we may have programmed the thing of what to say, but you know how you're doing,

and we can respond. But if instead of asking that, they may ask like, hey, did you watch the game last night? And we just have no idea how to respond to that. About a decade ago, Louis says Ai, researchers started trying a really different approach. Rather than trying to teach computers every rule, they started throwing massive amounts of documents and texts at computers and essentially telling the

computers figure out the patterns in all these documents. So when somebody writes something like did you watch the game last night? The computers should be able to predict what kinds of answers might follow. This strategy clearly has not entirely worked yet. That's why it's still a problem solving. It will take both more text and more clever algorithms to help computers make sense of that text. But Louis says, you can see progress every time you open your Gmail

or a Google Doc. And I don't know if you've used, for example, you use Google Docs lately or Gmail like it finishes off your sentences now. And basically the way this works is, you know, this system has looked at a ton ton of text that has been written by a lot of people. In the case of Google Docs, I actually don't know what they look at, but I wouldn't be surprised if they look at everything that has

ever been written in Google Docs. I'm going to tell you one that happened to me in Google Docs today when I was typing notes for this interview, I typed zone of pr and then you know what, you know how it completed it proximal development. Yes, it knew I was going to write zone of proximal development. Yep. No, this is amazing, And they just see that if you'd write the zone of per there's like a ninety five percent chance that it ends in proximal development. What is

the zone of proximal development? You know? In teaching, you know, there's this concept of just keeping you at this zone of proximal development, which is always kind of challenging you, giving you things that you don't know. But but there are all things that are fair to give you. Proximal means like close to or next two. Right. So it's the idea is like you know a thing, yes, then like what you want to teach the person is the very next thing, right, It's like that's right, It's like

the frontier of your knowledge. I like it because it applies to like the way you teach, but also to your work, right, And like it's just a nice life idea, right, It's like the next thing you want, the next thing. And I feel like the chat bot is maybe a version of that at the level of your company. Yeah, yeah, it really is. It really is a nice idea, and it is I mean, and if you think about it, this is what a great teacher does. You know. I've

said this inside the company at due Lingo. UM. All we need to do is first figure out what you know, by the way, not that easy to figure out what you know. But let's first figure out what you know and then just take you to that zone of proximal development. Because now we know what you know, just take you to the frontier and then just keep expanding it as fast as possible. That's all we need to do. Of course, this is easily said, hard to do. Yeah. Um, And is there a limit to what you can do with

a computer? Is there anything a teacher can do? Is that a computer will never be able to do? You know? Of course I do lingual We love teachers. If they are a good teacher and also have the time, they are much more able to adapt to their students than a computer is. Um. But I don't believe that will always be the case. I mean, I think at some point it's not just teachers. I mean teachers, this is one thing. I mean, at some point my belief and

this is of course just my belief. People, not everybody agrees. They believe that computers will be able to do every single thing that humans can. Now you may start asking really tough questions like can they love? Yeah, I don't know, I don't know what they can love or not but from the outside it will look just as if they love so who knows who knows what's going on inside?

Who knows that they that's like a big yeah, we're big philosophical questions that I'm not here today, and that's right, nor am I. But I do think from input output behavior, I don't see why. I don't see any reason why computers won't be able to do everything that humans can. So they can teach, but they can also write a computer code. They can also run companies, they can also make podcasts, they can do everything. Should be able to do that. I think they should be able to do that.

I don't know when that'll happen, but they should be able to do that. In a minute, the lightning round, well, hear what job Luise would love to do but thinks he wouldn't be very good at. And the real reason treasure Chess keep showing up in duo lingo. And now back to the show. We're going to finish with a lightning round, not counting duo Lingo. What's your favorite app on your phone? Spotify? What have you been listening to on Spotify? I'm always a huge fan of the band

called Churches with a v to Virchase. So that's what I was listening to this morning on my work at work. If you have a ten minute break in the middle of the day, what do you do to relax? Played this game called Class Royale. We are a lot of the gaming mechanics that we use for duel and will come from gaming companies, like the treasure chests, exactly right, the treasure chests. If you ever played Class Royale, they

have the treasure chests. If somebody's going to go to visit Guatemala for the first time, what's one thing they should definitely do? Oh? Um, Decal is the Mayan ruins. Um. You know, if I feel very strong, I've been to southern Mexico where they have chi Chenitsa. It's a joke compared to the Mayan ruins in Guada. There's there's one pyramid in Chichenitsa. There are four hundred in Guatemala in Decal,

So yeah, they should like I like that. Not only are you recommending Tikal, you're also taking as I have no trouble with Chichenitsa. It's just they are very good at marketing. Amazing. What would you do if you couldn't do the job you do. Now, well there's what what would I actually do? On? What would I'd like to do? I would love to be a writer. I don't think

i'd be a very good one. Um So if I if I wasn't doing the job that I'm doing right now, you know, I'd probably be back to being a professor. How will you know when it's time to retire? I'm never retiring, That's what everybody says. Well, maybe I will, but I mean right now, I don't. I don't want to do that. Luis Vaughan is the founder and CEO of Duelingo. Today's show was produced by Edith Russelo. It was edited by Kate Parkinson Morgan and Robert Smith, and

it was engineered by Amanda kay Wong. Theme music by Louis Kara. Our development team is Lee, Tom Mulad and Justine Lang. A huge team of people makes What's Your Problem possible. That team includes, but is not limited to, Jacob Weisberg, Mia Lobel, Heather Fain, John Schnars, Kerry Brodie, Carli mcgleory, Christina Sullivan, Jason Gambrell, Brand Hayes, Eric Sandler, Maggie Taylor, Morgan Ratner, Nicolemrano, Mary Beth Smith, Royston Deserve,

Maya Kanig, Daniello, Lakhan, Kazia Tan and David Clever. What's Your Problem is a co production of Pushkin Industries and iHeartMedia. To find more Pushkin podcast Us, listen on the iHeartRadio app, Apple Podcasts, or wherever. I'm Jacob Goldstein and I'll be back next week with another episode of What's Your Problem.