Artificial vs. Human Intelligence with Mathematician/Author/Edtech Head of Product Junaid Mubeen

turned educator. He spent over a decade working on innovative learning technologies reaching students of all ages and abilities from around the world as head of product and Director of Education at whiz education and CEO of right the world. Junaid has a de fille, which is like a PhD in the UK in mathematics from Oxford and a master's in education from Harvard where he studied as a Kennedy scholar in international

education policy. Junaid is also the author of mathematical intelligence, a story of human superiority over machines, and is currently working with best selling science author Simon Singh on developing the world's largest online Math Circle parallel.org.uk He also earned fleeting fame as a winner on the game show series countdown Junaid Mubeen Welcome to EdTech insiders.

math enthusiast at school. And so it wasn't a huge surprise that I studied maths at university here, I ended up doing a PhD in math, and then got to the end of that and realize that I had two choices in front of me either try to become a fully fledged mathematician, or try to do something a bit more practical with my life. And for various reasons I chose the latter, I should say one of those reasons is that, I think to be a full time mathematician is incredibly tough. So that was one reason.

But also, I did always have this gnawing feeling that I really should be putting my skills to some practical use. And I came from a very humble background and working class background where the opportunities that I had weren't always extended to people in my communities, I always want to do something about that. I applied for a scholarship to the US to do a master's in education at Harvard. And I couldn't quite believe it when they went when

they offered me one. Because the idea of doing a master's in education after completing a PhD in math is a slightly novel idea. But they bought into it. And so I guess I did too. And then I had the chance to sort of continue my academic studies for another year. So it was eight years of studying method in higher education, and one more year of the masters. And then it

was time to get a job. And I started out in that master's program, it was international education policy at Harvard, and the Graduate School of Education there, I think it says it's at the nexus of policy practice and research. And in going in, I thought I would emerge with a landing and in a policy role, it felt that that's what made the most sense, but this was around 2011 2012. And there was just a lot happening in edtech. At the time, innovation was being seen in various different pockets.

And I was inspired by examples like Khan Academy and Dreambox, especially especially looking at some of the things they were doing in the area of math. And so that piqued my interest. And I got involved in a couple of startups. One was right the world which was a social writing platform, so not math related, but just an opportunity to get

my foot into edtech. When I returned to the UK, in 2013, I found a company called wares education that was based in London, which was easy enough to commute to, and they were doing work that was very similar to Khan Academy and very much focused on math and adaptive

tutoring. And it was just a lovely opportunity to, on the one hand exploit these analytical skills that I've developed over the years and math but to apply them in an educational context where hopefully I could actually help kids to experience math in a more positive way. And that really set the tone for the last 10 years or so. I was at West for about seven years. And over

the past couple of years. I started to work with Simon Singh, the best selling author along the way I wrote a book myself and that's It's in that context that I met Simon Amen. But the underlying thread is the same trying to use my mathematical background to help the next generation not only get better at math, but to actually enjoy it and to remove this horrible element of math anxiety while we're at it.

a better place. And I think you're a prime example of that you, as you mentioned, math, PhD from Oxford, international education policy from from Harvard Graduate School of

Education. And, of course, you had the opportunity to do policy, but instead, then spent years making whiz and write the world into, you know, incredibly powerful tools to support the next generation, I want to ask a little bit about what it's like to come into edtech from this particular background, because I don't think it's that common, you know, you when you're talking about Khan Academy, and Wiz, you're clearly not teaching math nearly at the level that you were studying it at, as as a

PhD student. But you have such a deep mathematical intelligence, as your book talks about what is it like taking the sort of advanced education that you get at institutions like that, and translating it to really sophisticated online education for much younger students,

define good pedagogy. And so I did find myself often having to remind myself that I wasn't looking for a proof. As a mathematician, you're looking for a gold standard proof that meets every standard of rigor, and logic. And that standard just isn't so relevant in education, obviously, there's much to be said, for taking an evidence based approach. But a lot of it is just informed guesswork, and you have to be quite prepared to rely on your instincts and to rely heavily on

human experience as well. I think as time has gone on, I've realized that actually that mathematics relies on a lot of that as well. But certainly within education, that there's no denying that when the rubber hits the road, and you find yourself confronted with the reality of a classroom, when you're interfacing directly with a student, or when you're showing your products to a teacher, many of your high level assumptions just just don't hold

up. And so I find myself wanting to create these wonderful algorithms or wanting to mine data in a specific way. Because it you know, it, that's, that's what came naturally to me. But then you often have to rein yourself in and ask what's actually going to have impact on the brand. And it's not necessarily the most technically glamorous creation that you can conceive quite often. This is especially true of adaptive tutoring and the kind of work

that we did there. There's a lot of very sophisticated research that went on in the background. But quite often, the conclusion was actually simple is better. It's much better to develop these algorithms in a way that's very transparent, that actually just behave in very common sense ways. And so even though we can have a lot of fun doing things like machine learning, and big data type stuff, we've got to be

aware of our intentions. And while that stuff can be fun, it's not necessarily what's going to make the difference for children's education. So it's definitely been a learning journey, trying to figure out that balance.

getting to the very root. You mentioned that the role of intuition in this and I think it's a really interesting segue here. Because, you know, in your book about mathematical intelligence, you make the case that machine learning and AI cannot necessarily compare with human intelligence, especially when it comes to these intuitive

mathematical breakthroughs. And, you know, I'd love to dig into this because, you know, we're at this moment of enormous AI acceleration, we just saw Khan Academy, you know, launch a tutoring system based on chat GPT four. And it may sound a little counterintuitive to those who associate AI with computation with very advanced, you know, numeric calculation or with scientific breakthroughs.

Tell us a little bit about your thesis about mathematical intelligence, and why humans continue to outperform artificial intelligence in it.

particular. So the first I'll call school math, that is the kind of math that we're all we've all experienced at school and and even though there's variations across different countries and from one generation to the next by and large school math is rooted in what I would call a procedural paradigm, it's very heavily rooted in calculations and learning very specific routine methods and procedures, algorithms, formulas that were then expected to memorize, not always with much conceptual

understanding, and then recall and rehearse when an exam comes around. It's no surprise to me that so many people are then afflicted with math, anxiety, and it's the one subject where people declare so readily that they can't do math. You never hear that in any of the subjects. But then there's this other brand of mathematics, if you like, which I fell in love with which mathematicians experience everyday and which I believe can be made more

accessible to the mainstream. So the entire thrust of my work is trying to figure out that journey from novice to expert, and how we can capture some of the magic that mathematicians experience and actually infuse that into mainstream education. And actually, the core of it is this idea that we're actually born as natural mathematicians. If you think of mathematics, beyond just routine calculations, think of it in more creative and open ended

terms. Think of it as a subject that rewards inquisitiveness and the asking of, of interesting questions, or that are subject that encourage us not only to learn rules, but to also break them occasionally just to see the consequences. Well, these are things that come very naturally to young children, but are somehow educated out of us

through our formal schooling. So for me, this book was an attempt to just articulate these elements of mathematical intelligence that come naturally to us as humans on the one hand, and that aren't so easy to automate. Now, I should say the book was published about a year ago, and a year is a lifetime in

AI. And we're seeing every week new developments at breakneck speed we've just referenced GPT four, which is just a few days old, in terms of its its release, and who knows what else is, is coming through the pipeline. And I do pay a lot of attention to these tools, because it may well force us to

examine our assumptions. Now in the book, I outlined several aspects of mathematical intelligence, like rule breaking, which I call imagination, or questioning, which is just another term for curiosity, I guess, which I think is currently lacking in these systems. And it's important to say that AI at the moment is dominated by one particular approach, which is

large language models. And for all of their emerging capabilities, it's not yet clear that they're that these systems will will acquire those particular skills. So I think my thesis is safe for a while. But I do caveat in the book that I'm not making any fundamentalist claims. I'm not suggesting that computers of the future, perhaps design in a more pluralistic way than today's large language models might eventually be able

to emulate those skills. And so to the skeptics, and the fervent enthusiasts of AI, I would simply hold these aspects of mathematical intelligence up as just a standard bearer for the kind of intelligence that we should remind ourselves of. And what we should also recognize is that just because these systems can now pass the SAT or the bar, and answer various math questions, they're not those papers, the kinds of questions they asked, they're not reflective of our most ingenious

fact, mental faculties. And it may force us as educators to examine whether we're expecting the right things of students, if computers are already able to eat into a lot of the skills that we're judging students on, that's really

younger. And I remember, you know, there being this moment in high school math, and it often came around the quadratic formula, where it was like, for many students, that's the first formula that is just so big for them that it's hard to remember, it's hard to apply. And they don't have any conceptual understanding of why it works underneath the hood. And I think it throws many people for this

enormous loop. As they're learning math, it takes some of as you say, it educate some of the joy out of them by saying, oh, instead of finding roles, instead of discovering how this works, and experimenting and playing with it, I have to memorize this thing that has all of these letters in it and plus minus and exponents. And this is just a memorization exercise. I'm memorizing and applying. That's not what I want to do.

And no other to your point. No other subject really asked me to do that you're not memorizing any formulas in English or history. And I think it really, really confuses people. I'd love to hear you talk a little bit more about the sort of math anxiety and how you feel that you know that the current system educates math joy out of our current generation. I think you've thought a lot about this.

internet. And ad tech is just an explosion of interest in how you can take these seemingly arcane concepts that feel very abstract, and actually just bring them to life. So then if you know the grant Sanderson three blue one brown YouTube channel, he does this very advanced math concepts. And I mean, my goodness, it's, it's mind blowing. You know, as a seasoned mathematician, I've learned so many new things, what I've learned is how to see familiar concepts in completely new ways that really

strengthened my intuitions. And a lot of this is about taking symbolic representations and making them more visual, making them more pictorial making them more concrete, when we expose

very young children to math. So my children are both below the age of five, my four year old daughter when she experiences math, at nursery and around the house, it's very tactile, I want her to physically experience numbers and shapes, we've got pattern blocks, we've got various representations of numbers, so that she can recognize three in as many ways as possible. And I think that that does capture the joy and creative energy, that math

brings. Somewhere along the way, we replace all of that and rush to symbols, but symbols should only ever be there as a as a representation after the fact after you've actually grasped the concept. And then it's just there as a shorthand, to help you communicate ideas. And unfortunately, what we've allowed is for this symbol barrier to emerge, where you have to now master the symbols, before you even gain an intuition of these ideas. So take the quadratic formula, for

example. And I'll mention two things on that specific one. The first is that many people don't realize that the quadratic formula, and the method known as completing the square are actually one. And the same thing. If you've ever taken the time to think about where the quadratic formula comes from, you know, a little proof. You know, it's quite taxing, because you've got to negotiate all the symbols, but actually, it is just the most generalized application of completing the

square. And so there are people that say, Oh, I know how to complete the square. But don't get me started with the quadratic formula, or vice versa. And I think, really, you've never been taught how to join the dots on that you've just been taught these two methods as if they're so

separate. If somebody actually took the time to visualize the quadratic formula, and help you to actually deconstruct the symbols and think of them more visually, you'd realize why it's a, it's not just a tennis equivalent is identical, it's exactly the same as completing

the square. And then the second reflection I have a complete from the square is when you learn what about quadratic equations for the first time, you're told that that bit in the square root of b squared minus four AC, the determinant has to be or the discriminant, whatever it's called, has to be a non negative because you're not allowed to take the square root of negative numbers. And I remember reading that in a

textbook. And then it said almost cryptically, it said, although advanced, mathematicians have figured out a way to do this, I felt so shortchanged that they're telling me that actually, you can take the square root of negative, you're not going to elaborate. And then I would learn about imaginary numbers much later. And this was before the internet before I could do a quick Google search or a Bing

search. Whereas now, I think we should be encouraging students even as we're teaching them, these these methods, we should be teaching them that the real point of mathematics isn't to absorb a fixed set of rules. But to get familiar enough with a set of rules that we also understands what it means to step outside of that rule set,

and to break them up. So it's all well and good say that to solve quadratic equations, we're going to only allow the square root of non negative numbers, because we want real number solutions fine. But what happens when we do then allow ourselves to take the square root of negatives, or now you open up a whole world of imaginary numbers and complex numbers, where every quadratic equation, even the equation x squared plus one

equals zero, has a solution. And it's just that mindset that says this, you know, I have agency here to go beyond what what is prescribed. And if I'm learning a set of rules, it's not because my entire worldview is going to be shaped by this. But because I recognize it as one particular set of systems through which to tackle problems. And that's what mathematics is, for me. It's giving us the agency to mess around to play with rule, something that comes very

naturally to young kids. And that actually comes naturally to the most creative people. And we mustn't forget that that is at the heart of mathematical thinking.

interesting. I'd love to dive into this role breaking idea even more, because I think it's so key. We're having this conversation as you just mentioned, a few days is after the launch of chat GPT. For we've seen Duolingo and Khan Academy embrace these Quizlet has launched AI based tutoring system based on its already Onyx data. And there's just this amazing moment where every edtech tool, every, you know, technology tool is starting to

look for ways to use AI. I want to ask a general and specific questions. But let's ask, we'll start with a specific, you know, as you mentioned, rule breaking. I'm a dilettante in AI. But I've worked with a number of people

who are brilliant at it. And I know that there's this concept of introducing small amounts of randomness into the system, to try to force artificial intelligence to think differently, to actually not always follow the rules and, and try to, you know, break through and think of things in a little different way. But I don't know

if it's really working. I'm curious, as somebody who knows, you know, math and AI quite well, do you think there is the potential to teach AI through sort of tweaking the parameters to break through and think about things like, what if there were imaginary numbers,

as a human. But you also have to have a grounding to then examine the consequences of your choices, because there are times where you'll break the rules. And you just won't get very far. Whereas, you know, taking the square root of negative numbers, it turns out was was a great boon for mathematicians, and actually many real world application as well. Now, what I've seen so far is that these large language models are not very good at Common Sense math.

So I had a got myself into an exchange with Bing the other day, because we were disputing the notion of place value, I had a decimal like 12.37. And because I was asking Bing to behave like an adaptive tutor, because I wanted to see just how effective it could have been, and it did reasonably well. But then we got into this debate about what the seven represents, is it the hundreds, or is it the tents and being which was

unrelenting. So I'm skeptical that right now, large language models can be trusted to deviate from a set of rules and make sense of their own choices, what I think they can do, which we're seeing, we're seeing a lot of this in generative AI in text, the image software, in particular, when you add that bit of randomness, it will spit something out, it has no sense of what is actually produced, and whether it's meaningful or not, but put that back in the hands of a human, and we can be

empowered to make those judgments. So I do think these tools that can have incredible augmenting power, I came across another example yesterday, which was I use a tool called Lex I've been looking at Lex. Anyway, it's, it's an AI writing assistant. And now it's promising to write first drafts for you. And now it's promising to just add that bit of creativity by adding a bit of randomness. But I do think there's more to creativity than randomness. I think creativity

is also about selection. And, you know, I think part of it is about just giving ideas, a space to ruminate in our minds, and almost letting go of the process and then seeing what surfaces. But once those ideas surface, we then have to make very conscious choices about which ones to run with. And I don't think it's it can be characterized by randomness alone. And I think what the creativity that's being offered by large language models in this context is really just a

function of randomness. And what it needs to pair that with is a common sense, grounding in whether those choices make sense, we bring that at the moment as humans, I guess systems could develop to grapple with such ideas. It doesn't seem to be in the modus operandi verbalize language models. But you know, combine that with a more kind of hard wiring of, you know, what makes sense in this world. And I could see it happening. Yeah. But that's why

a hold up. Imagination is one of the standards for AI to clear can they not only solve complex math problems to which the solution is no not only solve problems that humans haven't yet solved, but actually conceive of entirely new and meaningful branches of mathematics, because that's how methods always evolved. We've broken rules, we've invented new branches in the process, and that, to me, signals the highest form of creativity. And in the sciences, this is called a paradigm shift.

We look at disruptions across across the arts, how pioneers usher in new genres by completely violating old rules. But they've also got to be able to make sense of their choices and communicate that to the world. And that's, that's the part I don't see machines doing right now.

was super excited. I'm a big you know, I was a big Monty Python fan and still, and I went to see him and he spent the entire talk talking about this book by a Guy Claxton called harebrained tortoise mind and the entire premise of the book and he was so excited about it, as it median, as you know, as an author was that your brain thinks in a certain way, you know, there's a certain Quick mode to your mind, which tries to solve things very quickly

with heuristics. And then there's the deeper tortoise mind that slowly puts things together in completely new ways. And going deep into the tortoise mind is how you actually get new ideas. And it's just, I mean, it's a metaphor, but it's a metaphor that I think speaks very directly to the exact type of you know, two different types of math you're thinking about.

And when you mentioned the idea that AI can use randomness to generate new ideas, but that has no capacity to understand whether there's new ideas have any value. You mentioned, that concept of augmenting, and this is something that's very, very, you know, in vogue right now, because we're all trying to figure it out this idea of AI co pilots, or, you know, rather than full auto pilots, where AI does everything for you, co pilots, where AI and human intelligence can work together

to really create new things. And I love your metaphor of, you know, AI could spit out 100 random ideas, but you may need somebody to sit and look at them and dwell on them and say, you know, that one is really the one that actually has any capacity to mean anything. I'd love to ask you about this, you make the point in the book, specifically that this combination of AI and human intelligence may be more sophisticated than either type alone. Tell us about that

theory? And how do you envision a world in which human mathematicians will be using AI copilots to guide them to these paradigm shifts?

calculations. But they it's just, it's we're just not hardwired to do it. Because that side of mathematics is more like acquiring a second language, it's, it just doesn't come so naturally to us. And so it's natural than that. we've innovated along the way and found these tools that alleviate us of that mental burden. And that was true of the abacus. That was true of the slide rule, it was true of the pocket

calculator. And so in that characterization, I see AI as a continuation, albeit an exponential continuation of that same paradigm, which is, if we can outsource particular aspects of cognition, it frees us up to do what we do best. Now mathematicians have made incredible use of these tools. They made incredible use of calculators for the past several

decades. And there may be a thought, well, that well, AI will reach a tipping point where actually it will take over all aspects of cognition, and there'll be nothing left for human machine mathematicians to do. And I certainly think we're gonna have to work harder as time goes on to identify the aspects of mathematical intelligence that are our unique preserve, because there was a time where calculation itself would have cut the mustard, right? The original computers were human, but we didn't

struggle. Or I should say mathematicians did not struggle to adapt, when calculators went mainstream. In fact, they saw them as an opportunity to supercharge their work, I would say education hasn't really adapted to the potential of calculators, it's still rooted in a paradigm that says, We have to train humans as calculators. But I see human mathematicians adopt two attitudes towards AI.

The first is, it's not relevant to me, because the ideas I'm working with are so abstract and removed from the very kind of blunt pattern recognition systems that these tools are based on, that it just doesn't bear on my work. But there was an interesting example, about a year ago, it was a collaboration between a small number of pure mathematicians who work in very

abstract ideas. And they worked with DeepMind, the same company that was behind AlphaGo and Alpha zero, which defeated the world's best Go player and has done lots of amazing things in protein folding and drug discovery. They basically worked together to see if they could apply the same approach to help those mathematicians move forward their research, and they got a very positive answer. So what they did was, the mathematicians train the labeled a large dataset, about 3 million

items in this dataset. And they were looking at these very abstract objects called knots just as like, you know, the same knots that you tie your laces into, but other mathematical kind. The only difference with a mathematical knot is that the two ends are tied together. And so you're interested in the different types of shapes that these knots can take. It sounds very simple, but it can very quickly become quite complex and becomes hard to visualize these

things. But the mathematicians have some ideas of, you know what kinds of relationships might exist between these knots. Once they had that data set label. They handed it over to the deep learning algorithm and it came back and said, Here are the patterns and it was for the cue mathematicians to then sift through those patterns and make sense of them. And actually, they were able to then go back and check their own intuitions and actually see which ones were right and which ones needed to

be refined. And out of that came a hot came whole new theorems that they say they would not have been able to prove otherwise, unless somebody had alerted them to those patterns, these patterns that were so subtle, that it took a deep learning program to find them. And to me, that is just augmentation in practice, right?

Even mathematicians at the forefront of research, seizing upon these pattern recognition tools, each one doing what they do best, it will be interesting over time to see if AI goes beyond just finding patterns in data, and actually gets into this whole enterprise of learning how to do proofs. And there's there's one school of thought that says, well just train them on, just feed them

lots of proof than that. And then they will learn how to construct proof, they will learn about inference and reasoning and logic and deduction. There's another school of thought that I subscribe to that says they will they can learn to do that to an extent. But they're rare. This all sounds quite mystical, but there's almost X factor to mathematical discovery. And some people call it abductive. inference, it's it's where you make this connection between two

ideas. But you can't fully explain where it comes from. It's not that you're just working through a series of logical deductions in step. There's a kind of novelty to it. And maybe it's hopeful to say that's the unique preserve of humans. But certainly, I don't see any evidence right now that large language models are going

to eat into that. But I guess, I guess, at the moment, this conversation is situated in terms of whether there'll be a profession for mathematicians, but the message I think, for everyone else, including myself, right, I should say, because I'm no longer a professional mathematician, is we too, should be thinking about the aspects of mathematical thinking, that play to our strengths that can't be so easily automated, that will stand that actually puts us in a position to work with these

tools to scrutinize them, and to make sure we get the best out of them in the long run. And to be sure, just crunching numbers and performing calculations religiously isn't going to do that on its own.

that's very different. That sort of blackbox of machine learning may be very different than the abductive sort of mysteries that happen at the depth of the human mind. And you know, one of the things I find incredibly interesting about your research and your book, and just how you think about math is that it has so much shared with creativity. I mean, you're really looking at mathematics as a creative act, which a lot of people don't

always see it that way. Because they don't, they don't, it doesn't really get that to be creative until you get it to relatively high levels. But I'm curious if that abductive concept also would apply to you know, what traditionally called creative endeavors in humanity, like Art and Writing and, you know, historical theory, when somebody has an idea to do a Shakespeare play or to do a, you know, a school for wizards or to do a, you know, the Peanuts cartoon. And it changes, you

know, culture. That is, I would consider that some version of abductive reasoning as well, you know, a dog that lives on top of his dog house and pretends he's a fighter pilot, that is extremely random set of ideas, but you put them together, recognize the value, and then you can create a character that, you know, 100 years of student of kids will remember. And I'd love to hear you sort of just dive even deeper into this idea of what is this abductive you know, essence in humanity? And

can we protect it? Or is it something that is unique to humans? Or do you do we really think that machine learning might catch up and start to be able to recognize creative juxtapositions in this new way? It's such an interesting concept. There's

different ways. And so I I've just observed that you know, kids have this professional mathematicians and people use maths in their everyday lives have this and you know, I met them fuzziness will also speak to the creativity that their subject requires. And then there's this huge, sort of soggy middle of it. will have to endure math at school and are somehow convinced that there's no art to it. It's just a rigid

set of rules to be learned. And of course that comes from the curriculum the way they're assessed and so on. And so the entire thrust of my work is trying to bridge that gap to try to bring that creative essence of math into into mainstream schooling easier said than done. But I think one of the ways to do it is to remove this false dichotomy that is so often proposed between math and the humanities, I think I'd like to write a book one day, or maybe

now I'll just get GPT. To write that just demonstrates all the different ways in which math intersects and interacts with the humanities, there is a psychology to mathematics, there is an intense philosophy that undergirds mathematical inquiry to there is so much ambiguity that mathematicians have to work with, I mean, you know, what is the dilemma of any creative is, they don't quite know the answer they're seeking, they don't quite have the rules written for

them. Most of my time as a mathematician was spent trying to figure out the right question to ask, let alone find the answer. So I just felt a certain empathy with the creative struggle. And I've tried to preserve that even though I don't do mathematics for a living anymore. When I use math in my everyday life, I try to, firstly outsource the stuff that is fully specified and easily automated, and leave myself and then grapple with the unknown.

So in an educational context that could be trying to figure out how do we define and evaluate student learning outcomes? Because it's very easy to crunch the numbers once you've got the metrics. But how do we actually define those metrics in the first place? So I do think mathematics at its core, is rooted in that ambiguity. And we will do well to embraces overlap with the

arts and the humanities. And there is so much more to be said on this, you know, the number of artists that are inspired by math and mathematicians who say that, you know, they, they derive their inspiration from seeing applications, in music, and in artwork, and so on. It's very hard. You know, when you when you see math beyond just routine calculations and numbers, it's very hard not to

see math everywhere. Because if you see math as the science of patterns, and not just finding patterns, but making sense of them, distinguishing meaningful ones from misleading ones, everything, every dialogue, you have every conversation, trying to connect things that you're discussing, at home with conversations you've had at work with things that you've seen on the news, with things that you'll sit you're being exposed to on social media, everything just becomes a mathematical

dialogue in one's mind as you're trying to piece together all the dots. I see children doing it, as I say all the time. And I just wish we had more of that kind of willingness to just allow students to just breathe and lean into that more creative and playful aspect of the subject. And then to the question of whether computers can come up with these unexpected just juxtapositions? Well, I think last year was the tipping point for generative AI. And I think, in the chat bots

really caught our attention. But so did text to image software. So you've probably played with the likes of mid journey, Dolly, stability AI, and I think we were seeing for the first time images that we couldn't quite believe were computer generated, and we're going to see more of that. Now, there is one argument that says, well, in the end, that is just latent rendering of human artwork, because ultimately, these systems are bootstrapped by human generated

data. And so it's the human artists that deserve the credit for the these renderings, but I think we should give computers their due to by recognizing that, you know, with the right prompts, within a fine tuning, they are able to produce things that frankly, we probably wouldn't otherwise see, at the very least, they are iterations of familiar themes that we might, you know, they're like sequels that we might otherwise not have been exposed to, it will be interesting to see if a

form of art emerges that it seemed to be like a complete paradigm shift, or a form of write a genre of writing that GPT or whatever produces that we say, is completely unrecognizable, because it's one thing to say, you know, give me a response in the style of, you know, Snoop Dogg or Shakespeare, but quite another for it, say, actually, you know, Here's a rendering and, and is judged not only on the basis that it it's human like, but actually it's human like but can't be traced

back to in a familiar work. And that would represent a form of creativity that I think we haven't yet seen from these systems. Maybe we'll get there. My feeling is that will ultimately get there through augmentation. It will be, you know, semi competent human writers whose raw skill is with writing prompts and editing, that are able to use these tools, and then generate text and maybe repurpose it to produce stuff that's really quite mind blowing. And we'll probably see the same with

Arthur as well. But yeah, we are was up to keep an open mind to I think I have to confess plainly, there are things that Genet that we're seeing in generative AI that I wasn't expecting to see so soon. And I think anyone who says they were completely unsurprised by this as lying.

same world. And so I can imagine a world in which people start putting prompts into the journey that are saying, okay, you know, we know that Picasso started incorporating African art after seeing African art exhibit and completely changed European art as as a result, why can't we say create an image that combines, you know, the art of Papua New Guinea with 17th century Spanish architecture with this, and that, and this, and then, you know, as you say, the generative ai ai spits out things that

you've never imagined 99% of it might be terrible, but that 1% that humans can recognize as mind blowing, suddenly becomes its own genre.

and, and modes of expression. So it may feel like the software is doing the heavy lifting, because it's actually producing the artwork. But we might need be prepared to spare some credit for the human at either end of the process, feeding at the prompts, and then sense checking the outputs. And I think there's also a reflection there for educators, which is students are already being exposed to these tools. They're just like,

iPhones and social media. I mean, they're just too mainstream to ignore, you can ban them in within one environment, but that only sort of piques their interest, and makes them more likely to engage with them elsewhere. So it seems to me that we need to be adapting our education system and teaching students that firstly, this, this technology is evolving so rapidly. And so that adaptability itself is the key skill that we need to

ingrain. But secondly, to try to situate them within a context alongside these machines, so they don't just defer to them and see them as things that dish out content to them without scrutiny. Because that process you described, I think, presumes a degree of knowledge and awareness and the ability to sense check on part of the human right. And it's not a given to me that those are natural outcomes of our education system.

Just as the way you say, blurring the lines between, you know, traditional mathematics and creativity is very exciting. We see at tech companies like, you know, Desmos, or GeoGebra, or, you know, there's a few really intriguing company or wiz that are going out of their way to say, hey, let's not think of math as purely symbolic and abstract, but think of it as tangible and visible and, and malleable and creative. It's a really exciting way to push

education forward. You know, we're talking a lot about AI. And it is so exciting. But one of the other areas of expertise that you've gained over the years is international education policy. And I do want to ask about sort of math through that

lens. So we're at this moment where post pandemic, American math scores are in enormous decline, the 2022 nape score, said that average math scores for the US were lower by five points at fourth grade lower by eight points in eighth grade, compared just to three years ago. And the US, you know, is in the very bottom, or, you know, below average for the OECD. In contrast, students in the UK are a little above the average. And then students in countries like, you know, Estonia or Korea, or

Japan are at the very top. My question for you is, how should we think about this data? You know, is this just the old paradigm of math that's calculation based, and we shouldn't take it that seriously? Or, you know, is it data that we should we should look at and say, okay, these different educational systems are coming out with very different outcomes. You know, maybe we should be combining the best of each element. I'm curious how you think about about international math scores.

data, I look at two things. The first is, is the analysis of that data paired with context. And in an international comparison contexts, that just seems absolutely essential, because it would be awfully tempting to just cherry pick findings from the top placed systems. But if you have no awareness of what's actually happening on the ground in that system, and the investment that's being made, system wide, it's very unlikely that that

success will translate over. So identifying and transferring best practice is a decent identify there. But you have to then engage with the context to understand whether a practice in one environment will carry over elsewhere. And then the second thing I would be asking is, how is that assessment data? Given all the costs that goes into acquiring that data and analyzing it? How much of it actually benefit students and

teachers in the classroom? And my worry with most of these assessment schemes, and particularly international comparisons, is that there is a cost burden to accumulate the data. But actually, for your, you know, typical teacher, the data is absolutely useless. But I spent several years working on these tutoring platforms where the goal was to teach kids how to learn math, and they were adaptive tutoring algorithms.

But one of the byproducts was that because you were assessing students at every turn on a lesson by lesson question by question basis, the assessment was baked into the whole learning journey. And then the assessment data was automatically generated and made available to teachers and parents. And you know, there are many companies doing this, and they'll give it different names.

But at its heart, it's the idea of real time learning analytics, if use with intent, you can then recycle them into actionable improvements, whether it's improvements to your teaching,

or whatever else. And so assessment that's in service of learning, I think has its place assessment that is timely, that actually informs practice that is actually used to bring about continuous cost corrections that I've seen work very well, I've seen it work run in large scale implementations that wiz was involved with where everyone involved in the project from students and teachers through to program managers, local ministry stakeholders, everyone was looking at the same data, having

the same dialogue around that data, using it within context, to drive improvements. But there is a risk in all this, and that is the Ed Tech ends up perpetuating the same limitations to math education that I've been touting all along. So the problem is that the easiest assessment data to generate is data that's predicated on very structured closed questions, in other words, the most procedural elements of math. And so what this data often gives you is a measure of students procedural

knowledge. In other words, the exact kind of math that I was, I've been saying, for so long isn't what we should focus on. And there may be a best of both worlds scenario here. Because I do think every education system should be empowered and held accountable to some minimum

standard. So I do think the Sustainable Development Goals SDG, for quality education for all UNESCO's framework, I think it's laudable in its aims, and the idea that we want to be able to measure outcomes, and ensure that every system is achieving a basic minimum, I think it's a

good idea in theory. But then, of course, it leads to the question of what do we mean by minimum outcomes, because in literacy, it's quite well defined, we want an eight year old to be able to read a paragraph fluently and with comprehension. There's no obvious analogue in math and trust me, it's not knowing your times tables, that's akin to like knowing your grammar rules without being able to read or write, and I've seen instances of that. So it's harder in math. And then of course, the other

challenges. As I say, it's how do you make sure the assessment data filters down in a timely and impactful way? I think the challenge for ethics, I think one thing attic is doing well is it's ushering in a new era of content is bringing math to life. It's showing us that math can be more open ended, that you can take the same idea and represent it in 17 different ways.

not performing as well? Not necessarily, but the formative assessment constantly learning what students know and don't know feeding that back into the education system is incredibly powerful. And it's great to hear that Wiz is at the forefront of that as our other terrific You

know, math ed tech companies? I have one last question you mentioned Simon saying earlier, and you're working with Simon Singh, who is a polymath author who's written about cryptography for Fermat's last theorem that you know, the math of The Simpsons, and he has a charity, specifically about virtual math initiatives for students from disadvantaged backgrounds, or at least that's the work you are

working on. When you mentioned the Sustainable Development Goals, you know, math for all that feels extremely relevant. So I'd love to hear a little bit about that work. And what is the state of the art for moving the needles for students from disadvantaged backgrounds?

high potential. But we're talking about finding students as early as age 10, or 11, that have a demonstrating a major aptitude for math that really have the potential to thrive as mathematicians whether or not they go on to do math degrees, math is going to be a huge part of the academic and lifelong

story. And we're trying to identify students, particularly from at risk backgrounds, who may not get the support they need, and may not then realize that potential and we're trying to figure out what we can do to support them over a long period of time to ensure that that

potential is felt. Now that resonates with me, because in I came from a very simple background, and I felt I just about made it but for my success, there are many others that slipped through the net high potential students that just didn't get the support they need that is just a lottery. And so that resonated with me now, it is a different focus to the kind of work I was doing it was which was a lot more inclusive in its ethos, where the idea was to raise the floor standard for

all students. And then to also make sure that higher attaining students were also able to progress at their own individual pace. But I think the work with Simon is differentiated because we're taking a much smaller segment of students and offering a much less scalable form of support. So we do these large, online math [email protected], they're all completely free. So you should check them out and come along to them. And they're good, you know, I deliver some of

them. I think they're good, Simon does a few of them, we have other leading math educators. And they're very, we try to make them very entertaining, very enriching, very challenging, very interactive. But you know, it's hundreds of students at a time. And so there are limits to that. But then the more specialized things that we work on are working with different partners, to then bring students into small tutorial environment. And again, it's typically done

online. But now we're talking five or six students at a time, because we think that's the level of mentoring and coaching and support that students need week on week, every week for several years, so that by age 16, or 17, when they're thinking about their options beyond secondary school or high school, we've helped to hopefully level the playing field, we've taken all of the luxuries that are currently afforded to students from more affluent backgrounds, and hopefully brought that to

the mainstream. So that's the main aim. Now, we are very clear, therefore, that the kind of math that we're offering is at a level that may not be suitable for everyone. And it just may not be of interest to every student. But we absolutely believe there's a critical mass of students that are out there that are worth identifying that would otherwise be lost in the system.

Without getting back into it? One of the things I'm so curious about in the world is, is this something that generative AI might be able to help us with?

You know, if you think of the data set of all the different sides of cohorts and the different math classes or different classes in any of every size, and every topic that we've done over the last 100 years, is there a world in which you know, generative AI can look at it and say, oh, you know, what, I just found a pattern, every time teaching is done in this way, with this type of assessment with this type of cohort with this type of, you know, heterogeneous model of

students, then it works and it can work,

emotions, their mood. Now, of course, AI is moving in the direction of very soon going to be able to do that we already have masses of people falling in love with chatbots. So it's not hard to believe that in the not too distant future, students will be interacting with virtual tutors that they easily mistake for humans. I think that could be okay, as long as we're absolutely transparent, that they are interacting with a bot

rather than a human. And that we're also then very clear on what the dangers of that are, or the unintended consequences and side effects and how we're mitigating against that, because our trial runs with the kinds of chat bots that are coming out of generative AI at the moment,

aren't very encouraging. So I think there are as any algorithmic aspects of education that can be reduced to pattern recognition, like coming up with the correct scheme of work, I think is fair game, the more humanistic elements of relationship building and nurturing students, as learners as humans, that's where I'm more skeptical, because even if we have a motion detection tools and facial recognition tools that can pick up things like mood, I'm not sure I'd want to unleash that on my children,

certainly, or need any others, until we have the appropriate safeguards in place. And I'd like to see a proof of concept outside of an educational context. And all the evidence so far, is that we're going way too quickly. And for all the good that these tools are doing all the exciting developments, there's quite a lot to be concerned about into in terms of how it erodes our humanity. Absolutely. By negative points. No, no, no, not

site, and it's a person. And the whole screen is just a person, it feels like you're doing a zoom chat with somebody, that the person is a chatbot, that you can make them look however you'd like and you build a relationship with them over time, like a tutor. And I think

impersonators. And certainly right now there is zero evidence that these systems inhabit any kind of consciousness or sentience. So to put it bluntly, they don't care about your child, they have no notion of caring, it's an unfair premise. And I think I'd want to be aware of that, if I'm sending my child to achieve that. I want to know that any suggestion that he cares about my child is rooted

in reality. And if it's a fabrication, I at least want to be aware of that, and probably make my child aware of it, too.

much. Is there something else that you've seen coming, rising? In your years, it was in your time working with Simon saying that you're really excited about? Yeah, well, I mean,

normalized online learning. And I guess we're seeing some of the fruits of that parallel when hundreds of students join us at a time. And it's great to just have overcome some of those early barriers and the Zoom bombings and just the general assimilation, and that we've just got to the point where we're all reasonably comfortable with appropriate guardrails of

just getting online. But there remains a challenge to make sure that those benefits are felt across the board and that we're not just extending the digital divide. Terrific answer. And by the way, and by the way, there will be another pandemic in our lifetimes. I'm unfortunately, growing, ever confident of that. So we need to be prepared and need to make sure that we don't repeat the same mistakes.

prescient. Now, if you think about a lot of the hype around generative AI, it's beautifully written and it was written about 2530 years ago, and it's so relevant for our times, and should challenge all of us parents, educators, to reflect on why we send our kids to school and what the purpose of education is. He uses the term the end in the end of education in two ways, ends in purpose and endless in combination. So highly recommend that

had in a long time. And I think AI math and creativity and AI, it's going to be this enormous boom, we'll see. Hopefully, it will be positive and not destroyed.