こんにちは、Choimirai Schoolのサンミンです。

はじめに

2023年5月2日、Khan AcademyのCEO、Sal Khan氏がまたもTED Talksへ登壇です。今回のタイトルは「The Amazing AI Super Tutor for Students and Teachers」。AIが教育に与える影響を説いていて、話の中には同氏ならではの明快で研ぎ澄まされた知見が息づいています。日本でもぜひ拡散してほしい内容でしたので日本語訳（DeepL）をつけたスクリプトをシェアします。

YouTube

Descript

Introduction: AI in Education

So, anyone who's been paying attention for the last few months has been seeing headlines like this, especially in education. The thesis has been, students are going to be using ChatGPT and other forms of AI to cheat, do their assignments, they're not going to learn, and it's going to completely undermine education as we know it. Now, what I'm going to argue today is not only are there ways to mitigate all of that, if we're going to get rid of all of that, if we put the right guardrails, we do the right things, we can mitigate it, but I think we're at the cusp of using AI for probably the biggest positive transformation that education has ever seen.

The Two-Sigma Problem and Personal Tutors

And the way we're going to do that is by giving every student on the planet an artificially intelligent but amazing personal tutor, and we're going to give every teacher on the planet an amazing, artificially intelligent teaching assistant. And just to appreciate how big of a deal it would be to give everyone a personal tutor, I show you this clip from Benjamin Bloom's 1984 two-sigma study, or he called it the two-sigma problem. The two-sigma comes from two standard deviations, sigma the symbol for standard deviation, and he had good data that showed that, look, a normal distribution, that's the one that you see in the traditional bell curve right in the middle, that's how the two-sigma comes from, the traditional bell curve right in the middle, that's how the world kind of sorts itself out, that if you were to give personal one-to-one tutoring for students, then you could actually get a distribution that looks like that right, it says tutorial one-to-one with the asterisk, like that right distribution, a two-standard deviation improvement.

Just to put that in plain language, that could take your average student and turn them into an exceptional student, it can take your below-average student and turn them into an above-average student. Now the reason why he framed it as a problem was he said, well, this is all good, but how do you actually scale group instruction this way? How do you actually give it to everyone in an economic way?

Khanmigo: AI Tutor in Action

What I'm about to show you is, I think, the first moves towards doing that. Obviously, we've been trying to approximate it in some way at Khan Academy for over a decade now, but I think we're at the cusp of accelerating it dramatically. I'm going to show you the early stages of what AI, which we call Khanmigo, what it can now do, and maybe a little bit of where it is actually going.

So this right over here is a traditional exercise that you or many of your children might have seen on Khan Academy, but what's new is that little bot thing at the right. And we'll start by seeing one of the very important safeguards, which is the conversation is recorded and viewable by your teacher. It's moderated, actually, by a second AI. And also, it does not tell you the answer. It is not a cheating tool. Notice, when the student says, tell me the answer, it says, I'm your tutor, what do you think is the next step for solving the problem?

AI Tutor Identifying Mistakes and Misconceptions

Now, if the student makes a mistake, and this will surprise people who think large language models are not good at mathematics, notice not only does it notice the mistake, it asks the student to explain their reasoning. But it's actually doing what I would say not just even an average tutor would do, but an excellent tutor would do. It's actually able to divine what is probably the misconception in that student's mind, that they probably didn't use the distributive properly. Remember, we need to distribute the negative 2 to both the 9 and the 2m inside of the parentheses. This, to me, is a very, very, very big deal.

AI Tutor in Computer Programming

And it's not just in math. This is a computer programming exercise on Khan Academy where the student needs to make the clouds part. And so we can see the student starts defining a variable, left x minus minus. It only made the left cloud part, but then they can ask a Khanmigo, what's going on? Why is only the left cloud moving? And it understands the code. It knows all the context of what the student is doing, and it understands that those ellipses are there to draw clouds, which I think is kind of mind-blowing. And it says, to make the right cloud part as well, try adding a line of code inside the draw function that increments the right x variable by one pixel in each frame. Now, this one is maybe even more amazing because we have a lot of math teachers. We've all been trying to teach the world to code, but there aren't a lot of computing teachers out there. And what you just saw, even when I'm tutoring my kids, when they're learning to code, I can't help them this well, this fast. This is really going to be a super tutor.

Understanding the AI Super Tutor

And it's not just exercises. It understands what you're watching. It understands the context of your video. It can answer the age-old question, why do I need to learn this? And it asks socratically, well, what do you care about? And let's say the student says, I want to be a professional athlete. And it says, well, learning about the size of cells, which is what this video is about, that could be really useful for understanding nutrition and how your body works, et cetera. It can answer questions. It can quiz you. It can connect it to other ideas. You can now ask as many questions of a video as you could ever dream of.

Providing Guidance and Coaching

So, another big shortage out there, I remember the high school I went to, the student-to-guidance counselor ratio was about 200 or 300 to 1. In a lot of the country, it's worse than that. We can use Khanmigo to give every student a guidance counselor, academic coach, career coach, life coach, which is exactly what you see right over here. And this is all, you know, we launched this with the GPT-4 launch. We have a few thousand people on this. This isn't a fake demo. This is really it in action.

Learning Literature and History

And then there's, you know, things that I think it would have been even harder, it would have been a little science fiction to do with even a traditional tutor. There's a student, we run an online high school with Arizona State University called Khan World School, and we have a student who attends that online school based in India, her name is Sanvi, and she was doing a report on "The Great Gatsby". And when she was reading the Great Gatsby, Jay Gatsby keeps looking at the green light off into the distance. And she's like, well, "why does he do that?"" She did some web searches, and people have obviously studied this and commented about the symbolism of that, but none of it was really resonating with her. And then she realized that she had Khanmigo and that she could talk to Jay Gatsby himself.

Debating and Developing Arguments

And so, ah, splendid choice, old sport, I am now Jay Gatsby, the enigmatic millionaire from F. Scott Fitzgerald's Classic. And so "why do you keep staring at the green light?" "Ah, the green light, old sport, it's a symbol of my dreams and desires, you see. It's situated at the end of Daisy Buchanan's dock, across the bay from my mansion. I gaze at it longingly as it represents my yearning for the past and my hope to reunite with Daisy, the love of my life." And what was cool is, Sanvi said, I had this long conversation, she called him Mr. Gatsby, and at the end she actually apologized for taking his time, which I thought was very polite of her. But you can imagine, this unlocks learning literature, learning, you can talk to historical figures, we're even probably going to add an activity, you can talk to, like, the Mississippi River. It brings things to life in ways that really were science fiction even six months or a year ago. Students can get into debates with the AI. And we've got, here's the students debating whether we should cancel student debt, the student is against cancelling student debt, and we've gotten very clear feedback, we started running it at Khan World School, in our lab school, that we have, Khan Lab School. The students, the high school students especially, they're saying, this is amazing to be able to fine-tune my arguments without fearing judgment, it makes me that much more confident to kind of go into the classroom and really participate. And we all know that Socratic dialogue debate is a great way to learn, but frankly, it's not out there for most students, but now it can be accessible to, hopefully, everyone.

Collaborative Writing with AI

A lot of the narrative, we saw that in the headlines, has been, it's going to do the writing for kids, kids are not going to learn to write, but we are showing that there's ways that the AI doesn't write for you, it writes with you. So this is a little thing, and my eight-year-old is addicted to this, and he's not a kid that really liked writing before, but where, you know, you could say, I want to write a horror story, and it says, ooh, a horror story, how spine-tingling and thrilling. Let's dive into the world of eerie shadows and chilling mysteries. And this is an activity where the student will write two sentences, and then the AI will write two sentences. And so they collaborate together on a story, the students write, "Beatrice was a misunderstood ghost, she wanted to make friends but kept scaring them by accident", and the AI says, "poor Beatrice, a lonely spirit yearning for companionship, one day she stumbled upon an old abandoned mansion", etc., etc. I encourage you all to, you know, hopefully one day try this, this is surprisingly fun.

Prototype for Enhancing English and Language Arts

Now, to even more directly hit this use case, and what I'm about to show you, everything I showed you so far is actually already part of Khanmigo. What I'm about to show you we haven't shown to anyone yet, this is a prototype, we hope to be able to launch it in the next few months, but this is to directly use AI, use generative AI, to not undermine English and language arts, but to actually enhance it in ways that we couldn't have even conceived of even a year ago.

Reading Comprehension and AI-driven Questions

This is reading comprehension, the student's reading Steve Jobs' famous speech at Stanford, and then as they get to certain points, they can click on that little question, and the AI will then, socratically, almost like an oral exam, ask the student about things, and the AI can highlight parts of the passage. Why did the author use that word? What was their intent? Does it back up their argument? They can start to do stuff that, once again, we never had the capability to give everyone a tutor, everyone a writing coach, to actually dig in to reading at this level.

AI Writing Coach and Feedback

And you could go on the other side of it, we have a whole workflow that helps them write, helps them be a writing coach, draw an outline, but once a student actually constructs a draft, and this is where they're constructing a draft, they can ask for feedback, once again, as you would expect from a good writing coach. In this case, the student will say, let's say, does my evidence support my claim? And then the AI not only is able to give feedback, but it's able to highlight certain parts of the passage, and says, you know, on this passage, this doesn't quite support your claim, but once again, socratically, says, can you tell us why? So it's not only fooling the student, it's making them a better writer, giving them far more feedback than they've ever been able to actually get before, and we think this is going to dramatically accelerate writing, not hurt it.

AI for Teachers: Personalized Education and Saving Time

Now, everything I've talked about so far is for the student, but we think this could be equally as powerful for the teacher to drive more personalized education, and frankly, save time and energy for themselves and for their students. So this is an American history exercise on Khan Academy, a question about the Spanish-American War, and at first it's in student mode, and if you say, tell me the answer, it's not going to tell the answer, it's going to go into tutoring mode.

Teacher Mode: Lesson Planning and Grading Assistance

But that little toggle, which teachers have access to, they can turn student mode off, and then it goes into teacher mode, and what this does is, it turns into, you could view it as a teacher's guide on steroids. Not only can it explain the answer, it can explain how you might want to teach it. It can help prepare the teacher for that material. It can help them create lesson plans, as you can see it doing right there. It will eventually help them create progress reports, it will help them eventually grade. So once again, teachers spend about half their time with this type of activity, lesson planning, all of that energy can go back to them, or go back to human interactions with their actual students.

The Power of Large Language Models

So, you know, one point I want to make, these large language models are so powerful, there's a temptation to say, like, well, all these people are just going to slap them onto their websites, and it kind of turns the applications themselves into commodities. And what I've got to tell you is, I kind of thought that, that's one of the reasons why I didn't sleep for two weeks when I first had access to GPT-4 back in August. But we quickly realized that to actually make it magical, I think what you saw with Khanmigo a little bit, it didn't interact with you the way that you see ChatGPT interacting, it was a little bit more magical, it was more Socratic, it was clearly much better at math than what most people are used to thinking.

The Importance of AI Thinking Before Speaking

And the reason is, there was a lot of work behind the scenes to make that happen. And I could go through the whole list of everything we've been working on, many, many people, for over six, seven months, to make it feel magical. But perhaps the most intellectually interesting one is we realized, and this was an idea from an open AI researcher, that we could dramatically improve its ability in math and its ability in tutoring if we allowed the AI to think before it speaks. So if you're tutoring someone and you immediately just start talking before you assess their math, you might not get it right. But if you construct thoughts for yourself, and what you see on the right there is an actual AI thought, something that it generates for itself, but it does not share with the student, then its accuracy went up dramatically, and its ability to be a world-class tutor went up dramatically. And you can see it's talking to itself here. It says, the student got a different answer than I did, but do not tell them they made a mistake. Instead, ask them to explain how they got to that step.

Addressing the Two-Sigma Problem

So I'll just finish off. Hopefully, what I've just shown you is just half of what we are working on, and we think this is just the very tip of the iceberg of where this can actually go. And I'm pretty convinced, which I wouldn't have been even a year ago, that we together have a chance of addressing the two-sigma problem and turning it into a two-sigma opportunity, dramatically accelerating education as we know it.

AI's Potential Impact: Optimism vs. Pessimism

Now, just to take a step back at a meta level, obviously, we heard a lot today, the debates on either side. There's folks who take a more pessimistic view of AI. They say, this is scary. There's all these dystopian scenarios. We maybe want to slow down. We want to pause. On the other side, there are the more optimistic folks who say, well, we've gone through inflection points before. We've gone through the Industrial Revolution. It was scary, but it all kind of worked out.

The Importance of Active Participation

And what I'd argue right now is I don't think this is like a flip of a coin, or this is something where we'll just have to wait and see which way it turns out. I think everyone here and beyond, we are active participants in this decision. I'm pretty convinced that the first line of reasoning is actually almost a self-fulfilling prophecy, that if we act with fear and if we say, hey, we just got to stop doing this stuff, what's really going to happen is the rule followers might pause, might slow down, but the rule breakers, as Alexander mentioned, the totalitarian governments, the criminal organizations, they're only going to accelerate. And that leads to what I am pretty convinced is the dystopian state, which is the good actors have worse AIs than the bad actors.

Fighting for Positive Use Cases

But I'll also talk to the optimist a little bit. I don't think that means that, oh, yeah, then we should just relax and just hope for the best. That might not happen either. I think all of us together have to fight like hell to make sure that we put the guardrails, we put in, when the problems arise, reasonable regulations, but we fight like hell for the positive use cases. Because very close to my heart, and obviously there's many potential positive use cases, but perhaps the most powerful use case, and perhaps the most poetic use case, is if AI, artificial intelligence, can be used to enhance HI, human intelligence, human potential, and human purpose. Thank you.