I dag, skal jeg tale om kunstig intelligens og os. Forskere har altid sagt at vi mennesker ikke skal bekymrer os, fordi kun det kedeligste arbejde bliver overtaget af maskiner. Er det virkelig sandt? De siger også, at kunstig intelligens skaber nye arbejdspladser, så de der mister jobbet kan finde et nyt. Selvfølgelig. Men, det rigtige spørgsmål er: Hvor mange af dem som mister deres arbejde til kunstig intelligens vil kunne finde nyt når kunstig intelligens er bedre til at lære, end de fleste mennesker?
Today, I'm going to talk about AI and us. AI researchers have always said that we humans do not need to worry, because only menial jobs will be taken over by machines. Is that really true? They have also said that AI will create new jobs, so those who lose their jobs will find a new one. Of course. But the real question is: How many of those who may lose their jobs to AI will be able to land a new one, especially when AI is smart enough to learn better than most of us?
Lade mig spørge jer om noget: Hvor mange af jer tror at kunstig intelligens kan bestå en optagelsesprøve til universitetet inden 2020? Nå, så mange! Ok. Nogle af jer siger, "Ja, selvfølgelig!" og tænker singularitet. Og andre siger, "Måske, fordi kunstig intelligens har slået en top-Go spiller." Stadig andre siger, "Nej, aldrig. Slet ikke." Det må jo betyde at vi endnu ikke kender svaret? Det var grunden til at jeg startede Todai Robot projektet og skabte en kunstig intelligens som består optagelsesprøven på Tokyo Universitet, Japans top-universitet.
Let me ask you a question: How many of you think that AI will pass the entrance examination of a top university by 2020? Oh, so many. OK. So some of you may say, "Of course, yes!" Now singularity is the issue. And some others may say, "Maybe, because AI already won against a top Go player." And others may say, "No, never. Uh-uh." That means we do not know the answer yet, right? So that was the reason why I started Todai Robot Project, making an AI which passes the entrance examination of the University of Tokyo, the top university in Japan.
Det er vores Todai Robot. Og selvfølgelig arbejder robottens hjerne på en selvstændig server. Lige nu skriver den en 600-ords stil, om maritimhandel i det 17. århundrede. Vildt, ikke?
This is our Todai Robot. And, of course, the brain of the robot is working in the remote server. It is now writing a 600-word essay on maritime trade in the 17th century. How does that sound?
Hvorfor valgte jeg prøven som grundlag? Fordi vi bør studere hvad kunstig intelligens kan præstere sammenlignet med mennesker, specielt de evner og ekspertise man tror at kun mennesker mestrer og kun igennem undervisning. For at blive optaget på Todai, som er Tokyo Universitet skal man bestå to forskellige prøver. Først en statslig nationalprøve i 'multiple-choice' stil. Man skal op i syv fag og opnå en høj score - nok mere end 84 procent - for at kom videre til den anden prøve, som Todai står for.
Why did I take the entrance exam as its benchmark? Because I thought we had to study the performance of AI in comparison to humans, especially on the skills and expertise which are believed to be acquired only by humans and only through education. To enter Todai, the University of Tokyo, you have to pass two different types of exams. The first one is a national standardized test in multiple-choice style. You have to take seven subjects and achieve a high score -- I would say like an 85 percent or more accuracy rate -- to be allowed to take the second stage written test prepared by Todai.
Lade mig først forklare hvordan moderne kunstig intelligens virker, ved at bruge "Jeopardy!" som et eksempel. Her er et typisk "Jeopardy!" spørgsmål: "Mozarts sidste symfoni deler navn med denne planet." Interessant nok, så spørger en "Jeopardy!" spørgsmål altid, ender altid med "denne" noget, "denne" planet, "dette" land, "denne" rockmusiker, og så videre. Sagt med andre ord, har "Jeopardy!" ikke mange forskellige typer spørgsmål, men kun en type, der kaldes "faktuelt spørgsmål".
So let me first explain how modern AI works, taking the "Jeopardy!" challenge as an example. Here is a typical "Jeopardy!" question: "Mozart's last symphony shares its name with this planet." Interestingly, a "Jeopardy!" question always asks, always ends with "this" something: "this" planet, "this" country, "this" rock musician, and so on. In other words, "Jeopardy!" doesn't ask many different types of questions, but a single type, which we call "factoid questions."
Forresten, kender du svaret? Hvis man ikke kender svaret, og vil vide det, hvad gøre man så? Man googler det, ikke? Selvfølgelig. Hvorfor ikke? Men man skal vælge relaterede ord, som "Mozart", "sidste", og "symfoni" til at søge med. Maskinen gør nogenlunde det samme. Denne Wikipedia rangerer højt. Så læser maskinen siden. Nej, desværre ikke.
By the way, do you know the answer? If you do not know the answer and if you want to know the answer, what would you do? You Google, right? Of course. Why not? But you have to pick appropriate keywords like "Mozart," "last" and "symphony" to search. The machine basically does the same. Then this Wikipedia page will be ranked top. Then the machine reads the page. No, uh-uh.
Ingen af de moderne kunstig intelligenser, herunder Watson, Siri og Todai Robot, kan læse. Men, de er gode til at søge og optimere. Den ville genkende at "Mozart", "sidste", og "symfoni" dukker op mange gange her. Så hvis den finder et ord som er en planet og som dukker op sammen med de andre ord, så må det være svaret. Det er sådan at Watson finder svaret "Jupiter" i dette tilfælde.
Unfortunately, none of the modern AIs, including Watson, Siri and Todai Robot, is able to read. But they are very good at searching and optimizing. It will recognize that the keywords "Mozart," "last" and "symphony" are appearing heavily around here. So if it can find a word which is a planet and which is co-occurring with these keywords, that must be the answer. This is how Watson finds the answer "Jupiter," in this case.
Vores Todai Robot fungerer på en lignende måde, men lidt bedre, når den svarer historiske ja-nej spørgsmål som "Charlemagne udstødte magyarerne". Robotten skaber selv et faktum spørgsmål: "Charlemagne udstødte [persontype]". Her er det "avarerne", ikke "magyarerne", som rangerer øverst. Sætningen er sandsynligvis falsk. Vores robot læser ikke, forstår ikke, men er statistisk set korrekt i mange tilfælde.
Our Todai Robot works similarly, but a bit smarter in answering history yes-no questions, like, "'Charlemagne repelled the Magyars.' Is this sentence true or false?" Our robot starts producing a factoid question, like: "Charlemagne repelled [this person type]" by itself. Then, "Avars" but not "Magyars" is ranked top. This sentence is likely to be false. Our robot does not read, does not understand, but it is statistically correct in many cases.
I anden skriftlige test, skal man skrive en 600-ords stil som denne:
For the second stage written test, it is required to write a 600-word essay like this one:
[Diskuter vækst og fald for maritimhandel i Øst- og Sydøstasien i det 17. århundrede]
[Discuss the rise and fall of the maritime trade in East and Southeast Asia in the 17th century ...]
og som jeg viste tidligere, tog vores robot sætninger fra fagbøger og Wikipedia, kombinerede dem, og formede en stil uden at forstå noget som helst.
and as I have shown earlier, our robot took the sentences from the textbooks and Wikipedia, combined them together, and optimized it to produce an essay without understanding a thing.
(latter)
(Laughter)
Men overraskende nok skrev den en bedre stil en de fleste af de studerende.
But surprisingly, it wrote a better essay than most of the students.
(latter)
(Laughter)
Hvad med matematik? En fuldautomatisk matematikmaskine har været en drøm siden fødslen af ordet "kunstig intelligens", men har holdt sig på niveauet af regning i langt, langt tid. Sidste år, lykkes det os til sidst at lave et system som løste komplette opgaver på pre-universitets-niveau, som denne her. Dette er den originale problemformulering på Japansk og vi måtte lære den 2.000 matematiske aksiomer og 8.000 Japanske ord før den forstod opgaven på på naturligt sprog. Nu oversætter den de oprindelige opgaver til maskinsprog. Underligt, men nu tror jeg at den er klar til at løse den. Løse den så. Ja! Nu udfører den symbolske udregninger. Endnu mere underligt, men det her er nok den sjoveste del for maskinen.
How about mathematics? A fully automatic math-solving machine has been a dream since the birth of the word "artificial intelligence," but it has stayed at the level of arithmetic for a long, long time. Last year, we finally succeeded in developing a system which solved pre-university-level problems from end to end, like this one. This is the original problem written in Japanese, and we had to teach it 2,000 mathematical axioms and 8,000 Japanese words to make it accept the problems written in natural language. And it is now translating the original problems into machine-readable formulas. Weird, but it is now ready to solve it, I think. Go and solve it. Yes! It is now executing symbolic computation. Even more weird, but probably this is the most fun part for the machine.
(latter)
(Laughter)
Nu udskriver den et perfekt svar, men beviset er umuligt at læse, selv for matematikere. Sidste år var vores robot blandt de 1% bedste i den skriftlige prøve på andet niveau matematik.
Now it outputs a perfect answer, though its proof is impossible to read, even for mathematicians. Anyway, last year our robot was among the top one percent in the second stage written exam in mathematics.
(publikum klapper)
(Applause)
Tak.
Thank you.
Kom den ind på Todai? Nej, ikke som jeg forventede. Hvorfor? Fordi den forstår slet ikke mening. Lade mig vise jer en typisk fejl, som den lavede i engelsk prøven.
So, did it enter Todai? No, not as I expected. Why? Because it doesn't understand any meaning. Let me show you a typical error it made in the English test.
[Nate: Vi er næsten ved boghandleren. Bare et par minutter mere. Sunil: Vent. _____. Nate: Tak! Det sker altid...]
[Nate: We're almost at the bookstore. Just a few more minutes. Sunil: Wait. ______ . Nate: Thank you! That always happens ...]
To mennesker taler. For os, som kan forstå situationen -
Two people are talking. For us, who can understand the situation --
[1. "Vi gik i lang tid." 2. "Vi er der næsten." 3. "Dine sko ser dyre ud." 4. "Dit snørebånd er gået op."]
[1. "We walked for a long time." 2. "We're almost there." 3. "Your shoes look expensive." 4. "Your shoelace is untied."]
- er det klart at nummer fire er det rigtig svar, ikke? Men Todai Robot valgte nummer to, selv efter at havde lært 15 milliarder engelske sætninger ved brug af deep-learning teknologier. Ok, så nu forstår i måske hvad jeg sagde, moderne kunstig intelligenser læser ikke, forstår ikke. De lader kun som om at de gør.
it is obvious number four is the correct answer, right? But Todai Robot chose number two, even after learning 15 billion English sentences using deep learning technologies. OK, so now you might understand what I said: modern AIs do not read, do not understand. They only disguise as if they do.
Det er fordelingsgrafen af en halv million studerende som tog den samme prøve som Todai Robot. Vores Todai Robot er blandt de 20% bedste, og den kunne havde bestået på mere end 60% af universiteterne i Japan men ikke på Todai. Bemærk at den overgår flokken af dem som bliver kontorarbejdere.
This is the distribution graph of half a million students who took the same exam as Todai Robot. Now our Todai Robot is among the top 20 percent, and it was capable to pass more than 60 percent of the universities in Japan -- but not Todai. But see how it is beyond the volume zone of to-be white-collar workers.
I tror måske at jeg var rigtig glad. Min robot vandt jo over studerende overalt. Men i sted for, var jeg bekymret. Hvordan i alverden kunne den her dumme maskine præstere bedre end studerende - vores børn? Ikke? Jeg bestemte mig for at undersøge det der skete i den menneskelige verden. Jeg tog hundredevis af sætninger fra gymnasiebøger og skabte nemme 'multiple-choice' opgaver og fik tusindvis af gymnasiestuderende til at svare.
You might think I was delighted. After all, my robot was surpassing students everywhere. Instead, I was alarmed. How on earth could this unintelligent machine outperform students -- our children? Right? I decided to investigate what was going on in the human world. I took hundreds of sentences from high school textbooks and made easy multiple-choice quizzes, and asked thousands of high school students to answer.
Her er et eksempel:
Here is an example:
[Buddhisme udbredt til ..., kristendom til ... og Oceanien, og islam til ...]
[Buddhism spread to ... , Christianity to ... and Oceania, and Islam to ...]
De originale opgaver er på Japansk, deres modersmål.
Of course, the original problems are written in Japanese, their mother tongue.
[ _____ er udbredt til Oceanien. 1. Hinduisme 2. Kristendom 3. Islam 4. Buddhisme ]
[ ______ has spread to Oceania. 1. Hinduism 2. Christianity 3. Islam 4. Buddhism ]
Det giver jo sig selv at svaret er kristendom? Det står der jo! Og Todai Robot valgte også rigtig. Men en 1/3 af eleverne i 1.g kunne ikke svare på spørgsmålet. Tror i at det kun er tilfældet i Japan? Det tror jeg ikke, for Japan ligger altid i toppen i OECD PISA testen, som vurderer 15-åriges evner i matematik, naturvidenskab og læsning hver tredje år.
Obviously, Christianity is the answer, isn't it? It's written! And Todai Robot chose the correct answer, too. But one-third of junior high school students failed to answer this question. Do you think it is only the case in Japan? I do not think so, because Japan is always ranked among the top in OECD PISA tests, measuring 15-year-old students' performance in mathematics, science and reading every three years.
Vi har haft opfattelsen at alle kan lære og lære godt, så længe det rigtige materiale er til rådighed gratis online så de kan tilgå det via internet. Men disse gode redskaber hjælper måske kun dem der læser godt, og andelen af dygtige læsere er måske meget lavere end vi tror. Hvordan vi kan sameksistere med kunstig intelligens er noget vi må tænke nøje over, baseret på solid information. Men samtidig må vi tænke hurtigt, for tiden er kort.
We have been believing that everybody can learn and learn well, as long as we provide good learning materials free on the web so that they can access through the internet. But such wonderful materials may benefit only those who can read well, and the percentage of those who can read well may be much less than we expected. How we humans will coexist with AI is something we have to think about carefully, based on solid evidence. At the same time, we have to think in a hurry because time is running out.
Tak.
Thank you.
(publikum klapper)
(Applause)
Chris Anderson: Noriko, mange tak.
Chris Anderson: Noriko, thank you.
Noriko Arai: Tak.
Noriko Arai: Thank you.
CA: Din tale giver os et smukt indblik i hvordan kunstig intelligens tænker, hvad de kan gøre super godt og hvad de ikke kan. Men - mener du virkelig, at vi har brug for en hurtig revolution inden for undervisning, for at hjælpe børn til at gøre det vi allerede gør bedre end kunstig intelligens?
CA: In your talk, you so beautifully give us a sense of how AIs think, what they can do amazingly and what they can't do. But -- do I read you right, that you think we really need quite an urgent revolution in education to help kids do the things that humans can do better than AIs?
NA: Ja, ja, ja. Fordi vi mennesker forstår meningen. Det er noget som virkelig mangler i kunstig intelligens. Men de fleste elever suger bare viden, uden at forstå betydningen af den viden, så det er ikke at lære noget, det er bare at huske, og kunstig intelligens kan gøre det samme. Så vi må opfinde en ny type undervisning.
NA: Yes, yes, yes. Because we humans can understand the meaning. That is something which is very, very lacking in AI. But most of the students just pack the knowledge without understanding the meaning of the knowledge, so that is not knowledge, that is just memorizing, and AI can do the same thing. So we have to think about a new type of education.
CA: En ændring fra viden, tom viden, til mening.
CA: A shift from knowledge, rote knowledge, to meaning.
NA: Ja.
NA: Mm-hmm.
CA: Nå, der er da en udfordring for lærerne. Mange tak.
CA: Well, there's a challenge for the educators. Thank you so much.
NA: Tusind tak. Tak.
NA: Thank you very much. Thank you.
(publikum klapper)
(Applause)