Can a robot pass a university entrance exam? | Noriko Arai

Meet Todai Robot, an AI project that performed in the top 20 percent of students on the entrance exam for the University of Tokyo -- without actually understanding a thing. While it's not matriculating anytime soon, Todai Robot's success raises alarming questions for the future of human education. How can we help kids learn the things that humans can do better than AI?
The TED Talks channel features the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes (or less). Look for talks on Technology, Entertainment and Design -- plus science, business, global issues, the arts and more.
Follow TED on Twitter: / tedtalks
Like TED on Facebook: / ted
Subscribe to our channel: / ted

[@gabriellaruth83]

This is what a bunch of people having been saying for years, the Bad student forgets all of the content BEFORE the test, and the Good student forgets it all AFTER the test. Memorizing isnt learning.

[@m_sedziwoj]

I age where books was expensive, and time to access to them was in hours or even days, memorizing stuff was needed. But now we leave in different time, were we find information very fast, so we should stop try to memorized things, we should start more understand things.

[@clairegittens3707]

Wow! That’s a brilliant summary. Last month, I was doing some review work in an English as a foreign language class. My partner teacher whispered to me, “Hey, that’s not on the test.” But here’s the thing. Language skills are cumulative. You need to build on top of your old skills. So, theoretically, every test should involve everything you’ve ever learned. But that’s not the case. Just remember this chapter and forget everything else.

[@wisdomsquare28]

Finally, someone who is awake. Thank you.

[@janaebert3059]

I wait for the day when the presenter reveals that he/she is a robot.

[@MRLONG758]

Oh what a creepy day that would be.

[@macrobody]

Jana Ebert I am sure I would have understand a robot better.

[@awethesound]

pshhhh

[@niveshproag8660]

"I would have understand" Oh the irony.

[@macrobody]

Excuse me, English is not my native language so I don't know what you mean.

[@MaZe741]

Congratulations on passing the math exam. Now, Captchas!

[@lovefoolishh]

best comment here

[@JL2579]

Impressive talk and great summary at the end !

[@catherine_404]

Entrance exam is not a test of being able to do any type of a job. It's a test of being able to pass a test. It also shows, you don't need to really understand a test to pass it.

[@waveforms]

Need robots to give relationship advice. It'd probably be better than 80% we get.

[@davidswe98]

There is always an asian robot better than u

[@SirNightmareFuel]

kzbin.info/www/bejne/jqnOfpKZo895nNk

[@emersonlee6422]

we dont need robot! fcking satan!

[@lizzam]

I'm surprised a talk about AI ends up about our quality of education, our learning, teaching and assessment method.

[@Morkvonork]

I love the math answer :D

[@nyx211]

If an AI program were able to get a high score on an IQ test would you say that the program is intelligent? If you say, "no", then isn't that an admission that IQ tests don't measure intelligence?

[@fetchstixRHD]

Interesting point you raise there.

[@qncsc]

you don't understand the purpose of IQ tests.

[@samikilic9149]

Marcus Aurelius I agree 👍🏻

[@lachlansleight]

The IQ test was designed to measure human intelligence. It's an interesting thought to think about what an IQ test designed to directly compare humans and AI systems would be like...

[@redtails]

Replace AI with monkey and answer the question yourself ;)

[@thanasisathanasiou1260]

It responded with "2. Were almost there" probably due to the procedural understanding of events starting and finishing. Or probably because "wait" is a remark directing your attention to time. Possibly a combination of the two. One thing is for sure tho, AI's will never forget to tie their shoes... so yeah; makes sense.

[@donjuan69420]

I call dibs on are you smarter than a robot tv show

[@memoryhero]

I kept hoping Robert would eventually get into the university.

[@sethgrasse9082]

memoryhero Robert? Isn't it Todai Robot?

[@rahb3rt]

Or it shows how useless these entrance exams are.

[@pzrkstinger]

Robert Davis explain

[@rahb3rt]

Youre taking an exam that does not show any real life application just how fast you can recite things from memory. Real world doesnt require you to memorize facts but instead apply knowledge and be able to learn on the run new concepts and adapt. The education system is a scam sold to students thinking that if a then b then c, but sometimes that intermediate step b is unnecessary to accomplish any of it. Its an imposition of impracticality. The robot was able to memorize facts and look them up, thats easy. But interpreting meaning or having any depth application is what makes it still a robot. If society wants to raise robots thats fine, but they will all get out paced by real robots that are better at performing these sorts of tasks. Its like asking a student to memorize the multiplication table when they have a calculator, sure its a great skill to have but why not teach students to learn about the theories of mathematics and why numbers can be multiplied or proofs than something that is easily doable by a machine.

[@qncsc]

*you missed the whole point: the exam is to identify a distribution.* and that distribution is reflecting some amount of rigor that will be coincident to the material that will be covered in future higher education. those who can accommodate more complex forms of data, analysis will reflect that and be able to reflect similarly in future domains. education is about (a) selecting those who are cognitively capable (and to a lesser extent willing), and the (b) curriculum that deconstructs the basic principles, allowing for future efforts of complexity and abstractions upon which to build greater competency. what the robot shows is that even rudimentary competency can be achieved where vast numbers of humans have great inability. 1 million robots will easily displace those 1 million humans; the economics of a robot are all superior to the human. and therefore, the humans will have no compelling economic argument to advance their economic purpose and agenda. the expansion of artificial intelligence can then scale up the economic value chain overtaking more and more human functionality. at every turn the robot will offer a more compelling economic expression, so long as it outcompetes the human on task. this is what the lecturer in the video sees -- but is only implied. this was an excellent video!

[@rahb3rt]

Education has nothing to do with being "cognitively capable" because there are plenty of people who are. You cannot translate a subject like english or social studies to be a numbers game when inherently the subject itself is not a numbers game. Something like physics and chemistry sure to some extent you can translate to numbers. But it makes no sense that a subject like English can be translated to a functional number where subject matter is opinion based and grammar is not a definitive reflection of anyone's ability to speak, read or write.

[@nofanfelani6924]

The entrance exams is useless for most of us, yes. But one in a thousand of student who pass the test will become a great scientist, another one will become a physicist or another great person who will invent a new things for humanity. We wouldn't know who is that "one person" if we only being thought the basics and practical knowledge at school. Who do you think invented the smartphone you have now? It's one of the student who passed that "useless" entrance test

[@SnowmansApartment]

so robots are dyslexic.. 😵.. i might bee a robot after all..

[@5even5eals]

As well as a majority of students, it seems.

[@PersonalPower]

Could a robot pass most bachelors degrees? Yes, because you only have to memorize a lot of phrases to pass. Might struggle at the bachelor thesis, but otherwise I dont see a Problem. I probably learned more from fellow educational KZbin channels than from my 2 semesters at university.

[@BagoGarde]

If Ai can write news article , i believe they can write a thesis.And with some physical aid, even collect information.

[@s.vincentparish8168]

What university did you attend? UT specifically aimed to measure student understanding by gearing exams towards critical thinking rather than memorization.

[@PersonalPower]

I'm currently studying in a smalltown in Germany. We have some sort of critical thinking but most of the time its memorizing I think.

[@yanusanders7686]

Depends completely on the school system. In the UK all tests are made in a way so that just memorisation gets you a maximum of a C or D.

[@Sdhector21]

Bago Garde The whole point of the talk is that AI cannot "write", rather summarize existing text. Writing involves creativity, which implied understanding abstract ideas. Until AI can understand meaning it won't be able to "write".

[@MegtaBubble]

This is what we want from Ted!

[@xYoz]

The book which was written by Ms.Arai is very interesting!

[@Alex55555]

One of the best talks on AI

[@akritiarora1904]

This robot will easily get into Indian universities

[@MRLONG758]

How so?

[@akritiarora1904]

MRLONG758 because the education system sucks that's why I cleared my entrance by mistake and got a rank of 18 all over India thousands of people register for it it was MCQ based and I barely studied

[@akritiarora1904]

MRLONG758 I also gave other entrances like of computer application course and business administration I don't even know maths science or accounts I got 500 rank in computers which could easily get me into best college and 3000 rank for business it is kind of a hit and trial

[@suryasharma9410]

Akriti Arora : Hey Akriti , In which entrance exam did you get AIR 18 ? , because in country like India . It is very tough.

[@akritiarora1904]

Surya Sharma GGSIPU

[@metleon]

I could barely understand her accent so I turned subtitles on. Further proof that the robots are winning.

[@TheRealmDrifter]

Did she say 'Fucktard questions'?

[@DheerajBhaskar]

TheRealmDrifter factoid questions. I heard it in a weird way as well 🙂

[@smoldogbigbobs]

factoid

[@logiglish-cofounder]

She misunderstood the concept of standardized test. We don’t have any standardized test for college entrance examinations in Japan. We only have common subjects based test. I hope she can be a little bit more accurate.

[@TheReject182]

The answer is yes because University = Learn things and copy past them from your mind to your sheet of paper :/

[@HakunaKun]

Probably it's because the old evaluation systems are probably outdated and the teaching process requires an important review urgently 😔...

[@Classycardrawings]

Who else is watching this in summer 2156? Now how many of y'all watching with your robot significant other? Tell your robocat I said bee boop beepity meow meow

[@CDmc98]

I can't believe the overtime sign isn't more discussed 12:16

[@editorjohn8803]

As an English professor, I know that there is no way for an AI to write a B+ or above essay (based on the standards of the top 50 universities in the world). There may be a great deal of excitement over an AI program's potential, but I have yet to see even one sample circulating anywhere, including the Internet, that shows that AI can write an essay that reveals a semblance of wisdom and critical thinking, which are necessary for any reason for a student to write an essay.

[@Afterlifesinner]

My only gripe with this talks is that she could have elaborated her conclusion a little more. It was worth listening to.

[@Axphey007]

She should also start lullaby yt channel

[@alexjackson4103]

Lmao, at 8:12 she does the body language equivalent of "Please Clap"

[@pigeonwizard]

If its problem is understanding meaning in written languange that might be the reason why AI tends to develop its own language (as it happened with the Facebook AI tests) when it has to "talk" to other AI. The human brain is wired in a way that our language is the best way to communicate for us, but for a machine built differently this might not be the case. Therefore it doesn't perform perfectly in the human communication field, but if it were allowed to develop its own communication system without being shut down it could outperform humans in all fields.

[@Sdhector21]

Jessica Cremonese Can you point me in the direction of those Facebook AI tests? I'm skeptical an AI could develop their own language before even replicating an existing one. I'm curious what exactly is meant by saying it developed it.

[@pigeonwizard]

Hi! Yes, here is a couple of articles from supposedly respectable sources of information: www.telegraph.co.uk/technology/2017/08/01/facebook-shuts-robots-invent-language/ www.independent.co.uk/life-style/gadgets-and-tech/news/facebook-artificial-intelligence-ai-chatbot-new-language-research-openai-google-a7869706.html I cannot find the original website that provided the info to me first... sorry for that. But I read these articles and they explain it quite well. Long story short, the supervisors of the project noticed that there was a change in the pattern of words which did not make sense at first. However it seems that there was a pattern in the "new" language, so they reprogammed the AI with a new rule: stick to English. This is very fascinating!

[@danthetravelman9113]

The AI got an open book open source test while the students did not, more and more universities are expanding their student base for profit, and standardized tests aren't necessarily the best tellers of knowledge. We have all of the same capabilities and resources as AI and that is why universities should teach how to reach solutions with the given resources and technology. More tests should require abstract thinking in order to reach a new idea or solution, and for this reason that is why the computer failed the language questions. Any one of us can find the answer to simple bits of history or math problem, but lets say I ask you. . . How can we effectively change the undesirable situations that await us in the future? You must include a plan that deals with the monetary system, government, international relations and everyday lifestyle. . .

[@alph4966]

暗記型人間≒エキスパートシステム ロボットで十分代替可能なホワイトカラー職が危ない

[@lloydgush]

Don't get all alarmed when robots do what they are good at. Now go ask the same robot to load the truck on a unspecified location.

[@quyetthango2848]

Little UK take me come here !!

[@Sarruji]

Well I would pass any exam if i had the internet as a tool during said exam.

[@MarkVanReeth]

Can someone please develop an AI that can translate whatever she's saying into human speech? I tried listening to the podcast version in the car, but wasn't able to understand half of it. :(

[@植松京子]

何このかわいらしい声📣

[@vincenr8822]

Amazing talk.

[@RoyMuffinStar]

This speech brought an attention to me, personally in my studies. Throughout my diploma learning I've made used of countless memory techniques to ace my exams, combined with true understanding I can bring out the answer that no AI can, however memory was one of the lacking thing, memorisation. If AI could produce meaning and understanding, the end of humanity will be close. I doubt it will happen any time soon though.

[@BJ-xh8tg]

11:11 "...failed to answer (to) this question.." u BOT no hUm@N !!

[@liemnguyenhuynh8245]

thank you for giving me the meaning of AIs

[@Myrslokstok]

But let the kids surf the web as the AI did, and then compere. In reality you have google at your fingertips.

[@uditrockx]

She made me worry.

[@overwrite_oversweet]

I wonder if we can get some of those AIs into teaching.

[@aposslex]

Even the subtitles are struggling. Think I got the jist though

[@drpporto]

Ridiculous questions so far. I am really upset about this. What is the point in an test about this?

[@revdcdon970]

NO - Instructions - Turn in your test at the point you cannot answer any question. Second question - What is your Blood Type?

[@desiredditor]

Hey did u allow the kids to Google just like the ai

[@RidiculousThisIs]

AI isn't thinking, its still doing a very narrow, specific task which is written by a human. This is not AI yet.

[@fleXcope]

Needless to wait till 2020, when university campuses will no longer be necessary.

[@tobiasurey1434]

Lol . congrats of predicting the future

[@crazyprayingmantis5596]

Pretty sure this woman is a robot

[@Xbotto]

video is 13:37 aswell

[@Sdhector21]

I wonder how much better would the students do if they are allowed to Google their answers...

[@suryasharma9410]

Hector Sanchez ; remember one thing . Only those students will answer perfectly which are sharp and having orientated brains . I believe, nothing can match human intelligence, but if we are using our brain, only then. Rather Al would be far better than us.

[@BagoGarde]

What is a good to have when the AI will be doing and replacing everyone?

[@meamzcs]

Sensei why that? I don't think software engeneers are irreplaceable but i think if you are one you will have a job at least almost until there are no more jobs. Because if AI can program, it can basically replace every job with it's software.

[@drewkavi6327]

Bago Garde Then you don't have to work and can live a life of leisure forever why are you complaining about not needing a job

[@kinsmed]

+zomge Yeah, get rid of all those uninformed judgmental people, right?

[@BroCactus]

Programmers will be among the first or second class of employment to be lost. They are literally coding their replacements.

[@meamzcs]

Eric, maybe but as i said, it could write software so fast, that it wouldnt take long for it to replace every other job with its software

[@mohsonkhalafjabbar9655]

about that issue i think ai like notebook (smart notebook)

[@VictorGallagherCarvings]

So "educators" will have to shift from indoctrinating students to teaching critical thinking and logic.

[@kanwaldeeppuri8761]

This thought could be sheared without involving the country

[@Leandro80446]

She is robot!

[@BenTajer89]

"How was this robot able to outperform our students?" - Robot has access to the internet and unlimited memory... The task that the students have is fundamentally harder than that of the robot, give the students a smart phone and then compare the scores.

[@overwrite_oversweet]

The robot could simply load say, 1 TB into cache. And, no, 1 TB is not larger than the capacity of the human brain, whatever that might be, though yes, it's a different type of memory. Computer hardware uses a fundamentally different paradigm to wetware, on which information stored is too lossy for truly accurate reproduction even for "well memorised" items. Humans truly have the advantage in some cases, but in certain tests, "consciousness" and the ability to comprehend is a disadvantage, not an advantage. Maybe we ought to look at optimising tests to measure something more useful.

[@anhsinhvienKyThuat]

I have an idea for a machine that moves objects through space and time using quantum theory and mathematical coding ....... I am in need of advice and suggestions from the you ...........STM

[@nicolasmicaux8674]

Very interesting

[@basharatullah285]

Google could you the same all you have to do is put the information into a system not a big deal

[@ばりーぞ]

I've read her book before. I think we have to bring up the future generation(and ourselves¯\_(ツ)_/¯).

[@joehealy1136]

Please say it's no

[@unitednationscommand9356]

this is why I didn't enter HARVARD. I shoul have been tauhgt in right way.

[@holycow6954]

これ好き

[@andrewclover1462]

"Can a robot pass a university entrance exam?" I'm not sure. Thankfully, if it cannot pass the entry exam, the government will give it low interest loans so it can waste everyone else's time being nursed through classes in college it should have passed in high school.

[@Gribbo9999]

I wrote essays and didn't understand much too. Now 50 years later I now understand a bit (maybe) . The real issue is I took 50 years to get here. Maybe our Japanese robot friend will get there in 50 minutes or 50 seconds if it goes quantum. We might be right to be worried. Very soon the decision to pull the plug on AI will be gone and we won't necessarily be AI's first priority.

[@mohsonkhalafjabbar9655]

many thanks for arab translate شكرا للترجمه بالغه العربيه

[@Nanuedward]

Strongly no

[@theCUBE403]

Will AI speak in bad engrish too?

[@camerondale6529]

So the robot cheated by looking up the answers on the internet. Hardly impressive.

[@janaebert3059]

This is irrelevant. Would you be more impressed if it would have a copy of the internet on its local hard drive? It's not "cheating".

[@Sdhector21]

Jana Ebert Of course it would be cheating. The equivalent for a student would be to have read everything relevant for the test and having perfect memory. Give Google to all those students and I doubt the AI would have beaten more than 10% of them (which would actual be alarming for those students).

[@Myrslokstok]

Hector Sanchez Agree!

[@ZiemniakPospolity]

Why should we think that AI is smarter than a human when AI can use google to find correct answers during the test and a human can't? Give google to the human and he will probably score better.

[@kornx10]

Your shoelace is untied

[@gamingstreams4703]

JEE advanced....challenge

[@jahdeedmanlungat8143]

It depends on how many terabytes and information loaded on a robot.

[@carlhenning]

13:37

[@Simba97]

So you're creating something that you don't even understand?

[@Simba97]

Well now it may not seem like a problem, but if they build self-aware robots then it won't be doing any good to humanity. What good are the children if the parents themselves are complete savages.

[@harleythesalami6956]

These tests of an AI within a limited rule-set and with a single focus are easy for current AI. It's a sophisticated kantian logic upon game theory. Here are rules, Here are successful outcomes. Once AI can do that task while maintaining social norms, whilst managing a subconcious and create focus from willpower, without distraction, while being able to multitask and do all that without being programmed intentionally to do all that. THEN i'll call it AI, until then it's software in a robot, doing what it's told.

[@Sdhector21]

Harley The Salami Exactly. It's impressive and useful, but it's still just a computer.

[@DheerajBhaskar]

Harley The Salami oh my friend, we're all going to be so out classed 😀

[@nryle]

I don't think that is really the takeaway from this presentation. She straight refers to moder AI as "Artificial Unintelligence". Computer's don't currently have any method for understanding what they are reading. You will notice she didn't make a big point of the computer identifying History questions or Mathematical questions better than students. The big point was that the computer was able to answer questions that were strictly reading based questions better than students, and the computer can't even really read.

[@aaronmugabe2125]

There is a reason it is called _ARTIFICIAL_ Intelligence. You have some skewed view of what AI is.

[@qncsc]

it is artificial intelligence ... not mimic human intelligence or human intelligence 2.0 or alt- human intelligence. it needs only to express reasoning in an artificial manner that shows ability to manage domains of complexity.

[@fedswithful]

Is she AI...

[@justrandom6751]

Todai robot ?

[@DheerajBhaskar]

Just Random oh I heard it as todai Robert

[@Kインスタント]

Just Random She says "Todai robot" It means Tokyo Daigaku (university in Japanese) Robot. Many Japanese call the university of Tokyo "TODAI". That is the most famous university in Japan.

[@chenchen1232]

Man-made can't beyond God's creation!

[@seanmcguire7974]

Her jokes are lost in translation.

[@vikramsinha3642]

Madam Gao

[@defaultmesh]

Plot twist: she is an AI

[@adventureswithfrodo2721]

Japanese are rote learners not free thoight.

[@SubjektDelta]

1337

[@littlesometin]

I hate university!

[@AkizaVesper]

Then don't go there.

[@valairlines]

Why would a robot go to the university? If we really need a robot to take a job, why not just program it to do only what he has to do?

[@hqcart1]

That is crap. AI can understand the meaning if it is programmed to.

[@dariusduesentrieb]

I think, she talked about near future problems and solutions and although it's not safe to say that AI will not be able to understand in the next few decades, it is unlikely that it will happen in next near future. However, specialised AI exists and threatens to take over jobs from people who just memorized things to do them.

[@nurajannatpinky]

Bad

[@kazedaizaemon6668]

AIを東大生に近づけるという目標設定。そんな目標にしてしまう発想の貧困。 大学受験で勝ち組になったことしか誇れるものがないんかね。

[@jonathanlebon9705]

1.) We need a universal minimum income (employed or not). 2.) We need to let AI take over everything. 3.) So we can live in a eutopian society.. Alas! Only No. 2 may ever come true....

[@brendarua01]

Jonathan Well 1 needs to happen if 2 does, or people will revolt. And capitalism will collapse with no consumer purchasing power.

[@jonathanlebon9705]

Brenda Rue Exactly! Thing is 2 will happen (eventually) so they need to start thinking about 1 now. There are trials going on in Sweden or Denmark at the moment...doing exactly that. 3 is just wishful thinking... :-(

[@Asgar06]

This is what almost all ppl are overlooking when they talk about AI and they are taking our jobs. Casue when nobody works nobody can buy stuff thus no need for AI to produce anything. So the only solution is the universial income. They are not taking our jobs they are freeing us from the need of jobs. Everything else would result in a colapse.

[@ChristianCTaken]

So basically Star Trek.

[@playzor911]

Asgar06, and I think we are heading towards a collapse. Look at history. Collapses are not the exception but rather the rule. Every society before now has fallen even though some lived longer than others. Greatness is not ensured and we are not entitled to a free world, it is a never ending fight.

[@ujjvalpatel5353]

Any Idea , What to comment when you are one of the firstone to comment ????

[@VCGConstruction]

Hey guys I hope you are having a great day!!! From a small youtuber who dreams of making it big!!

[@Senecamarcus]

This lady thinks we can make humans better if we educate them better that is like saying stuff your money under your mattress and expect it to multiply every day

[@cursebop6055]

No, it really isn't like saying that.

[@Senecamarcus]

XFaded Foil well them tell us whats true so I have some clue and become glue.

[@cursebop6055]

What?

[@Senecamarcus]

XFaded Foil read more books and expand your knowledge beyond your comfort zone

[@cursebop6055]

I was referring to your inadequate grammar. It sounded more like you were rapping than trying to prove a point.

[@サトウヒロアキ-y5e]

日本で2019年のベストセラー「AI vs 教科書が読めない子供たち」の著者・新井紀子氏の講演、もっと早く見たかったです。 復習を兼ねて、非常に参考になりました。大変良い内容ありがとうございました。

[@Erikulum]

Wait, you can't use the internet for university entrance test right? That robot is cheating!

[@suryasharma9410]

Erikulum ; but they are talking about the capabilities; even if humans are allowed to use internet, they will not be performing that well.

[@Sdhector21]

Surya Sharma Yes they will. If students were given Google I doubt the AI would be better than 10%, maybe even none of them.

[@suryasharma9410]

Hector Sanchez ; I totally agree with you , but if we take time duration in consideration, then they are far better than us. I BELIEVE humans are great and having incredible powers . If human brain is able to create AI , then it is already very far in that race with AI.

[@5even5eals]

The one who is cheating here is university. Test answers should not be googleable in the first place.

[@suryasharma9410]

5even5eals ; all the questions are from our discovered and invented knowledge , how can you prevent that ? Google is a search engine and not a helping hand in the examinations.