What we learned from 5 million books

www.ted.com Have you played with Google Labs' NGram Viewer? It's an addicting tool that lets you search for words and ideas in a database of 5 million books from across centuries. Erez Lieberman Aiden and Jean-Baptiste Michel show us how it works, and a few of the surprising things we can learn from 500 billion words.
TEDTalks is a daily video podcast of the best talks and performances from the TED Conference, where the world's leading thinkers and doers give the talk of their lives in 18 minutes. Featured speakers have included Al Gore on climate change, Philippe Starck on design, Jill Bolte Taylor on observing her own stroke, Nicholas Negroponte on One Laptop per Child, Jane Goodall on chimpanzees, Bill Gates on malaria and mosquitoes, Pattie Maes on the "Sixth Sense" wearable tech, and "Lost" producer JJ Abrams on the allure of mystery. TED stands for Technology, Entertainment, Design, and TEDTalks cover these topics as well as science, business, development and the arts. Closed captions and translated subtitles in a variety of languages are now available on TED.com, at www.ted.com/translate.

[@toothasaur]

they're both really good at making their data entertaining. One of my favorite talks recently

[@dravarian26]

I remember this from watching the Tedx talks. Its pretty awesome that the director version is so good at picking out the awesome lectures from the all right ones.

[@soshimyk]

Just an accreditation note: the comic panel used in the presentation was not from XKCD. It was taken from a web-comic called Dresden Codak ( the comic used was from May 2nd, 2008).

[@GHortaV]

Loved the talk. Very informative and very entertaining. Those two together can't fail.

[@Yaarrr]

This talk was awesome! Informative but also very entertaining.

[@liquidminds]

it's personal conversation > video > textbook nothing beats audio + video + feedback so far ;-)

[@Evija3000]

I love that Google tool. Been playing with it for at least an hour. One of the interesting results I found was that words 'kind, give, love, charity, forgive, pure' have been dropping in popularity for the last 2 centuries, but suddenly around year 2000 start becoming more popular again. And the words 'need, want use' do the exact opposite. Maby there still is hope in humanity. :P

[@mowriter]

@dayati : the role of censorship in the arts and communication was highly relevant information. Also the evolution of language (Thrived/Throve, Beft/Best). The Beft/Best thing of course was less about language (that was always meant to be an "S") as much as what I believe was standardization of letters, arguably as part of public education?

[@brazilianirishcomedy]

loving the ngram was on it for an hour before i noticed how much time went past

[@PickUpLiveLife]

That webcomic in reference is actually Dresden Codak, but xkcd is just as cool.

[@Flyborg]

Random correction: Although the "stand back I'm going to try science" quote is from XKCD, the image is from an equally awesome comic called "Dresden Codak". And on the image it says "I will do science to it". Very different types of comics, but if you like one you'll probably like the other.

[@penetrationskommentar877]

Look at all these subtitle languages! Amazing

[@McArrowni]

That was an image from Dresden Codak, not XKCD. But fewer people know about that one because it updates like, once a year.

[@ABitOfTheUniverse]

Relativity vs Quantum Mechanics, 1500 - 2008 First Appearance: Relativity: 1503 Polyhedron: Volume 9, Issues 13-18 Quantum Mechanics: 1567 J.C.S. Faraday II: Volume 74, Part 3 Subject, Date, Results - vs - Subject, Date, Results Relativity, 1500-1800, 174 - Quantum Mechanics, 1500-1800, 9 Relativity, 1801-1900, 29,200 - Quantum Mechanics, 1801-1900, 80 Relativity, 1901-1950, 251,000 - Quantum Mechanics, 1901-1950, 32,900 Relativity, 1951-2008, 1,080,000 - Quantum Mechanics, 1951-2008, 1,020,000

[@egokick]

00:15 start of talk

[@goshinbi44]

one of the more interesting TED talks in a while.

[@Gmagee]

Is there a way to include web pages words as well as books?

[@ABitOfTheUniverse]

I love this bit of Google Labs, but I still feel, from playing with it for the last hour, that it has some bugs. Their are many "books" called "Catalog" that exist from 1500 (the farthest back you can set the time counter) up to 1800 that say they contain references but will not show the pdfs they seem to represent. There are also, what appear to be journals and magazines that are dated from 1500 - 1700 that have printed, color, pictures that look like they are less than 60 years old.

[@Pyjamas22]

That was a great reference to Hitch Hikers Guide To The Galaxy! Too bad no one got it

[@cnmaster01]

@McArrowni I love DC wish it was more frequent....

[@dayati]

I thought this talk was going to be about some profound truths they learned from 5 million books. Instead, they charted how many "A"s people used in the word "argh". Great job.

[@udaybharadwaj9387]

You have done really some great stuff!!!!

[@SEThatered]

Amazing what new researches people can make today. Thank you Google, i hope i can work with you one day!

[@niriop]

Culturomics: you first heard it here in 2011 ladies and gentlemen! What a fantastic creation!

[@Merthalophor]

This is mindblowingly amazing. Holy damn.

[@starburstayla]

@LowestofheDead The webcomic XKCD had a comic featuring the line "Stand back, I'm going to try science"

[@ABitOfTheUniverse]

The real trick is not to have all information but to be able to prioritize it, to search for what you want, when you want it. You want to learn calculus? A search engine with a tool like this could help you find the right teacher that uses words you understand to explain concepts to you in your vocabulary. You put the right words into a good search engine and you get the results you want. A perfect tool to find the perfect information for the perfect result. That is what Google strives to be.

[@Dixavd]

That was not only enjoyable but also interesting. Especially the graph on Marc Chagall; I have to do a presentation and work on censorship in Nazi Germany. Explaining what they did is easy, but showing sources is much more difficult (since a lot of them were censored haha) but this is such a fantastic graph to show it. What coincidental luck.

[@ABitOfTheUniverse]

@dayati Well, September 19th was "International Talk Like A Pirate Day", seemed kind of fitting to me.

[@ilotitto]

and this is how you save the world. Bring knowledge to people, not to companies

[@ErichoTTA]

Very nicely done.

[@simonepemp]

Awesome! Brilliant! Can't stop playing with the Books Ngram Viewer

[@EricELT18]

Illuminating and inspiring!

[@mandypac2854]

They make this so insanely funny. Ngrams are great. They tell us so much about ourselves.

[@NetSelect]

Brilliant talk!

[@LockeNotLoki]

@113Doctor How about checking the phrase 'lights in the sky" or "lights in the air."

[@AlderDragon]

@damarh As a mathematician I find this very interesting.

[@BassAceReturns]

So... Trevor Moore is a statistician now... Seriously though, awesome talk.

[@moneycrab]

This talk was high on the awesome axis

[@zuppers]

Best talk in a long time!

[@chessdude67]

Excellent! Thumbs Way up!

[@ABitOfTheUniverse]

I really wish EVERY BIT of information was publicly available TO EVERY HUMAN on Earth. I know there is a lot of potential for problems in that sort of a scenario, but there is even more potential for productivity. Consider how much trust would be shared. Without secrets there would be no confusion. Everything could be figured out, every action explained, we could UNDERSTAND EVERYTHING humanly possible.

[@darucha17]

Imp has few meanings. One of the meaning of 'imp' is 'creature' that looks like 'little devil'. Another meaning is 'to give feather'. I think sagar mean it in 'poetical' word and I 'think' he means something like this: Imagination gives you more 'things' than knowledge.

[@Christosch_]

I love it. Thanks!

[@DahSlayah]

Awesome talk

[@srgwarcock]

@egokick thankyou... i just wanted to let you know: good luck and we're counting on you

[@kevindrake8686]

I'm afraid of what happens when they digitize everything and the server crashes

[@TharsiSera]

06.35 LOL hes rapping.....

[@DeathSoulkt]

0:15 Here we go.

[@KingKong19777]

God-damn... The guys at google never cease to amaze!

[@poarak]

Lightly shaky talkers, but a very important talk.

[@MrJosh6889]

Fun talk. These guys are really funny!

[@darucha17]

It's kind of creature...but I guess it has different meaning as in: "gives you more"

[@BitcoinMotorist]

@AndrewDBarker Left Turn by Tim Groseclose or Slander by Ann Coulter

[@codingoutloud]

"Stand back I'm going to try science" is on an xkcd t-shirt

[@MrPointless]

I think that aaargh-thing might have something to do with Monty Python's Holy Grail, actually :)

[@DrogoBaggins987]

With a good text book and teachers like these guys I might have been able to get through statistics.

[@gulllars]

Now this is awesome, as in awe inspiring. Totally going to try that thing out now. I wonder if they have suggestions like in normal searches for thing people search a lot about, or statistics over what people query the engine about the most. I bet there are a lot of people amusing themselves looking at historical populatity of fetishes and other wierd stuff.

[@psycool666]

How much is A Video worth in comparison with Words. Because I'm too lazy to read textbooks, I love watching Video Lectures.

[@BurkeLCH]

amazing

[@ABitOfTheUniverse]

@D5932 Why wouldn't it be? T and t are 2 different codes If you say Big Apple or big apple they clearly mean different things. Any High School English text will explain the rules of capitalization.

[@gamzer]

Wow the crowd is way too enthusiastic! Great talk though

[@vdubs4life1964]

good stuff!

[@huwaw]

I think they were kind of CUT? They were like rushed at the end?!

[@SkatingErinsMom]

The English subs link goes to the mp4 file.

[@GrieferKing007]

.... that crowd laughs at everything .....

[@SailfishSoundSystem]

I typed (war, peace) into it. Pretty interesting. There was way more talk always about war. And you can see World War 1 and World War 2 spike like crazy. The U.S. Civil War spiked, but not as much.

[@jung123456]

Check facts on Marc Chagall. Look him up in only ONE BOOK.

[@ABitOfTheUniverse]

@D5932 There are 286 results on the word Internet from 1500 - 1711 McKinney's consolidated laws of New York annotated: Book 27 AND Constitution of the United States annotated Both mention the word Internet in 1501... so IT says...

[@lil-mi-777]

I cannot stop giggling when I was watching this video:)

[@TheChanclasVerdes]

In wireless communication, a picture is actualy worth a thousand words, in terms of data capacity of a radiowave. :P

[@petravanessa]

@petravanessa definitely*

[@ThomasHaberkorn]

@paradoXeI yes.

[@royohz]

YOU HAVE TO BREATHE HARDER! YOU'RE NOT TRYING HARD ENOUGH!

[@Daniel15au]

Guy on the right is pretty cool, even if his voice keeps breaking :P

[@christian4ever4]

What's imp ?

[@tarohoa]

knowledge to the people

[@khaughey66]

I want a t-shirt with Anrrgh on it. (Only the n has gotta be superscripted like it's supposed to be) I simply must have one made!

[@nonchalantd]

Except for the fact that they're both smart and worked together, these presenters seem to be diametrically opposed.

[@paradoXeI]

Am I the only one who finds intro/outro audio volumes normal ?

[@LowestofheDead]

@dashiellv @starburstayla You're right, they were referencing two webcomics, thanks peeps! "I'm going to try science": thinkgeek [dot] com/tshirts-apparel/xkcd/dacb/

[@poonamabbi7448]

Do you know Google has DISTORTED the books depending on which ones they decide need to "have alternate explanation/interpretation?"

[@florianwicher]

Wow, Michael Shanks sure reads a lot.

[@gjsterp]

Try putting 'genesis' and 'evolution' together on nGram. Good to see evolution winning...

[@gigantibyte]

"Flying cars" was most mentioned in the early 1800s? Am I reading NGram Viewer correctly?

[@mccartycris]

@IAmKingBonzo lol, you definitely got the point of that talk

[@Nablydatelru]

Вот кстати уютное место vk.com/ngram_viewer для обсуждения Ваших Google Books #Ngram. Там прямо на стену можно добавить скриншоты своих исследований..

[@danebeach6558]

SO FREAKING INTERESTING!!!! :D

[@mojoismyrealname]

way to drop xckd

[@LowestofheDead]

That's not xkcd, that's Dresden Codak! And it's 'I will do science to it!"

[@dashiellv]

@LowestofheDead You are wrong, just search for it. It's on a t-shirt even.

[@steve0281]

The guy on the right is channeling Sheldon from the Big Bang Theory.

[@dragonvegeta]

Aaaargh! The loud intro music :P

[@acajudi100]

So many books and so little time, so now I am using audibledotcom, and making video and journaling .,

[@creativetampabayphotography]

Interesting

[@SiCGaming123]

@paradoXeI On my headphones... it is too loud.

[@UTubeSL]

@dayati I too was hoping for something a little more profound... they could have afforded to be a little bit less the comedians and concentrated on demonstrating the potential of this resource.

[@TheSpankymonkey]

If i wanted to understand about a planet and there was existing biolgical organisms i would set them to work telling me everything there is to know rather than do the work myself. And here we have it, everything you need to know about earth, it's inhabitants, cultures and memes all in digital format. Thanks planet earth.

[@petravanessa]

@paradoXeI yes, you most definately are

[@DeoMachina]

As soon as I saw that guy I thought "oh no, this guy probably likes XKCD". THEN HE HAD TO GO AND REFERENCE IT I GOT SO MAD