Deep Learning: Intelligence from Big Data

Deep Learning: Intelligence from Big Data
Tue Sep 16, 2014 6:00 pm - 8:30 pm
Stanford Graduate School of Business
Knight Management Center - Cemex Auditorium
641 Knight Way, Stanford, CA
A machine learning approach inspired by the human brain, Deep Learning is taking many industries by storm. Empowered by the latest generation of commodity computing, Deep Learning begins to derive significant value from Big Data. It has already radically improved the computer’s ability to recognize speech and identify objects in images, two fundamental hallmarks of human intelligence.
Industry giants such as Google, Facebook, and Baidu have acquired most of the dominant players in this space to improve their product offerings. At the same time, startup entrepreneurs are creating a new paradigm, Intelligence as a Service, by providing APIs that democratize access to Deep Learning algorithms. Join us on September 16, 2014 to learn more about this exciting new technology and be introduced to some of the new application domains, the business models, and the key players in this emerging field.
Moderator
Steve Jurvetson, Partner, DFJ Ventures
Panelists
Adam Berenzweig, Co-founder and CTO, Clarifai
Naveen Rao, Co-founder and CEO, Nervana Systems
Elliot Turner, Founder and CEO, AlchemyAPI
Ilya Sutskever, Research Scientist, Google Brain
Demo Companies**:
Clarifai | SkyMind | Ersatz Labs | AlchemyAPI
** Follow (@VLAB) on Twitter and Event Hashtag #VLABdl

[@siyengar9821]

This is by far one of the most insightful discussions regarding deep learning and machine learning and its impact going forward. Thank you Steve for doing this.

[@quantummath]

Stephen Jurvetson .. what an eloquent speaker with such agile mind. Great panel! thanks for sharing.

[@freddykrueger5503]

A. M. Goudarzi he sure talks fast

[@adaylateacoinshort]

+Freddy Krueger Imagine what must be going on inside his head given the limits of the human mouth

[@quantummath]

+Freddy Krueger speed is perhaps the most obvious feature of his speech, but under the surface there comes the complexity of the Concepts, their inter-relationships and a good degree of mathematical understanding of the concepts. It's more complex than "just the speed".

[@r3nd3rm4n]

I usually watch talks like this at 2x speed but I have to actually turn that off for stephen cuz he's naturally 2x

[@HandsomeSubu]

renderman kashmlrbajn

[@HELLios6]

Really interesting. Finally someone who doesn't talk at a rate of 10 words per minute. More information delivered in shorter time

[@Panj0]

Oh how I appreciate people who don't speak so slowly I have to turn the speed up :)

[@r3g4rds]

+kcuf0 Pfft, even this guy talks too slow for me.

[@Enigma758]

+MrRabastan I recorded this video and then played it back at 4x speed.

[@r3g4rds]

i went back in time then got all my time back afterwards.

[@ПавликГерман]

it's very convenient to see speaker instead of slides during his explanation what's going on at the picture.

[@ytyt1611]

Usually I speed up such videos to Micky Mouse. But he ist pushing it. Dense and informative. I like it. 1x ist just right. Extremely fast speaking, but still comprehensive. Opening the mouth the right way is also a skill.

[@wesamadel3612]

I'm at 27:16 and I'm frustrated cause i can't see the clips that the audience see really bad director

[@CharbelKeyrouz]

Great video about Machine Learning with a perfect speech speed.

[@jedics1]

'everything is going to go great until it becomes self aware' haha, I cant believe this comment wasn't expanded on as it is clear, at least to me and the presenter that this is the ultimate destinations artificial learning will achieve. This will happen sooner than any of us think, indeed within the next 20 years and then what, we would of created ourselves out of a job, the job of being at the top of the food chain, the job of being the most evolved species on the planet. and we all know how well that has gone for all the species below us. Im both excited to see it happen and fearful of when it does. Lets just hope that its empathy evolves beyond ours when it's intelligence does.Of course it will need to feel for this to occur so I hope its made from more than a lump of silicon by then. Or maybe its lack of feeling is what will prevent it from ever becoming self aware in the first place! to reproduce ourselves is in our programming, to reproduce a superior version of ourselves is the logical evolution of this program and so long as it is indeed superior eg. spiritually and intellectually Im ok with it taking our place.

[@vsrat]

totally accurate according to the current layout of my neurons

[@elidrissii]

This guy is very animated, and a very good speaker. He uses everything he can use to communicate, hand gestures, weird facial expressions, body movements, tone. Wonder if he trained for this.

[@HeliosWorksAV]

Very concise opening statement explaining exactly what Deep Learning is all about. Definitely the video clip of choice when trying to explain it to others.

[@RobertHouse101]

Oh, thanks. That's good to know. Seriously.

[@frzzldotcom]

Mind-blowing and extremely thought-provoking.

[@rikimitchell916]

excellent host..fantastic command of the English language...

[@pondeify]

segway segway segway....

[@AlexBonillas]

Is this guy a Rapper? He is throwing like 5000 words per minute

[@AnantMall]

I'm convinced he's a robot with deep learning imbedded!

[@JoeSmith-jd5zg]

I like it! Gets a lot of info in in minimal time.

[@csskiller]

I thought I had youtube playing at 1.5X or something holy shite that guy

[@toshinakae6397]

he repeats words a lot though, like um, uh, and, you

[@inoffensiveusername4684]

he has a completely clear mind, everything is just flowing for him without any mind fog. Even watching this video feels looking through a foggy lens lol, his mind is more blurred with anxiety than lack of clarity of thought.

[@stevechaszar2806]

+ had me at Wolfram alpha... remarkable insight, thanks for sharing... how thoughts become things and moments in time. The math of life is thought

[@reveal102]

These are some of the most important ideas of our lives being discussed here. Wow.

[@le-manu298]

Love it. Just the big data example of a phone book. :) Actually that is structural data. Anyway..Very good presentation

[@mypsychhealthcare3183]

Excellent lectures.

[@Gunth0r]

Don't be discouraged by the 'actually engaging in this field is hard' comments and statements in this talk. ML is incredibly easy if you are patient enough with the documentation. You don't even need to understand the math behind it (but it does help tremendously if you work your way through the papers). Just start with simple problems and work your way up. Some examples provided below. 0. Get to know the libraries, tools and algorithms you'll be working with (plenty of ebooks and papers are freely available, as well as open source code from major ML companies, youtube has great tutorials, lectures and examples). You should have some programming skills and be familiar with (or motivated enough to learn about) processing data or handling a database system. There are also plenty of API's that are useful for the generation of data sets. 1. Get some equations of moderate difficulty to solve and train a small NN to solve them for you. Run through different strategies to optimize. 2. Design and build a model to put images through a convolutional network to detect features (detecting sunrises and sunsets should be easy, there's a clear gradient and a bright light in the middle, easy to detect features. you could go one further and differentiate between rising and setting sun based on the colorspace, morning fog/dew, tides if they are beach pictures, etc). 3. Predict tomorrow's weather based on the past 30 days and/or historical weather data for your locality and day of the year. 4. Classify MIDI files (by genre, f.e). Just like with image classification, but a bit harder. 5. Use a movie or book dataset (api, db, ...) and let it suggest a movie or book based on what you've already seen and liked. 6. Perform sentiment analysis on an inbox of your choosing (e-mail, fb, yt, twitter?) or another data set. 7. Develop an AI (or ghost player) for an open source game (no matter how trivial it is) or a triple A game if you are up to it. 8. Lastly, develop a trading bot, based on sentiment analysis coupled with an ensemble of indicators. Let the statisticians inside you all come out!

[@KCM25NJL]

The Clarifai platform is very interesting.........able to contextualize video in near real-time. If someone could somehow take the meta-data generated from a live video feed and pad it out in some sort of natural language construction algorithm to form an intelligible description of the scene.......... this could pave the way for robotic sports commentators :). OK maybe not that much of an area for improvement, but I can certainly envisage things like natural language scene commentary from home security systems. e.g Motion Detected - Audio transposition of processed meta data is played out in the home - "Uncle Gerald is approaching the front door, don't try and escape by the back, as Aunt Bouquet has it covered" Surely not a giant leap?

[@majdiz3453]

I wish the presentation was embedded in the video!

[@xitec75]

9:15 As far as I remember, unsupervised learning (if you are classifying) gives you a class/group, not the definition of the cat. The data is unlabeled. On 28:52 we see some connected nodes, the speaker talks about deep learning, but the gfx has no clear definition about any input layer, hidden layer or output layer but seems to be able to classify some images (supervised learning). Finally I don't want to know if or how this technology is used to identify targets/people to kill them with weapon systems. Who will be responsible for this? The people who invented the algorithms or the people who used them in their weapons?

[@eleanor309]

Helping towards my research with the IDM

[@brunon.8962]

AI is coming and it will teach us that intelligence is ironically simpler than we thought.

[@kw_cooper]

where can i get one of those circuit brain t shirts, the grey one?

[@pascalpeeters1290]

He's just nervous and excited because he knows he has a point there and therefore starts rambling hoping that nobody interrupts him before he's done explaining what he thinks is correct ;)

[@pascalpeeters1290]

i mean Steve...

[@PanaeolusLBK]

Very interesting thoughts being explored here, great video

[@mbaxter22]

This is the Singularity unfolding before our very eyes.

[@ALPHADOG1900]

Man I would have a conversation with this guy 24hrs straight.

[@guyunger]

google should open the google restaurants which uses only recipes found by deep learning AI using online recipes and their user reactions/ratings

[@OleksandrFialko]

These machines will soon become intelligent , more intelligent than people.

[@Intel1502]

Define intelligent, and provide evidence for you claim.

[@Intel1502]

***** On your bike mike.

[@Intel1502]

***** Go fuck yourself pathetic attempt at a funny name

[@Intel1502]

***** U mad bro?

[@Intel1502]

***** I know you are~

[@cuckingfunt9353]

Those grids pictures and filters at around 36:00 are eerily similar to visions you get with migraines .

[@lizenus]

I didn't know James Woods was an expert in deep learning.

[@osubassdude]

This is awesome!

[@ewliang]

Very interesting video.

[@alexmoen4235]

Great video!

[@brianjanson3498]

I could have watched three episodes of Gilligan's Island instead of this. Time I'll never get back.

[@KR-wf5km]

Nervana is an excellent name

[@mkanalysis]

can you do a lecture on how to speak like you ? Amazing video. Thank you

[@liamhoward2208]

mostafa8998 take Adderal

[@DavidS992]

I am looking for the part of speech from this video which I cant't find now, something like "next 5 years when you'll see something that will make you say "wow how did they do that" - it's probably deep learning". Can somebody show me it?

[@metalandai]

2:43

[@DavidS992]

Tomi Kallava just in the beginning, ech, thank you :)

[@sikor02]

That "deep neural networks" aren't just multi layered feedforward networks that were invented decades ago?

[@SiddarthRIyer]

29:06 It sounds like we're training the system in a manner somewhat similar to what we'd do if we were training a pet.

[@HealthyFoodBae_]

Very nice

[@Beingskilled]

Normally I watch at 2x but not this one. This is already 5x.

[@SteveWhetstone-UXDesigner]

They've almost got it right. AI needs to involve like intelligence. Babies learn to see by learning to move their eyes and arms and such first. There's a feedback loop that seeks novelty that is hardwired. step 1) babies eyes closed: This is essentially sensory deprivation which we know is hardwired into us to be avoided so the baby follows pscyhology rule number 1 when experience discomfort agitate for change. the absence of simulation from eyes closed stimulates random behavior which eventually results in eyes opening and then the discomfort of sensory deprivation is removed. So the baby learns to open it's eyes. step 2) babies eyes don't move: a picture that doesn't change is not very sensory rich. there's a feedback loop in the eyes and the rest of the brain that builds gets desensitized if things don't change. So apply rule 1 and agitate for change to avoid the discomfort of seeing the same thing. the eyes move and that's exciting sensory data. so the baby moves it's eyes step 3) babies eyes learn to look at things and not just move at random: At this point the babies eyes can get sensory stimulation by moving the eye muscles the sensory input to the eyes change. BUT, not all eye movements are created equal. looking at a blank wall doesn't have much sensory deprivation relief. looking at the edge line between a wall and a ceiling or a corner where three lines converge is more stimulating to the eye than looking at a blank wall. So the baby learns to focus the eyes on areas of interest and learns things like edges and starts to build. Step 4) after learning to move the eyes babies are starting to learn at the same time about how to get food. rule #1 agitate for change applies and they cry and that gets them food. etc. etc. I don't think anyone working on AI will actually read this, so i'll let them figure out the rest or ask me if they want further details and the next steps.

[@SteveWhetstone-UXDesigner]

I guess this is what we call a learning principle. Gestalt theory is good for this and I"ve studied the brain a lot. Why doesn't someone give me a job for creating AI learning principles. First hint. you need to attach a motor to the camera and let it look at different things and simulate Saccades and image stitching of different views. You also need some way of affecting the what you see like an arm that can change the what the vision receptors see. And finally you need some high level rewards like push the button with the arm and get some extra stimulation or a novel view or rewarde such a food.

[@SteveWhetstone-UXDesigner]

Steve Whetstone another important feedback principle is prediction. This is probably fundamental rule number 2. The individual neurons in the eye can predict what they expect to see when the eye moves. it's what the neuron next to is saw just before the eye moved a little. When the predicted input from the camera or eye differs from the actual input it creates stimulation. The eye focuses on movement or change and points where the environment can be influence. blah blah blah. I keep assuming someone besides me in AI knows all this so why write it down.

[@SebastianSchwank]

valid code measure speed mutate simulate measure speeds select the "best ones" mutate simulate until speed is minimal WITH compilerdead WITHOUT compilerdead

[@thomasnorman7902]

So what they're saying is learning computers are getting much more powerful. Currently they use GPUS, but the process works even better on Quantum computers, and also might work much faster on ASICS. All unskilled labor is under threat from deep learning robots that can teach themselve by watching youtube videos.

[@SergioArroyoSailing]

Resume @ 22:28

[@the-arkk]

Great talk

[@thomasnorman7902]

great vid, but the guy mumbles and talks too fast.

[@taychoochuan5494]

can anyone provide English Caption Subtitle for this video? Thanks in advanced!

[@SuperMantaraya]

Is that a joke?

[@zahirjacobs716]

It's hard to believe this guy invested in Theranos. But he did.

[@Alex-qb1nt]

He also invested in Hotmail, Skype, Tesla, SpaceX and others

[@Myrslokstok]

Most scary about this is that the smartest people work with this now. It is right now scary.

[@brandonbahret5632]

I wonder if google is working on indexing the youtube videos with their image data so we can do reverse image searches for youtube videos.

[@Mathias_NB]

Good question.

[@human-mademovies768]

+Brandon Bahret they'll do it some day I guess

[@erikm9768]

Step into analysis please.

[@TheCastleVR]

How do I apply this to asset creation in video game design?

[@lau4893]

So your ai gets better (so instead of hard mode being : low health and low damage) the game is just better at beating the player

[@strangerenk]

thanks for sharing

[@szym1]

I like the energy

[@arcanaorbis.thesecretsofth4535]

Apropos the way he talks: though I am a Swede I don't have any problems following his thoughts. I think it's liberating with a speaker who doesn't have a script to follow and who has lots to say. This shows that the presenter knows hir (gender neutral) hir topic really well.

[@zan700]

Just say "his", he's a man/male. Don't be so liberal.

[@joelcastellon9129]

Just ask Andrew Ng he can give you clear answers in 5 min instead of all this chitty-chat

[@scada67]

+Joel Castellon Agreed.

[@donickoce1]

+Joel Castellon Hey, I found a -74% coupon to Learn Matlab Udemy course www.udemy.com/learn-matlab/?couponCode=ML25

[@jasonbates9906]

This guy is the Mark Rosewater of whatever it is he's on about.

[@drmosfet]

So neural networks started in the 80's? in the 80's they started getting back into a forgotten technology from the 50's "neural network", because digital computer ≠ 42

[@DavidS992]

I wish I had such t-shirt 44:00

[@mikezooper]

We need a neural network to train the host to not say segueway every sentence. So annoying.

[@jurvetson]

Photos and comments from the event: www.flickr.com/photos/jurvetson/15105407149/ I was reminded of this talk as the fellow on the right just had his company acquired by Intel. I am writing a blog post on that today... Draft:facebook.com/photo.php?fbid=10157283678955611

[@palfers1]

If the Google guy is right about breakthroughs in learning algorithms, then why not set Deep Learning on the problem? :)

[@SebastianSchwank]

where rent a neural network work data & processing cluster

[@robertweekes5783]

This deep learning image recognition will enable robots to interact seemlessly with the environment in the near future. "Jack get me a beer." "Ok Joe." Lol

[@ne0ph0enix]

Problems that exceed human understanding? all humans? ever?

[@tonybridgens6577]

Hmmmm...... To think that I made my living with a slide-rule! Oh well, I can also remember seeing a doodlebug flying over Clacton.

[@kazimrahman7499]

:) .... it's the information AGE....get smart pplz....It's not that important how fast or slow he is talking.....please judge people according to the quality of knowledge he/she holds and delivers to the people.

[@pauldacus4590]

Christopher Walken look alike..

[@SebastianSchwank]

scale infariance?

[@z44sms]

I disagree with that open source statement made has been only recent. Software has always been open in IT he is too young to know. It's hardware that has been closed which is where his background was. You could get the entire code for every OS except Microsoft. Graphics has been open in it's algorithms as well as database sorting, memory management, compression, video formats, image formats, and disk io. It's always been hardware that was hidden. In fact, I can think of only a few things in software not published in some paper of how something works. Even things you would think should be secret like encryption, ATMs, or encoding.

[@AhmedAlsayadi]

athan sound in 9:40 :)

[@vipergts109]

Really you got a convention full of highly intelligent dudes, talking about artificial intelligence, and they cant even get a projector to show full screen, I find it ironic and hilarious at the same time, LOL

[@robgannon1964]

What terrible camera work. I so wanted to see the slides, but they're only shown for a couple of seconds each.

[@dogg724]

Um that first panel speaker, um, uhh, ummmm god dammit....

[@spacemonkey1717]

Is that a young bernard, I'm sorry i meant ARNOLD, DUN DUN DUHHH!

[@yssun7679]

I thought it was 2x speed.

[@JustOneAsbesto]

Resistance is futile.

[@shimondoodkin]

just skip to 22 minute

[@jonathanbaraldi]

Jade 2

[@antolepore]

But that sir all in black is a gravedigger?

[@MickySupreme]

depths.

[@davidleitman]

data agriculture: the new hunters and gatherers, new anthropological history, but no redemption in sight. We need Jesus if we would have eternal life with G-d.

[@elms333]

Think again, Juventson i Estonian and he knows why he chose corrpotate with Elon :)

[@JazevoAudiosurf]

first i thought: did he shat his pants?

[@stephenblackwell2738]

second speaker needs more practice speaking in front of a audiences so that he will over come saying umm so much not saying this to be a butt I wish people where more honest you can only impove with honest feed back thanks stephen

[@Larrythebassman]

Mmmmm intellectual champagne 🍹🍸🍷🍻🍺🍾cheers

[@tophertodd2366]

This guy said "fungible"... I don't think so Chachi... Next!

[@santiagowechsler]

+Topher Todd what do you have against fungibility?

[@yarzarmin975]

ok

[@CaptainLoony]

What's up with his pants?

[@SuperMantaraya]

Let's Masonic sing along everyone!

[@MrQuaazga]

His lips are spinning and are in superposition. He should reach entanglement soon! Bring on the quantum computers. I am so sick of these humans

[@lunaticculpepper7003]

#thought#Greart#Data#ET#MixWith#Aii#Love&&&£iG#H+

[@GreenManXY]

Interesting topic, boring presentation. Uhm, aah...stopped listening at 30 mins.

[@JiddraInteMedZohan]

"People who actually understood this stuff"??? I'm having major problems understanding uhm uhm uhm uhm..... I'm obviously not intelligent enough to understand a lecture where every other word is "uhm". Need to bruch up on my "uhm" language....

[@bjornsundin5820]

*UUHHUMM*

[@rootready]

fuuuuck, so fast english speach, cannot understand it