Yoshua Bengio: From System 1 Deep Learning to System 2 Deep Learning (NeurIPS 2019)

This is a combined slide/speaker video of Yoshua Bengio's talk at NeurIPS 2019. Slide-synced non-KZbin version is here: slideslive.com/neurips/neurip...
This is a clip on the Lex Clips channel that I mostly use to post video clips from the Artificial Intelligence podcast, but occasionally I post clips from other lectures by me or others. Hope you find these interesting, thought-provoking, and inspiring. If you do, please subscribe, click bell icon, and share.
Lex Clips channel:
/ lexclips
Lex Fridman channel:
/ lexfridman
Artificial Intelligence podcast website:
lexfridman.com/ai
Apple Podcasts:
apple.co/2lwqZIr
Spotify:
spoti.fi/2nEwCF8
RSS:
lexfridman.com/category/ai/feed/
Connect with on social media:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman

[@LexClips]

Outline of the talk: 0:00 - Introduction 0:56 - The State of Deep Learning 2:20 - System 1 and System 2 4:09 - What's currently missing in deep learning 8:56 - Out-of-distribution generalization 12:16 - Compositionality 16:08 - Contrast with Symbolic AI 18:11 - Attention and Consciousness 27:07 - Consciousness prior 32:17 - Meta-learning 39:09 - Operating on sets of objects 41:21 - Recap 44:48 - Question: moral implications of building machines that are conscious 46:18 - Question: Integrated information theory 47:17 - Question: Spatial prior 48:28 - Question: Symbolic AI 50:51 - Question: What is a data distribution? 52:05 - Question: measuring progress 52:52 - Question: causality 53:34 - Question: relation and associative memory

[@carlossegura403]

I am so glad I found this video. The presenter gives rise to various useful points!

[@ans1975]

I was there, and after 8 months I am still studying what he said...

[@godbennett]

Excellent talk Bengio

[@DrShaikAhmad]

Great talk

[@billykotsos4642]

THANK YOU

[@Siiimo534]

Great man..

[@fennecbesixdouze1794]

It seems to me like there should be a way to run proof searches where you are constrained to converge to logical proofs but you allow a pre-trained intuitive neural network to make the decisions about which paths to explore first. Another idea is some non-symmetric adversarial system where one half of the system is optimizing to imitate human creativity and the other system is optimizing for logical correctness. A left brain/right brain system.

[@brandomiranda6703]

What is the main argument professor Bengio has against hybrid systems (DL + Symbolic AI)?

[@TumishoBillson]

I was part of the #NeurIPS2019

[@kkjc]

Thank you very much Professor for a so bright talk. Just a question : the System 2 Deep Learning architecture showed is very similar to the Bayesian network Open-universe Probability Model - OUPM (section 14.6.3 of the book of Russell and Norvig: Artificial Intelligence, a modern approach). Is possible to elaborate about the difference, please?

[@rodrigobraz2]

Bayesian network Open-universe Probability Model - OUPM is just one possible way to instantiate a System 2, but it is not neural network-based. Its main capability is to perform reasoning in the presence of probabilities, without making an assumption that there is only a fixed number of objects in the world. This is fine and useful, but Prof Bengio is also talking about the problem of creating a System 2 that integrates smoothly with a System 1. It seems that he suggests creating a neural network-based System 2 so that it is more easily integrable with System 1 neural networks (having both systems be neural network-based allows for joint training with gradient descent). This is one possible approach but there are other proposals for integrating non-neural network based Systems 2 with neural network-based Systems 1 (see for example Luc De Raedt's DeepProbLog).

[@kkjc]

@@rodrigobraz2 Thank you very much for your answer! What I got before your reply is that OUPM is a Deductive learning approach while the System 2 is Inductive (indeed, Professor Bengio does not want to come back to Symbolic AI). The possible mistake I think you could have made is when you talk about collaboration between system 1 and system 2. Indeed, System 2 is an extension of System 1, and when it will be created, System 1 as we know it now will disappear. We will therefore only talk about the System 2 as AGI. Anyway, also thank you for informing me about the DeepProbLog approach, the book of Russell and Norvig: Artificial Intelligence, a modern approach (that I cited) recommends these initiatives in section 27.1, page 1062, just before the last paragraph. And as I can read on the DeepProbLog paper (arxiv.org/abs/1805.10872), it is the first initiative of this kind. Thank you again!

[@rodrigobraz2]

@@kkjc You're welcome! Just a few more points: I do not think that the intention is to get rid of System 1 once we have System 2. This nomenclature came from the book "Thinking, Fast and Slow" by Daniel Kahneman. From its Wikipedia page: "The main thesis is that of a dichotomy between two modes of thought: "System 1" is fast, instinctive and emotional; "System 2" is slower, more deliberative, and more logical." So, the idea is that intelligent agents keep both, just like we humans do. For example, an AI system may have a low-level, perceptual level System 1 that instantly classifies an image while also having a System 2 that reasons logically about it (possibly even realizing incorrect perceptions as it sometimes happens in optical illusions).

[@torstenhuenger4827]

Dual-system theories are popular in psychology but highly disputed. Kahneman may have received recognition for research related to System 1 (heuristics and habitual responses). What is disputed, however, is System 2. Is a System 2 necessary to explain human "higher" cognition. It is probably a bad idea to base your AI ideas on this flawed idea from psychology.

[@jleelee6026]

ML research by brain research ?

[@anssim928]

As it has been since the first ML systems, and as it should be until we have implemented in ML systems all the principles we have discovered in brains