AI DEBATE : Yoshua Bengio | Gary Marcus

[@MontrealAI]

[ A G I D E B A T E ] 1 5 , 0 0 0 + A T T E N D E E S ! [ R S V P ] : aidebate3.eventbrite.ca It All Comes Down To AGI DEBATE [ P R O G R A M ] It takes a village . . . Four themes are at the crux of the AGI Debate, with four panels of the world's most distinguished researchers and experts on AGI : Panel 1: Ethics and morality Panel 2: Beyond System I Panel 3: Cognition and Neuroscience Panel 4: AI: Has it done more good or harm? Speakers: Erik Brynjolfsson, Yejin Choi, Noam Chomsky, Jeff Clune, David Ferrucci, Artur d'Avila Garcez, Michelle Rempel Garner, Dileep George, Sara Hooker, Anja Kaspersen, Konrad Kording, Kai-Fu Lee, Francesca Rossi, Jürgen Schmidhuber, Meredith Whitaker. Moderator and co-organizer (with Vincent Boucher): Gary Marcus. Professor Gary Marcus is a Scientist, Best-Selling Author, and Entrepreneur. He is Founder and CEO of Robust.AI. His newest book, Rebooting AI: Building Machines We Can Trust, aims to shake up the field of artificial intelligence and has been praised by Noam Chomsky, Steven Pinker and Garry Kasparov. Vincent Boucher is President of Montreal.AI and Quebec.AI. Join us as we engage in this historic and high-minded debate! [ W E B S I T E ] : agidebate.com/ #AIDebate #AGIDebate #MontrealAI #QuebecAI #QuebecArtificialIntelligence

[@jonatan01i]

My acoustic and language model has to generalize well in order to understand Vincent's sentences.

[@VincentBoucher]

Working on it! ;)

[@jonatan01i]

@@VincentBoucher It must be hard as one can hear that your french is in the way :D I like your cover image of your youtube channel, btw. Did you make that?

[@jonatan01i]

@@VincentBoucher You have an excuse for watching more movies/series in English :D

[@VincentBoucher]

@@jonatan01i movies/series?!? - what is that? -:D

[@jonatan01i]

@@VincentBoucher relaxation. Don't you know such things? :( :D

[@shashanks.k855]

Was a great idea to host this debate, thanks uploading it.

[@AR-iu7tf]

For those of us who have not watched this video yet and are skimming through comments - here is a verbatim exchange at the very end (2.00) of a 2+ hour debate. Yoshua Bengio says in a rare display of frustration (which his characteristic civil and gentle demeanor is unable to hide) to Gary Marcus - “just remove from your brain the 1989 notion a multilayer perceptron with feedforward learning - that is not deep learning”. Yoshua feeling compelled/driven to make such a statement at the very end of a 2 hour debate is perhaps the best synopsis of the substantive nature of this debate. Ironic that Gary starts the debate saying both of them have more agreements than differences and are seeking the truth. In reality if we watch the whole debate one can’t help feeling Gary lacks a clear understanding of what current deep learning is(from the perspective of what it can do; no one really knows why it works so well however) and its true limitations, other than a very fuzzy notion of a future deep learning symbolic reasoning hybrid. This sadly cripples this two hour debate from being the substantive debate it could have been - though Prof. Bengio does offer some clear insights into what is currently lacking in deep learning and what could be a path forward. If we want to really know what is lacking in deep learning and what could be one path forward this recent video by Yoshua Bengio may be much more worthwhile and truly substantive than this debate kzbin.info/www/bejne/fKW4k4B9aNeffNU

[@bossgd100]

I notice this about Gary Marcus also. Since this moment i stopped watching him

[@ahmadchamseddine6891]

@@bossgd100 Yea me too

[@tukity]

i think the frustration stems from the lack of actual engagement and debate over ideas rather than constantly defining deep learning as NN from 89' and refusal to put a scope to what symbols are (because they could be anything!).

[@stourleykracklite7663]

19:10 Marcus shows a slide of an email Bengio sent to student to illustrate how what Bengio says is misleading. I would hope we all can agree the publicizing of others' private emails is out of bounds.

[@osdai]

2022: GTP-3 is released I tested both challenges proposed by Gary Marcus and had failed using GTP-2, according him it's proves his prediction about language and deep learning. GTP- 3 give the correct answer for both challenges , new victory for deep learning

[@HectorZenil]

How difficult was it for Yoshua Bengio to say that his DL version 1 was not up to the task when Marcus was making his criticisms and Bengio was defending it against all odds as the ultimate solution for Artificial General Intelligence! Indeed, Bengio did a poor job defending his position. The argument that Deep Learning (DL) is an ever-moving target and therefore is bullet-proof against any criticism is cheap. The "new DL" or DL version 2, became then radically different by incorporating what Bengio today embraces but before mocked. Yoshua Bengio is surprised that manipulating trees is symbolic computation, of course, it is! It is not a neural network learning and implementing the function of a tree. He wants to have all both ways. I think Bengio was disrespectful and condescending with his debate opponent and guest dismissing him by calling him out-of-date on what DL really is while at the same time DL version 1 was evolving towards and adopting what Marcus (and others) have been working on and suggesting for years (myself included). Bengio's old argument that Gary Marcus' brain is a neural network and therefore everything is covered by a canonical deep neural network as they were when Bengio and others strongly defended their ground, is also a fallacy. Marcus could have easily replied that all Bengio's neural networks actually run on symbolic computers!! If they were playing at that game. Bengio should be awarded the fake rigour, gaslighting, mind gymnastics and reality-morphing-defence award, for his performance. Addendum: I am also surprised by the organisation of the debate with so much money pouring into these people, conferences and this institute's pockets. Version-1-DL scientists dismissing everyone else's work outside DL for years was very profitable but they are now very open to new ideas now that they cannot deliver what Version-1-DL promised.

[@TTgarland]

This comment is spot on. Marcus won both on the intellectual and personal level.

[@freakinccdevilleiv380]

Great points. Marcus may not have huge programming skills but he is way above all these researchers conceptually, they actualy bore him.

[@hzenilc]

@@freakinccdevilleiv380 unfortunately, prizes only recognise technical ability and fake rigour and that’s why Bengio behaves as a semi god and everybody bows to him no matter how silly something he says may be or how disrespectful he may act, as one can see from other comments in this thread.

[@freakinccdevilleiv380]

@@hzenilc I hadn't seen Bengio before but he was a bit of a defensive douche here honestly.

[@hzenilc]

@Richard Gleaves or scientists impersonating philosophers

[@omarnomad]

I think about a good hybrid system, something like GPT-3 plus being able to run Python code.

[@CristianGarcia]

It feels like Gary has good points but no clear path towards actual implementation because what he calls "hybrid" now is just using deep learning to e.g. parse an image/video/text and using a classical algorithm on top to e.g. plan. These systems are abundant (chatbots/assistants, self driving cars, Roombas, ect) but as Yoshua states they won't take us to the next level. The recent MuZero algorithm from Deep Mind is very interesting in that it "learns to plan", I think in spirit this is closer to a level 2 system than what Gary is hinting.

[@HectorZenil]

Bengio had (and I would argue, still does not have) any clear path either and until recently looked lost before proponents of hybrid computation.

[@stourleykracklite7663]

Agreed. Marcus has made himself a noodge for symbolic manipulation in the AI world, but he wants these debates to be definition-free... 1:38:25

[@schumachersbatman5094]

I'm no AI researcher, but I agree with Gary's (and Bengio's) instinct to put pressure on deep learning not to be satisfied with things like recognition. The field should focus on reasoning with rich conceptual representations. But I don't think Gary's alternative is convincing. It sounds like hand-coded solutions, written to specific tasks. That structure coming directly from the genome apparently. And these things for the "real brain" (the neural nets) to somehow learn to use. I believe it might be useful if the 'symbolic' types figured out how to do many difficult tasks humans do, but ML doesn't do (in robotics, cognition etc). If they did, then the ML folks might be inspired by the solution to think about what kind of models could learn and optimise such behaviour. But then I don't think AlphaZero/MuZero took too much inspiration from DeepBlue.

[@강동찬-h4s]

Thanks a lot for uploading

[@444haluk]

"There is nothing more than prejudice holding us back from embracing more innateness" BURNT!

[@DiogoNeves]

One thing I don’t understand in the nativist argument is that the hardcoded priors were also learnt, just by a slower learning algorithm. Even if we discovered that 99% of the brain’s inner workings are native, it wouldn’t necessarily disprove that they can be learnt. Nature has to balance survivability and adaptability

[@lassehurttila5037]

What I'd like to know is what separates system 1 and 2 on the physical level. To me it seems like both systems work in the same way fundamentally, while system 2 has the added functionality of attention due to it being on the conscious level of the mind.

[@nfiu]

These are metaphors to help understand different ways the system can solve problems.

[@tukity]

"operations over variables" generalized to the extreme could mean anything, which doesn't really add to totality of our knowledge. why implore someone to do that? Then 1st question from the audience, in response to how he would define symobols, "you can define a symbol in such a way that it encompasses everything or nothing. And I don't think that's where the debate should be". there's a lot of handwaving going on.

[@YaroslavVolovich]

This debate used to be on Lex Clips channel with lots of interesting comments, but it now disappeared for some reason....

[@VincentBoucher]

Here is the original live video commented as the #AIDebate was unfolding on Monday, 23 December 2019, at Mila : facebook.com/MontrealAI/videos/498403850881660/

[@shmameron]

Suggestion for future videos: increase the size of the slides on the screen. The text is very difficult to read at such a small size, and could have easily been larger without obscuring the speakers.

[@444haluk]

Gary Marcus is still my favorite sassy prof in the world.

[@aaabbbccc176]

Not only sassy, but also brave, particularly in the flood (more like a hype to me) of GPT.

[@niederrheiner8468]

Problem is that the relative new term "deep learning" is not sharply defined. Gary thinks just of classic neural nets only with more layers and for Yoshua "deep learning" already includes symbolic elements.

[@HectorZenil]

That does not make sense. It is defined well enough not to incorporate symbolic computation and contune calling it Deep Learning. According to your argument, Deep Learning (DL) cannot ever be criticised because by the time you do it would have already morphed to some unidentified place that no one understands while steering and incorporating the very criticisms made before. Makes no sense, that enlarged field is called ML or AI, not DL. A cheap way to defend DL and the claims made for poor Version-1-DL and current DL that is no longer DL. Proponents of neuro-symbolic computation have been there while Bengio, LeCun and others have been mocking and trashing them while they enjoy a lot of popularity (and resources) for their own research.

[@DistortedV12]

Can someone post the time stamps for opening statements, discussion, etc.? It seems like a similar video with these was taken down.

[@MontrealAI]

6h30m00s PM EST : Opening Address | Vincent Boucher - 3 min. 6h33m00s PM EST : Opening statement | Gary Marcus - 20 min. 6h53m00s PM EST : Opening statement | Yoshua Bengio - 20 min. 7h13m00s PM EST : Response | Gary Marcus - 7.5 min. 7h20m30s PM EST : Response | Yoshua Bengio - 7.5 min. 7h28m00s PM EST : Interview | Vincent Boucher : Yoshua Bengio & Gary Marcus - 15 min. 7h43m00s PM EST : Public Questions | Yoshua Bengio & Gary Marcus - 22,5 min. 8h05m30s PM EST : Int'l Audience Questions | Yoshua Bengio & Gary Marcus - 22,5 min. 8h28m00s PM EST : Closing Remarks | Vincent Boucher - 2 min

[@karimtit4897]

Génial, merci beaucoup Vincent. On pourrait avoir accès aux slides aussi ?

[@MontrealAI]

Gary Marcus, slides : montrealartificialintelligence.com/aidebate/slidesmarcus.pdf Yoshua Bengio, slides : montrealartificialintelligence.com/aidebate/slidesbengio.pdf

[@pratik245]

Yeah great calculus talk from algebraic mind

[@yassinelamarti4157]

Great debate

[@pqlamz976431]

Peut-on avoir la traduction en français pour ce débat?

[@jingoringo]

"merde c'est du keynote" lmao

[@freakinccdevilleiv380]

French detected without AI 😅

[@youjunhu236]

skipped all what Gary Marcus said( which is boring), then this is an interesting debate