Modern Artificial Intelligence 1980s-2021 and Beyond (Prof. Jürgen Schmidhuber)

[@alfcnz]

Welcome to KZbin, Jürgen!

[@bigpickles]

First saw Jurgen on YT 10 years ago

[@TheAIEpiphany]

19:00 Juergen: "I won't mention his name" also Juergen: "Dr Hinton". You got to love this man! He makes things spicy! Jokes aside, amazing lecture, here are my thoughts and video timestamps: (note numbers in the brackets are references to my comments + hints) 00:00 Intro - a look into the past, from Big Bang all the way to singularity (omega) in the 2040s 08:25 The beginnings of modern AI (self-driving, deep learning, meta-learning, etc.) 11:05 RNNs 19:35 DanNet CNN (2011) [1 ~ AlexNet] 22:40 Highway Nets (2015) [2 ~ ResNet] 24:20 Modern backdrop invented by Seppo Linnainmaa [3] 25:25 LSTMs and its industry use-cases 29:30 Compressed Network Search [4 ~ DQN] 35:50 Artificial Curiosity (1990s) [5 ~ GANs] 38:10 story about his company nnaisense 42:25 Outro - a look into the future, AIs take over the universe haha ------------------------------------------ Some thoughts: 1) Juergen mentions that DanNet has already achieved superhuman results before AlexNet came along, and it has shown that the GPU+big CNN+big data paradigm is the way forward. Honestly, DanNet is definitely a new one for me! AlexNet definitely took all of the public attention (no AI puns intended). That's very sad. I'd love to hear other people's opinions here. I wonder why didn't they just tackle the ImageNet competition since that was the most famous benchmark back then. Would a trivial tweaking of DanNet achieve the same results as AlexNet? 2) Juergen mentions that ResNet is a special case of Highway Net that came only half a year later after Highway Net paper was published - here is an excerpt from the 2015 ResNet paper: "Concurrent with our work, “highway networks” [42, 43] present shortcut connections with gating functions [15]. These gates are data-dependent and have parameters, in contrast to our identity shortcuts that are parameter-free. When a gated shortcut is “closed” (approaching zero), the layers in highway networks represent non-residual functions. On the contrary, our formulation always learns residual functions; our identity shortcuts are never closed, and all information is always passed through, with additional residual functions to be learned. In addition, high way networks have not demonstrated accuracy gains with extremely increased depth (e.g., over 100 layers)." Indeed it is the case that ResNets came half a year later! - very interesting. Considering the considerable piece of text the authors of ResNet used to distance themselves from HIghway networks it seems very plausible to me that they've been heavily inspired by this work. 3) He is, aside from stating the facts, also hinting (no pun intended) that Hinton was not the inventor of backpropagation - a common misconception. 4) Compressed network search - he is hinting that this is a precursor to DeepMind's famous DQN paper. He later says that some of his PhD students were the first employees at DeepMind further stressing the influence he exercised through DeepMind and OpenAI. 5) He claims that the GANs paper by Goodfellow et al. is a plagiarism. Check out his public dispute with Goodfellow here: kzbin.info/www/bejne/fni8iniLiNJgZrM :)) ------------------------------------------ Further thoughts: Brilliant start of the lecture! What I don't like about this video is that he basically depicts the history of AI as the history of his lab, which while (very) influential, is definitely not the whole story (to do him justice he does mention non-USA achievements haha, there is a big rivalry going on here that much is certain). e.g. He didn't mention transformers which are currently powering most of the apps that were previously powered by LSTMs. But considering the injustice done to him, I guess this is his way of going about it. Thanks for making this one Juergen! You definitely deserve the Turing award as much as Yann, Yoshua, and Hinton, IMHO.

[@krakatit7730]

Jurgen has a very accurate video lecture of AI history on this same channel where he mentions transformers.

[@michaelb.7071]

One of the greatest minds of our time sharing his thoughts for free to everyone. What a time to be alive! :)))

[@LoretoParisiLabs]

Such an amazing speech! And we shall all remark that the most successful milestones in Artificial Intelligence came out from Research taking place in old Europe, that afterall is not that old. Thanks to bring this to our mind, in some way it's mindblowing to think about it today, after 30 years of existing research.

[@HMexperience]

KZbin is so unfair. This video should be among the most popular out there but no. Thank you for this wonderful thought provoking lecture.😊

[@equitytruthandjustice8309]

Welcome to KZbin Juergen. Happy to be the 791 subscriber.

[@nazarenocesarmoreirareis3323]

Amazing! Outstanding! Thank you very much for sharing! …

[@virtuoskommunizieren]

His speeches are so fascinating. Watching them over and over again

[@buh357]

Thank you. World needs you knowledge. Please share more. ❤

[@mindaza0]

fantastic talk, interesting times coming within our lifespan.

[@guitarlicious]

It almost feels illegal that I can watch this for free. Thank you so much for posting this for everyone

[@luisluiscunha]

True

[@WarriorStatue]

That was a great overview! Also unintentional ASMR

[@eriklintsev]

Amazing! Thanks for the exceptional content.

[@AmericanBrain]

MORE MORE MORE - this was so good !!! Juergen - more - please more !

[@aadityaura3139]

Excellent talk.

[@hermes_logios]

Ha! "I won't mention this researcher by name ... but he's Dr. Hinton."

[@Gladiator7470]

Excellent 👍

[@Anti-DarkMatters4855]

I Loves U Hyung Juergen Schmidhuber ♥♥🧠🧠🧠

[@MaghrebProductions]

Thank you for the enlightenment!

[@Daniel-Six]

Crazy cool analysis of history.

[@LeandroAlexandreSilva]

Amazing speech!

[@BK-md2qw]

Thank you sir

[@dalton4035]

"British immigrant" is a funny way of saying Sir Tim Berners-Lee 😂

[@josecarloscastro1290]

Too good to be humble

[@deeplearningpartnership]

Awesome.

[@denispokrovsky2053]

I like how he said: "... mammals were invented"

[@cordewit5223]

Go Jürgen!

[@jonathansum9084]

I hope you will teach more in deep, such as math and the code(or code in Jutyer).

[@KRYPTOS_K5]

Excellent! Excellent! BRASIL

[@krish2nasa]

Enchanting talk! Hoping to see soon the replacement of the human politicians on this planet with no-nonsense/ethical AGI/ASI beings distributed across the world. Thank you very much.

[@rdgorbunov]

In 13 years people will be incorporated into the technological singularity in the order of their comments under this video.

[@mu-sum]

I wonder if you ever had a chance to discuss hierarchical RNN with David A. Huffman, before he passed? Because it seems that what you're describing at kzbin.info/www/bejne/pnjJpYh5idaUfck is a temporal Huffman Tree.

[@KRYPTOS_K5]

Juergen. Why there is no clue of any Von Newman probes in the Cosmos already now?

[@tg-gx8jc]

“Nur die Lumpe(n) sind bescheiden, Brave freuen sich der Tat”! Goethe “Only rascals are modest, The honest ones rejoice in deed”.

[@sarthakparikh5988]

Earthlings!…

[@angloland4539]

☺️❤️🍓

[@user-bz3jn2vw4v]

😁👍

[@jarikosonen4079]

Maybe if the "human multiplanetary species" is coming at 2030, when there will be mars landings. It probably involve AI as well.

[@KRYPTOS_K5]

Arent you afraid that differential equations come to substitute the humanity?

[@sfbillable]

Divide by 4. Hence 420

[@doyourealise]

hey jurgen :)

[@verystablegenius4720]

What an unnecessarily narcissistic talk ... Care to talk about anything other than his discoveries, his patterns, his theses, etc ...? The miracle years are NOT self-proclaimed, let OTHER PEOPLE claim it. Pathetic.

[@KlavdiLu]

According to C.G Jung probably you should ask yourself what you‘ve just commented ;)

[@chasemedsker]

Is it narcissistic to state facts? He’s dedicated his entire life to this. Has he not earned the right to be bold?

[@verystablegenius4720]

@@chasemedsker Yes, it is narcissistic to overly exaggerate one's importance and contributions. What you call "facts" are highly subjective. Last I checked, Schmidhuber did not win a Turing Award for this work.

[@2silkworm]

@@verystablegenius4720wow! Don't you think that you put the effect before the cause? It's a really flawed way logic.

[@verystablegenius4720]

you are right. I should rather start backwards and think that Schmidhuber deserved a Turing Award but they didn't give him one because there is a big conspiracy out to get him. There, now it is not logically flawed. Dumbo.@@2silkworm

[@KnowL-oo5po]

A.G.I Will be man's last invention