Hinton's Advice: #1 - Read the literature but don't read too much of it. #2 - For creative researchers, I think what you want to do is to read a little bit of the literature and notice something that you think everybody is doing wrong and contrarian in that sense, you look at it and it just doesn't feel right and then figure out how to do it right, and when people tell you that's no good....just keep at it. And I have a very good principle for helping people keep at it. Which is, either your intuitions are good or they're not. If your intuitions are good, you should follow them and you will eventually be successful. If your intuitions are not good, it doesn't matter what you do. #3 - Never stop programming! #4 - I think one should reach enough to start building intuitions and then trust your intuitions and go for it. And, don't be too worried if everybody else says is nonsense. #5 - If you think its a really good idea and other people tell you it's complete nonsense, then you know you're really on to something. #6 - one good piece of advice for new grad students, see if you can find an adviser who has beliefs similar to yours because if you work on stuff that your adviser feels deeply about you'll get a lot of good advice and time from your adviser.
@prathyusha53934 жыл бұрын
The 2nd one!
@Gabcikovo Жыл бұрын
#2 👏 #TomášMikolov
@Gabcikovo Жыл бұрын
29:59 this is where Hinton actually says that
@wk424010 ай бұрын
Good principals to follow. Regardless of what you are doing.
@ismailelezi6 жыл бұрын
Even Andrew looks super-exciting in the video, imagine the viewers. :)
@dbiswas6 жыл бұрын
I attended one of his lecture in Washington University,. He is extremely humble and a great speaker. He is the Einstein of our generation. I am really looking forward to his upcoming paper on Capsule theory. God ! it gives me a feeling like standing in queue for the latest iPhone. LOL !
@thesuryapolisetty2 жыл бұрын
00:30 How Hinton’s fascination with the brain led him to explore AI 3:33 The story behind his seminal 1986 paper on backpropagation that he wrote with David Rumelhart 8:16 The invention Hinton is still most excited about 12:55 Hinton’s work on ReLU activations 25:35 How Hinton’s understanding of AI has changed over the decades 15:27 Hinton’s thoughts on the relationship between the brain and backpropagation 19:09 Hinton’s work on dealing with multiple time scales 20:43 Hinton’s ongoing research on capsules 29:40 Hinton’s advice for someone who wants to break into AI 37:15 Hinton’s thoughts on the paradigm shift in AI
@srikararepalli Жыл бұрын
Thank you!
@saraths61846 жыл бұрын
Two legends talking on a domain they help build. No surprise it turned out so informative. Thank u for making this video.
@georgemaratos11225 жыл бұрын
thank you so much for doing this interview
@woolfel6 жыл бұрын
Thanks for this excellent interview. I'm totally geeking out at this interview!
@senthilvs17236 жыл бұрын
Thanks, Andrew Ng for the interviews.
@unoqualsiasi73416 жыл бұрын
Thanks for the amazing interview and for the sharing!
@Trackman20076 жыл бұрын
"Finally Andrew Ng bought a decent mic!" - nah.. have to edit myself. At around 27:00 and on he returns to toilet/pillow-filtered sound. That's just Andrew Ngs trademark!
@Joe-yr1em5 жыл бұрын
Thank you for this wonderful interview.
@doryds6 жыл бұрын
Utterly fascinating. Professor Hinton is inspiring.
@alexandeap4 жыл бұрын
Dear Andrew, first of all I wanted to thank you and the entire coursera team for the great academic contribution they give us. Secondly, I wanted to ask you to put the subtitles where possible so that we can enjoy 100% Spanish and South American speaking. finally ask you to include the course of neural networks of Jeoffrey Hinton since when you search through the search engine of coursera it does not find it. I will thank you for correcting this error because when I tried to search for the courses offered by the University of Toronto, I did not have a positive success either. Thanks again to you and the coursera team for the great innovative academic contribution and scientific and technological research.
@reoext6 жыл бұрын
This is very interesting in terms of Prof. Hinton's historical perspective on his interest and development of many algorithms. Also, Prof Ng contribution to making everyone dreams become a reality from learning ML and applying in their prospective jobs. Thanks to both!!
@morebaie34124 жыл бұрын
This interview is highly insightful and helpful for deep learners, recommended watching!
@yuwuxiong11656 жыл бұрын
Excellent interview...and very good advises from Hinton!
@jindagi_ka_safar5 жыл бұрын
Thanks for introducing us to the heroes of DL like Geoffrey
@jung89356 жыл бұрын
Great interview, but it's so deep. I wish I could understand more of it...
@aq1q6 жыл бұрын
Geoggrey Hinton and Andrew NG, ML Titans!
@falak886 жыл бұрын
Fascinating !
@wajdanali13546 жыл бұрын
honoured to listen to this discussion , subhanallah
@jsfnnyc7 ай бұрын
Best research advice ever!! "Read the literature, but not too much of it."
@CHECK3R56 жыл бұрын
38:00 onwards, brilliant philosophy of AI from this legend.
@harleyswick54496 жыл бұрын
This was great. Love hearing his more far out ideas
@xbronn6 жыл бұрын
Huge thanks to Andrew and Geoffrey for this interview!
@NattapongPUN6 жыл бұрын
Legend!
@briancase95279 ай бұрын
I so agree with Hinton: have an idea and go for it. I took this approach with something other than AI, but it also worked. What do I mean? I mean, even though my idea wasn't revolutionary and totally worthwhile, I LEARNED A LOT by just going for it and programming the heck out of it. The practical experience I gained served me well--very well--in my first jobs. Remember: your purpose is to learn, and you can do that following your intuition--which is fun--or following someone else's--which is less fun.
@mdougf5 жыл бұрын
Thanks for this interview, Andrew! Hey, fellow learners, if anyone is interested in joining my weekly Machine Learning Research Paper reading and discussion group, let me know!
@brishtiteveja5 жыл бұрын
I love Andrew so much. :)
@mehedihasanbijoy66094 жыл бұрын
listening to Geoff is something really fancy and super exciting.
@MilanAndric Жыл бұрын
Fav quote, little after minute 30... "Either your intuition is good or its not. If its good you should follow them and eventually you will be successful. If they are no good it doesn't matter what you do. There's no point in not trusting your intuitions."
@bobcrunch6 жыл бұрын
Three years ago I took Ng's Machine Learning Coursera class and it really got me hooked on the subject. Then in the spring of 2017 I took Hinton's Neural Networks for Machine Learning Coursera class and it was a big step in my understanding the subject. Hinton's class goes 15 weeks and is a little intimidating in both depth and breadth. I have the math background so that really helped. I guess the bottom line is that if you're going to study the subject, study math through partial differential equations.
@bobcrunch6 жыл бұрын
Most of the backpropagation is just add-subtract-multiply, but the tricky step is to calculate the updated weights using gradient descent.
@azr_sd4 жыл бұрын
Andrew looking at Hinton Like how we all see Andrew when learning from him.. Huge respect for everyone who contributed to AI. :)
@dixingxu6 жыл бұрын
NEVER STOP PROGRAMMING!! dope
@thegamechanger71572 жыл бұрын
Yes, I learned from his tutorial in Coursera in data science and technology
@ProfessionalTycoons6 жыл бұрын
Amazing
@Global_Pivot4 жыл бұрын
"When you have something you think is a good idea and other people think it's a complete nonsense, then you know you are onto something"
@KulvinderSingh-pm7cr5 жыл бұрын
Enlightened !!
@LuisGuillermoRestrepoRivas6 жыл бұрын
Interesting and informative. Thanks. But, on the last part: I believe that the symbolic approach of AI should not be totally abandoned by the researchers. One reason, by no means the only one, is that it gives AI a less of a "black box" approach than neural networks.
@tommygunhunter4 жыл бұрын
symbolic.... total baloney! Symbols and indeed mathematics in general are emergent entities of the brain. AI if it is to replicate the workings of the brain has to dig deeper, be more subatomic, less molecular as therein lies the route to a multitasking, general intelligence.
@KangZhang6 жыл бұрын
Never stop programming !
@ehfo6 жыл бұрын
I wish someone post the link to all the paper mentioned in the interview
@sajanrai90476 жыл бұрын
Godfather 🙌🙌🙌
@markgao116 жыл бұрын
Very inspiring interview. AI is changing the world fundamentally anyway.
@VineetBhatawadekar5 жыл бұрын
Legend.
@MattRiddell2 жыл бұрын
Wow - the capsule concept is pretty close to the thousand brains idea!
@zeus10826 жыл бұрын
The legend looks so innocent
@Al.Mo.6 жыл бұрын
BTW, that auto-generated caption is so creepy accurate
@ziyiguo92966 жыл бұрын
"The most beautiful one is the work I do between Terrence Sejnowski on Boltzmann Machines"
@koendejonghe15556 жыл бұрын
At 32:08 : Never stop programming!
@brandomiranda67036 жыл бұрын
that advice wasn't clear to me...is the advice to never stop programming cuz if a "bad student" doesn't make the idea work then if you don't lose your edge on programming then you can make work yourself?
@satvik0076 жыл бұрын
You only understand the tiny details when you can write a program to implement the whole thing yourself.
@brishtiteveja5 жыл бұрын
And stop copy pasting and make mistakes and fix the bugs I guess !! :)
@hiauoe6 жыл бұрын
Anybody know if anything was published about his stack of auto-encoders backprop idea? Would love to hear more about it
@chadwick35936 жыл бұрын
Anton He's talking about layer-wise pretraining.
@godbennett6 жыл бұрын
Based on Hinton's descriptions of capsules, is it possible he has overlooked manifolds? It is pertinently the behaviour of manifolds, where solutions or sub manifolds (i.e. latent vectors on the states of particular concepts in the input space - some latent z is entailed by some factor distribution : {position, scale…}) are observed to lie in local patches of the global manifold, that engender that particular factors may be learned; for example pixels in the neighbourhood of some other pixels may signify transformations on that same pixel, while other neighbourhoods may be disentangled from the sampled latent vectors of the aforesaid pixel altogether, (i.e. other pixel data and their transformation data are separable from the events of the pixel discussed above)
@AhmedKachkach6 жыл бұрын
Hinton surely did not overlook manifolds. The "routing by agreement" bit is where the main difference is; The way traditional CNNs learn means that they would not generalise as well as capsule nets; a lot of the information is lost / duplicated. Capsule nets (with this focus on "concepts") would be biased to learn more re-usable concepts, with different properties.
@a.gholiha68846 жыл бұрын
Nice!
@GreatUnwashedMass6 жыл бұрын
This guys' intuitions puts everyone else to shame.
@w.morillo6 жыл бұрын
GreatUnwashedMass 0
@corywiedenbeck15624 жыл бұрын
Nice and humble atheist
@runvnc2083 жыл бұрын
Its also his clarity of writing I think. His papers are easier to understand for me than a lot of others.
@chaidaro6 жыл бұрын
God has spoken.
@brandomiranda67036 жыл бұрын
Funny but important (?): “You either have good intuitions or don't. If you have good intuitions you’ll eventually be successful but if you have bad intuition no matter what you do it will suck, so follow your intuitions”-paraphrase of Hinton's research advice. What do people think?
@brandomiranda67036 жыл бұрын
Don't give up (but in a smart way) seems to be better advice and I agree. You know where he borrowed it from?
@allenwang33316 жыл бұрын
I think his intuition comes from his experience in diverse disciplines. He mentioned that he tried physics, physiology, psychology and philosophy during his undergrad. Intuition isn't some magical ability you're born with. It builds as you get exposed to information - similar to how variation in training data is beneficial to a good machine learning model.
@daviddav28456 жыл бұрын
if u live long enoguh bad become good. trouble is we have short life span... so if you suspect u have bad intuiation , look deeper into your heart to find a better one which yiu can use to explain everything happen to you and around the world. Then u should arrive on some decent intuition.
@PaulHigginbothamSr9 ай бұрын
I think the difference between wake and sleep is during sleep it is in the testing phase and during wake it is the operative phase of learning.
@ashiningworld6 жыл бұрын
Top ten anime crossovers
@onamixt6 ай бұрын
I watched the video as a part of Deep Learning Specialization. Sadly, it's way way over my head to comprehend much of what was said in the video.
@mahdiamrollahi8456 Жыл бұрын
Great
@Gabcikovo Жыл бұрын
38:10 a thought is just a great big vector of neural activity
@Gabcikovo Жыл бұрын
38:19 people who thought that thoughts were symbolic expressions made a huge mistake.. what comes in is a string of words, and what comes out is a string of words, and because of that, strings of words are the obvious way to represent things, so they thought what must be in between was a string of words or something alike.. Hinton thinks there's nothing like a string of words in between, he thinks thinking of it as of some kind of language is as silly as the idea that understanding the layout of a spacial scene must be in pixels :))
@Gabcikovo Жыл бұрын
18:00 2007 ignored Hinton and Bengio picked it up layer on
@vinodkumar-pv8qz6 жыл бұрын
youtube have only one like button :(
@Gabcikovo Жыл бұрын
35:08 our relationship to computers has changed.. instead of programming them, we show them, and they figure it out
@Gabcikovo Жыл бұрын
36:04 :D
@sivaprasadml65825 жыл бұрын
@ 8:34 what was Geoffrey mentioning ? Which algorithm ?
@tommygunhunter4 жыл бұрын
boltzmann machines
@vovos006 жыл бұрын
10:49
@Level63 жыл бұрын
힌튼의 조언 : # 1-문헌을 읽되 너무 많이 읽지 마십시오. # 2-창의적인 연구자들에게, 사람들이 좋지 않다고 말할 때 .. 그냥 계속하십시오. 그리고 저는 사람들이 그것을 지키도록 돕는 아주 좋은 원칙을 가지고 있습니다. 즉, 직관이 좋거나 그렇지 않습니다. 당신의 직관이 좋다면 당신은 그것을 따라야 하며 결국 성공할 것입니다. 당신의 직감이 좋지 않다면 당신이 무엇을 하든 상관 없습니다. # 3-프로그래밍을 멈추지 마십시오! # 4-직관을 쌓기 시작할 만큼 충분히 도달 한 다음, 직감을 믿고 실행해야 한다고 생각합니다. 그리고 다른 사람들이 말도 안된다고 해도 너무 걱정하지 마세요. # 5-당신이 그것이 정말 좋은 아이디어라고 생각하고 다른 사람들이 그것이 완전히 넌센스라고 말한다면, 당신은 당신이 정말로 무언가를 하고 있다는 것을 압니다. # 6-새로운 대학원생을 위한 좋은 조언 한 가지, 당신과 비슷한 신념을 가진 조언자를 찾을 수 있는지 확인하세요. 당신이 조언자가 당신에 대해 깊이 느끼는 일을 하면 좋은 조언과 시간을 많이 얻을 수 있기 때문입니다.
@haoshidi3 жыл бұрын
Concretely
@surkewrasoul47118 ай бұрын
Hey And , Do you still accept donations by any chance, I am hoping for 720p videos from now on.
@godbennett6 жыл бұрын
Hinton: "Thoughts are great big vectors, and big vectors have causal powers, they cause other big vectors" Me: "Thought Curvature" paper : www.academia.edu/25733790/Causal_Neural_Paradox_Thought_Curvature_Aptly_the_transient_naive_hypothesis
@evankim40964 жыл бұрын
WOW I can't believe Hinton et al came up with an algorithm for STDP decades before neuroscientists came up with the concept....He is talking about this algorithm that came in the late 1980s, while STDP arrived on the scene in the early-mid 2000s
@asutoshmittapalli3 жыл бұрын
All rise for the Godfather
@AlinNemet6 жыл бұрын
no question about AI and singularity!? sure he would've had a super funny answer :))
@mukuste6 жыл бұрын
The singularity is a pure scifi idea that actual AI researchers have no time for.
@AlinNemet6 жыл бұрын
Oh yeah!? then how come elon musk, stef. hawking, sam harris and even f**g bill gates, seriously talk about it as an existential threat!? surely they know something the rest of us don't :))
@mukuste6 жыл бұрын
None of them are AI researchers. They're laypeople on this issue.
@AlinNemet6 жыл бұрын
mukuste yeah it's ridiculous...though AGI is a very interesting topic, as Neil D Tyson noted, we are far from being on the path of something like that happening...so till then AI is just another tool to help us live better
@AlinNemet6 жыл бұрын
indeed
@Gabcikovo Жыл бұрын
0:19 Godfather 😎
@Gabcikovo Жыл бұрын
25:33
@billykotsos46424 жыл бұрын
Andrew "I see" Ng
@johningham18804 жыл бұрын
I wish my neurones could remember what they were doing when they pop out of a recursive call...
@salvatortermination46816 жыл бұрын
alien language to me right now, still it's great haha
@dileepjayamal99685 жыл бұрын
Still there are 17 dislikes, don't know why.. may be outliers...
@almostbutnotentirelyunreas1666 жыл бұрын
YESSS! A technical tour-de-force! The Godfather! Gotta love the consistent elephant in the room though. AI inevitability: Human intelligence becomes obsolete for any purpose beyond the banal. Luckily, the onset of autonomous AGI (consciousness optional) solves this permanently. ;-[ Intelligence is THE differentiator on earth; let's relentlessly pursue / design Humanity's successor as Apex Predator. GAN indeed. GONE, more likely. Just keep kicking the can down the road, you clever hand-wringers. Learning leads to Knowledge, Knowledge is Power, Power corrupts, absolute Power corrupts absolutely. A clearly sub-optimal general (Human) outcome, driven by an optimising, self-adjusting, closed-loop feed-back system. Brilliantly myopic.
@aniketphatak9513 жыл бұрын
damn that's deep lol
@almostbutnotentirelyunreas1663 жыл бұрын
@@aniketphatak951 Shallow: OK, you got it.
@wk424010 ай бұрын
Seriously doubt Geoffrey Hinton considers himself a hero - more like Dr. Frankenstein now. He's doing his part to spread the word on the dangers of reliance on AI.
@jamesking24396 жыл бұрын
The reflections of the screen in his glasses look like googly eyes.
@tommygunhunter4 жыл бұрын
Google eyes! Connected to Google Brain :)
@sujoyparikh53622 жыл бұрын
Absolutely love this, but Andrew, please stop saying "I see" constantly