Hinton's Advice: #1 - Read the literature but don't read too much of it. #2 - For creative researchers, I think what you want to do is to read a little bit of the literature and notice something that you think everybody is doing wrong and contrarian in that sense, you look at it and it just doesn't feel right and then figure out how to do it right, and when people tell you that's no good....just keep at it. And I have a very good principle for helping people keep at it. Which is, either your intuitions are good or they're not. If your intuitions are good, you should follow them and you will eventually be successful. If your intuitions are not good, it doesn't matter what you do. #3 - Never stop programming! #4 - I think one should reach enough to start building intuitions and then trust your intuitions and go for it. And, don't be too worried if everybody else says is nonsense. #5 - If you think its a really good idea and other people tell you it's complete nonsense, then you know you're really on to something. #6 - one good piece of advice for new grad students, see if you can find an adviser who has beliefs similar to yours because if you work on stuff that your adviser feels deeply about you'll get a lot of good advice and time from your adviser.
@prathyusha53934 жыл бұрын
The 2nd one!
@Gabcikovo Жыл бұрын
#2 👏 #TomášMikolov
@Gabcikovo Жыл бұрын
29:59 this is where Hinton actually says that
@wk4240 Жыл бұрын
Good principals to follow. Regardless of what you are doing.
@dbiswas7 жыл бұрын
I attended one of his lecture in Washington University,. He is extremely humble and a great speaker. He is the Einstein of our generation. I am really looking forward to his upcoming paper on Capsule theory. God ! it gives me a feeling like standing in queue for the latest iPhone. LOL !
@thesuryapolisetty2 жыл бұрын
00:30 How Hinton’s fascination with the brain led him to explore AI 3:33 The story behind his seminal 1986 paper on backpropagation that he wrote with David Rumelhart 8:16 The invention Hinton is still most excited about 12:55 Hinton’s work on ReLU activations 25:35 How Hinton’s understanding of AI has changed over the decades 15:27 Hinton’s thoughts on the relationship between the brain and backpropagation 19:09 Hinton’s work on dealing with multiple time scales 20:43 Hinton’s ongoing research on capsules 29:40 Hinton’s advice for someone who wants to break into AI 37:15 Hinton’s thoughts on the paradigm shift in AI
@srikararepalli Жыл бұрын
Thank you!
@ismailelezi7 жыл бұрын
Even Andrew looks super-exciting in the video, imagine the viewers. :)
@saraths61847 жыл бұрын
Two legends talking on a domain they help build. No surprise it turned out so informative. Thank u for making this video.
@doryds7 жыл бұрын
Utterly fascinating. Professor Hinton is inspiring.
@alexandeap5 жыл бұрын
Dear Andrew, first of all I wanted to thank you and the entire coursera team for the great academic contribution they give us. Secondly, I wanted to ask you to put the subtitles where possible so that we can enjoy 100% Spanish and South American speaking. finally ask you to include the course of neural networks of Jeoffrey Hinton since when you search through the search engine of coursera it does not find it. I will thank you for correcting this error because when I tried to search for the courses offered by the University of Toronto, I did not have a positive success either. Thanks again to you and the coursera team for the great innovative academic contribution and scientific and technological research.
@reoext7 жыл бұрын
This is very interesting in terms of Prof. Hinton's historical perspective on his interest and development of many algorithms. Also, Prof Ng contribution to making everyone dreams become a reality from learning ML and applying in their prospective jobs. Thanks to both!!
@MilanAndric2 жыл бұрын
Fav quote, little after minute 30... "Either your intuition is good or its not. If its good you should follow them and eventually you will be successful. If they are no good it doesn't matter what you do. There's no point in not trusting your intuitions."
@aq1q7 жыл бұрын
Geoggrey Hinton and Andrew NG, ML Titans!
@xbronn7 жыл бұрын
Huge thanks to Andrew and Geoffrey for this interview!
@jsfnnyc11 ай бұрын
Best research advice ever!! "Read the literature, but not too much of it."
@mdougf5 жыл бұрын
Thanks for this interview, Andrew! Hey, fellow learners, if anyone is interested in joining my weekly Machine Learning Research Paper reading and discussion group, let me know!
@senthilvs17237 жыл бұрын
Thanks, Andrew Ng for the interviews.
@azr_sd4 жыл бұрын
Andrew looking at Hinton Like how we all see Andrew when learning from him.. Huge respect for everyone who contributed to AI. :)
@jung89357 жыл бұрын
Great interview, but it's so deep. I wish I could understand more of it...
@Trackman20077 жыл бұрын
"Finally Andrew Ng bought a decent mic!" - nah.. have to edit myself. At around 27:00 and on he returns to toilet/pillow-filtered sound. That's just Andrew Ngs trademark!
@wajdanali13546 жыл бұрын
honoured to listen to this discussion , subhanallah
@yuwuxiong11656 жыл бұрын
Excellent interview...and very good advises from Hinton!
@briancase9527 Жыл бұрын
I so agree with Hinton: have an idea and go for it. I took this approach with something other than AI, but it also worked. What do I mean? I mean, even though my idea wasn't revolutionary and totally worthwhile, I LEARNED A LOT by just going for it and programming the heck out of it. The practical experience I gained served me well--very well--in my first jobs. Remember: your purpose is to learn, and you can do that following your intuition--which is fun--or following someone else's--which is less fun.
@morebaie34125 жыл бұрын
This interview is highly insightful and helpful for deep learners, recommended watching!
@Global_Pivot4 жыл бұрын
"When you have something you think is a good idea and other people think it's a complete nonsense, then you know you are onto something"
@bobcrunch6 жыл бұрын
Three years ago I took Ng's Machine Learning Coursera class and it really got me hooked on the subject. Then in the spring of 2017 I took Hinton's Neural Networks for Machine Learning Coursera class and it was a big step in my understanding the subject. Hinton's class goes 15 weeks and is a little intimidating in both depth and breadth. I have the math background so that really helped. I guess the bottom line is that if you're going to study the subject, study math through partial differential equations.
@bobcrunch6 жыл бұрын
Most of the backpropagation is just add-subtract-multiply, but the tricky step is to calculate the updated weights using gradient descent.
@woolfel6 жыл бұрын
Thanks for this excellent interview. I'm totally geeking out at this interview!
@jindagi_ka_safar5 жыл бұрын
Thanks for introducing us to the heroes of DL like Geoffrey
@mehedihasanbijoy66094 жыл бұрын
listening to Geoff is something really fancy and super exciting.
@Joe-yr1em6 жыл бұрын
Thank you for this wonderful interview.
@harleyswick54497 жыл бұрын
This was great. Love hearing his more far out ideas
@CHECK3R57 жыл бұрын
38:00 onwards, brilliant philosophy of AI from this legend.
@LuisGuillermoRestrepoRivas7 жыл бұрын
Interesting and informative. Thanks. But, on the last part: I believe that the symbolic approach of AI should not be totally abandoned by the researchers. One reason, by no means the only one, is that it gives AI a less of a "black box" approach than neural networks.
@tommygunhunter5 жыл бұрын
symbolic.... total baloney! Symbols and indeed mathematics in general are emergent entities of the brain. AI if it is to replicate the workings of the brain has to dig deeper, be more subatomic, less molecular as therein lies the route to a multitasking, general intelligence.
@brishtiteveja6 жыл бұрын
I love Andrew so much. :)
@PaulHigginbothamSr Жыл бұрын
I think the difference between wake and sleep is during sleep it is in the testing phase and during wake it is the operative phase of learning.
@unoqualsiasi73416 жыл бұрын
Thanks for the amazing interview and for the sharing!
@dixingxu7 жыл бұрын
NEVER STOP PROGRAMMING!! dope
@georgemaratos11226 жыл бұрын
thank you so much for doing this interview
@falak887 жыл бұрын
Fascinating !
@onamixt10 ай бұрын
I watched the video as a part of Deep Learning Specialization. Sadly, it's way way over my head to comprehend much of what was said in the video.
@thegamechanger71572 жыл бұрын
Yes, I learned from his tutorial in Coursera in data science and technology
@markgao117 жыл бұрын
Very inspiring interview. AI is changing the world fundamentally anyway.
@Level63 жыл бұрын
힌튼의 조언 : # 1-문헌을 읽되 너무 많이 읽지 마십시오. # 2-창의적인 연구자들에게, 사람들이 좋지 않다고 말할 때 .. 그냥 계속하십시오. 그리고 저는 사람들이 그것을 지키도록 돕는 아주 좋은 원칙을 가지고 있습니다. 즉, 직관이 좋거나 그렇지 않습니다. 당신의 직관이 좋다면 당신은 그것을 따라야 하며 결국 성공할 것입니다. 당신의 직감이 좋지 않다면 당신이 무엇을 하든 상관 없습니다. # 3-프로그래밍을 멈추지 마십시오! # 4-직관을 쌓기 시작할 만큼 충분히 도달 한 다음, 직감을 믿고 실행해야 한다고 생각합니다. 그리고 다른 사람들이 말도 안된다고 해도 너무 걱정하지 마세요. # 5-당신이 그것이 정말 좋은 아이디어라고 생각하고 다른 사람들이 그것이 완전히 넌센스라고 말한다면, 당신은 당신이 정말로 무언가를 하고 있다는 것을 압니다. # 6-새로운 대학원생을 위한 좋은 조언 한 가지, 당신과 비슷한 신념을 가진 조언자를 찾을 수 있는지 확인하세요. 당신이 조언자가 당신에 대해 깊이 느끼는 일을 하면 좋은 조언과 시간을 많이 얻을 수 있기 때문입니다.
@koendejonghe15557 жыл бұрын
At 32:08 : Never stop programming!
@brandomiranda67037 жыл бұрын
that advice wasn't clear to me...is the advice to never stop programming cuz if a "bad student" doesn't make the idea work then if you don't lose your edge on programming then you can make work yourself?
@falcon202436 жыл бұрын
You only understand the tiny details when you can write a program to implement the whole thing yourself.
@brishtiteveja6 жыл бұрын
And stop copy pasting and make mistakes and fix the bugs I guess !! :)
@KangZhang6 жыл бұрын
Never stop programming !
@brandomiranda67037 жыл бұрын
Funny but important (?): “You either have good intuitions or don't. If you have good intuitions you’ll eventually be successful but if you have bad intuition no matter what you do it will suck, so follow your intuitions”-paraphrase of Hinton's research advice. What do people think?
@brandomiranda67037 жыл бұрын
Don't give up (but in a smart way) seems to be better advice and I agree. You know where he borrowed it from?
@allenwang33317 жыл бұрын
I think his intuition comes from his experience in diverse disciplines. He mentioned that he tried physics, physiology, psychology and philosophy during his undergrad. Intuition isn't some magical ability you're born with. It builds as you get exposed to information - similar to how variation in training data is beneficial to a good machine learning model.
@daviddav28456 жыл бұрын
if u live long enoguh bad become good. trouble is we have short life span... so if you suspect u have bad intuiation , look deeper into your heart to find a better one which yiu can use to explain everything happen to you and around the world. Then u should arrive on some decent intuition.
@GreatUnwashedMass7 жыл бұрын
This guys' intuitions puts everyone else to shame.
@w.morillo7 жыл бұрын
GreatUnwashedMass 0
@corywiedenbeck15625 жыл бұрын
Nice and humble atheist
@runvnc2083 жыл бұрын
Its also his clarity of writing I think. His papers are easier to understand for me than a lot of others.
@MattRiddell3 жыл бұрын
Wow - the capsule concept is pretty close to the thousand brains idea!
@Gabcikovo Жыл бұрын
38:10 a thought is just a great big vector of neural activity
@Gabcikovo Жыл бұрын
38:19 people who thought that thoughts were symbolic expressions made a huge mistake.. what comes in is a string of words, and what comes out is a string of words, and because of that, strings of words are the obvious way to represent things, so they thought what must be in between was a string of words or something alike.. Hinton thinks there's nothing like a string of words in between, he thinks thinking of it as of some kind of language is as silly as the idea that understanding the layout of a spacial scene must be in pixels :))
@sajanrai90476 жыл бұрын
Godfather 🙌🙌🙌
@zeus10826 жыл бұрын
The legend looks so innocent
@shairuno7 жыл бұрын
God has spoken.
@Gabcikovo Жыл бұрын
35:08 our relationship to computers has changed.. instead of programming them, we show them, and they figure it out
@Gabcikovo Жыл бұрын
36:04 :D
@ProfessionalTycoons7 жыл бұрын
Amazing
@ziyiguo92967 жыл бұрын
"The most beautiful one is the work I do between Terrence Sejnowski on Boltzmann Machines"
@Al.Mo.6 жыл бұрын
BTW, that auto-generated caption is so creepy accurate
@NattapongPUN7 жыл бұрын
Legend!
@Gabcikovo Жыл бұрын
18:00 2007 ignored Hinton and Bengio picked it up layer on
@ehfo6 жыл бұрын
I wish someone post the link to all the paper mentioned in the interview
@surkewrasoul4711 Жыл бұрын
Hey And , Do you still accept donations by any chance, I am hoping for 720p videos from now on.
@KulvinderSingh-pm7cr6 жыл бұрын
Enlightened !!
@godbennett7 жыл бұрын
Based on Hinton's descriptions of capsules, is it possible he has overlooked manifolds? It is pertinently the behaviour of manifolds, where solutions or sub manifolds (i.e. latent vectors on the states of particular concepts in the input space - some latent z is entailed by some factor distribution : {position, scale…}) are observed to lie in local patches of the global manifold, that engender that particular factors may be learned; for example pixels in the neighbourhood of some other pixels may signify transformations on that same pixel, while other neighbourhoods may be disentangled from the sampled latent vectors of the aforesaid pixel altogether, (i.e. other pixel data and their transformation data are separable from the events of the pixel discussed above)
@AhmedKachkach6 жыл бұрын
Hinton surely did not overlook manifolds. The "routing by agreement" bit is where the main difference is; The way traditional CNNs learn means that they would not generalise as well as capsule nets; a lot of the information is lost / duplicated. Capsule nets (with this focus on "concepts") would be biased to learn more re-usable concepts, with different properties.
@hiauoe7 жыл бұрын
Anybody know if anything was published about his stack of auto-encoders backprop idea? Would love to hear more about it
@chadwick35936 жыл бұрын
Anton He's talking about layer-wise pretraining.
@mahdiamrollahi84562 жыл бұрын
Great
@vinodkumar-pv8qz7 жыл бұрын
youtube have only one like button :(
@VineetBhatawadekar5 жыл бұрын
Legend.
@wk4240 Жыл бұрын
Seriously doubt Geoffrey Hinton considers himself a hero - more like Dr. Frankenstein now. He's doing his part to spread the word on the dangers of reliance on AI.
@asutoshmittapalli4 жыл бұрын
All rise for the Godfather
@Gabcikovo Жыл бұрын
0:19 Godfather 😎
@Gabcikovo Жыл бұрын
25:33
@godbennett7 жыл бұрын
Hinton: "Thoughts are great big vectors, and big vectors have causal powers, they cause other big vectors" Me: "Thought Curvature" paper : www.academia.edu/25733790/Causal_Neural_Paradox_Thought_Curvature_Aptly_the_transient_naive_hypothesis
@evankim40964 жыл бұрын
WOW I can't believe Hinton et al came up with an algorithm for STDP decades before neuroscientists came up with the concept....He is talking about this algorithm that came in the late 1980s, while STDP arrived on the scene in the early-mid 2000s
@sivaprasadml65825 жыл бұрын
@ 8:34 what was Geoffrey mentioning ? Which algorithm ?
@tommygunhunter5 жыл бұрын
boltzmann machines
@johningham18805 жыл бұрын
I wish my neurones could remember what they were doing when they pop out of a recursive call...
@a.gholiha68847 жыл бұрын
Nice!
@ashiningworld6 жыл бұрын
Top ten anime crossovers
@sujoyparikh53622 жыл бұрын
Absolutely love this, but Andrew, please stop saying "I see" constantly
@dileepjayamal99685 жыл бұрын
Still there are 17 dislikes, don't know why.. may be outliers...
@vovos007 жыл бұрын
10:49
@haoshidi4 жыл бұрын
Concretely
@AlinNemet7 жыл бұрын
no question about AI and singularity!? sure he would've had a super funny answer :))
@mukuste7 жыл бұрын
The singularity is a pure scifi idea that actual AI researchers have no time for.
@AlinNemet7 жыл бұрын
Oh yeah!? then how come elon musk, stef. hawking, sam harris and even f**g bill gates, seriously talk about it as an existential threat!? surely they know something the rest of us don't :))
@mukuste7 жыл бұрын
None of them are AI researchers. They're laypeople on this issue.
@AlinNemet7 жыл бұрын
mukuste yeah it's ridiculous...though AGI is a very interesting topic, as Neil D Tyson noted, we are far from being on the path of something like that happening...so till then AI is just another tool to help us live better
@AlinNemet7 жыл бұрын
indeed
@billykotsos46425 жыл бұрын
Andrew "I see" Ng
@salvatortermination46816 жыл бұрын
alien language to me right now, still it's great haha
@almostbutnotentirelyunreas1667 жыл бұрын
YESSS! A technical tour-de-force! The Godfather! Gotta love the consistent elephant in the room though. AI inevitability: Human intelligence becomes obsolete for any purpose beyond the banal. Luckily, the onset of autonomous AGI (consciousness optional) solves this permanently. ;-[ Intelligence is THE differentiator on earth; let's relentlessly pursue / design Humanity's successor as Apex Predator. GAN indeed. GONE, more likely. Just keep kicking the can down the road, you clever hand-wringers. Learning leads to Knowledge, Knowledge is Power, Power corrupts, absolute Power corrupts absolutely. A clearly sub-optimal general (Human) outcome, driven by an optimising, self-adjusting, closed-loop feed-back system. Brilliantly myopic.
@aniketphatak9513 жыл бұрын
damn that's deep lol
@almostbutnotentirelyunreas1663 жыл бұрын
@@aniketphatak951 Shallow: OK, you got it.
@jamesking24396 жыл бұрын
The reflections of the screen in his glasses look like googly eyes.