Jeremy Howard: fast.ai Deep Learning Courses and Research | Lex Fridman Podcast #35

[@lexfridman]

I really enjoyed this conversation with Jeremy. Here's the outline: 0:00 - Introduction 1:18 - First program 3:07 - Favorite programming languages 15:01 - Programming languages for machine learning 23:35 - Fast.ai intro (to be continued later) 24:31 - Ai and deep learning in medicine 32:30 - Privacy 37:55 - Fast.ai 40:42 - Theory vs practice 45:43 - DAWNBench - Stanford deep learning benchmark 56:24 - Fusing multiple audio and image sources 59:01 - Learning rate & deep learning as an experimental science 1:04:32 - Working with data 1:06:16 - Deep learning cloud options 1:09:12 - Deep learning frameworks 1:17:51 - How long does it take to finish fast.ai courses? 1:19:49 - Lessons from teaching deep learning 1:21:34 - Advice for people starting with deep learning 1:27:02 - Startups and entrepreneurship 1:32:21 - Anki and spaced repetition 1:40:06 - Next breakthrough in deep learning 1:41:17 - Job displacement and Andrew Yang

[@sanyambhutani5237]

Adding to the discussion at 1:17:51, Here's a writeup on how NOT to do fast.ai: medium.com/@init_27/how-not-to-do-fast-ai-or-any-ml-mooc-3d34a7e0ab8c It's a summary of all the stupid ways in which I messed up while following the course.

[@okayokay1979]

@@sanyambhutani5237 make youtube video too!

[@anyaschukin]

Thank you for the outline! Super useful :)

[@wtfbbqpwnzercopter]

At the beginning you sounded more hesitant and inhibited. There are points where you escape from that and very fluid. May I suggest you rewatch the video, identify these points and try to remember your inner state. Replicate this state and your screen presence will improve. Thank you for creating and sharing.

[@eeshanbhagwat7257]

You can add all these timestamps in the description.

[@ellisyoung5248]

Jeremy Howard. one word: Underrated. okay two: brilliant. thank you for having him and Jeremy I am glad you decided to participate in this.

[@Moiez101]

third word: the GOAT of the deep learning world.

[@antoniomax]

Wow Howard is RAW! Really to the point. And Lex is mastering his interview skills too, talk about cool questions and cool tangent drive, 10/10.

[@Creativecells]

I've watched most of the interviews, but this is truly one of the most relevant and useful ones for a practitioner in AI/DL. Jeremy has great demeanor and attitude towards his field. I also like that he has been around for awhile and has a broad perspective on how languages, tools and frameworks come to prominence, something younger guys simply cannot have.

[@c.c.s.1102]

Jeremy Howard will get his recognition in due time. He has trained the next generation. They will remember who started them on the path of research and startups.

[@oed572]

This is one of those podcasts you listen to several times

[@quangho8120]

This is actually my 3rd time watching this exact episode. Jeremy has some very interesting thoughts

[@bingbong2179]

@@quangho8120 I've just started his fastai course yesterday. I've watched a few videos now of him and I really respect what he stands for and the tangible actions he takes towards his goals

[@quangho8120]

@@bingbong2179 Yeah true. I have expertise in PyTorch, but it lacks so much little bits and pieces from fastai that actually makes training a whole lot faster

[@FacadeMan]

A man this smart, explains Jupiter notebook, seems like the biggest waste of human time. But the fact is, he understands that it's a problem worth explaining. Humility....

[@khairulhaaziq2332]

Lex we need a version 2 of this

[@seeknndestroy420]

+1

[@andremourato22]

Thank you Jeremy Howard for the several things you do for the community. Thank you Lex Fridman for creating and exposing this.

[@qlee3652]

Jeremy Howard is a superstar. Thank you for the great interview. There are amazing nuggets of wisdom in this interview.

[@NadinaRama]

What a nostalgic start of the conversation. VBA gave me fundamental lessons about automation and probably about clean coding as well. Have built huge acccess databases as well. How times are changing.

[@JackSPk]

1:24:04 "you're a saint" 👌❤️

[@ashishlogmaster]

Thank you! This has been an amazing conversation. Learnt a lot from Jeremy.

[@miguelpereira170]

Damn, Joe Rogan's podcast just got a little bit boring now, because of yours.

[@miguelpereira170]

@@tafdiz Which is precisely my point. Also, Lex is a surprisingly (i don't really know why it would be surprising, but it is to me) good interviewer.

[@stmandl]

That's the one conversation that I wish it never ended. So many insights.. thanks a lot!

[@connorshorten6311]

Awesome guest, fast.ai was huge for me getting started with Deep Learning! Really excited to listen to this!!

[@mookee]

Your channel is great.

[@connorshorten6311]

@@mookee Thank you so much!!

[@vijayabhaskarj3095]

@@connorshorten6311 I see you everywhere. :)

[@connorshorten6311]

​@@vijayabhaskarj3095 lol!

[@TraditionalAnglican]

Machismo - But, that’s the exact opposite of what he said - Listen to how long he’s been studying Chinese...

[@newenglandbarbell4647]

Wow what a great conversation 🙌 Thanks Lex and Jeremy 👏 Started Fast.ai because of this podcast 👌

[@michaels8297]

New England Barbell how do u like it

[@mccallionr]

@@michaels8297 I started it a little over a year ago and it's been one of the best decisions I've ever made. Went from basically not knowing python to creating my own datasets and image classifiers. I've recently made my own object detectors, too, using my own datasets! (Object detection means locating and identifying specific objects within a single image). (Noting that I'm using my own datasets is important to mention because a lot of libraries spoon feed you clean datasets, so a real test of whether you can actually do it is by doing it on your own unique data).

[@michaels8297]

@@mccallionr congrats man that’s awesome. Stick with it!

[@morningstar3437]

@@mccallionr what frameworks do you use

[@Nonenone-rj9yp]

This has been my favorite interview I think

[@nazgulizm]

Lex’s podcast has turned to a treasure trove of some of the finest folks of this age.

[@ZandarKoad]

Lex, your speaking cadence is positively disarming. Your content is next level, and delivered in a down to earth manner that belies it's depth and import. Well done.

[@dimadaler]

Wow I learned so much from this podcast. I even had to stop multiple times and take notes. Great job Lex!

[@melissachiou738]

Very intriguing stuff. Thanks for the time marker outline.

[@sunnyyadav1521]

This podcast is the best one ☝️ in the list @LexFridman

[@jfjfcjcjchcjcjcj9947]

@Lex Fridman, thanks for the nice interview, any possibility you can share the links regarding the image datasets Jeremy was referring to (i.e., the smaller version of imagenet and the other with the different breeds)?

[@leromerom]

Great advice, Jeremy thank you for sharing your work!

[@gerardorosiles8918]

Just in 144 minutes I got the state of the art and what to use/pick, where to go and why!

[@CesarTreetops]

exactly! this is why i've been following Jeremy so closely for the last couple years

[@owaisahmad18]

Absolutely loved this episode, gained a lot of useful information. I cannot Thank you enough Lex for doing this episode and Jeremy for all the insights and info.

[@osazemeusen1091]

Jeremy Howard makes this stuff feel easy and everybody can do it. Impressive

[@Deric_Rocks]

Mr Howard and Mr Fridman - thank you for bringing these great digestible insights to the world.

[@ladycoder2095]

I feel like Lex read my mind and asked all the questions that a beginner like me will ask this legend James Howard. Thanks a lot Lex!

[@vishwasmehra3644]

What an awesome poadcast!! Probably the best for me i have heard 🔥🔥. Learned so much in this talk that I'll take today.

[@chrisogonas]

That was such an amazing 'deep' and inspiring scientific interview. Lots to learn! Thanks!

[@bloodypommelstudios7144]

Loved this, particularly the idea of making more efficient use of data and processing. You could take a tribesman who's never seen a videogame before and have him grasp the concept and mechanics of pong in seconds but an AI would take hours. Change the graphics of the paddle, add particle effects, change the background, speed, etc and the tribesman would get it instantly whereas move the paddle 2 pixels and the AI basically has to start from scratch again. Finding ways to narrow this gap is far more interesting to me than using increasingly bigger computers and bigger pools of data.

[@dmurphydrtc]

Another great conversation. Good ML can be executed on a one single GPU / small data sets. Tried but failed with the fastai course , just need to drink from the tenacious bottle. Keep up the great work.

[@jyotipch]

Omg! someone actually started with MS Access and VB 😱😱

[@spiral3546]

Hey Lex! How do you form the questions when using Anki to learn? I used Anki t learn for exams at university some time ago. What i noticed is that it worked better when answers were either short, or more focused on thinking, than just remembering the definition. For one course i learned large definitions, so my Anki answers were long, often whole lists of things, and focused on remembering, not solving a given problem. I passed the exam for that course, so it worked. After a few months I tried to remember the things I learned, and mostly could not. This gives me a feeling that how you form the questions to put into Anki is very important.

[@TheAIEpiphany]

Super loved his out-of-the-box, no BS mindset! Undervalued podcast.

[@prabhavkaula9697]

Thank you for this podcast. It really helped me take some important AI ML decisions and ideas.

[@johnschmitt3083]

Great Episode Lex! where do you use anki on your desktop computer or on mobile? i don't think one can synchronize between them?

[@AnkitSaini-bf5fn]

Waiting for this one for quite a long time

[@chilljlt]

This was absolutely great!

[@ianborukho]

So many good reactions to all the obscure languages and their features

[@imranq9241]

Jeremy Howard's approach to deep learning is awesome. Unlike Google or big tech players, he doesn't want to rely on massive compute to search the space of good DL models. Instead, he works through his own understanding of deep learning to intuitively get to good results. Ex: Don't need autoML and try 1000s of models when you can understand the mechanics of deep learning parameters and jump straight to the best one (1:03:00)

[@vicricciuti4690]

Love your new KZbin banner and this interview with Jeremy.

[@scottaye9999]

Amazing discussion, thank you!

[@spicytuna08]

so practical and so inspirational. thanks

[@jacobvanveit3437]

Love the show Lex!

[@thelaw3536]

Love the citations in the video based on time

[@sunnyyadav1521]

Omg Thank you Thank you Thank you 🙏🏼

[@icequark1568]

This one changed my life. TY

[@DanielAllenLuka]

I literally have no idea what yall are talking about.... and yet, I flipping love it!!!

[@snippletrap]

Jeremy Howard is one of the good guys.

[@MalachiGreb]

Is there a link the the course. I do industrial automation and see a major need for ai type data structures. Even with building recipes, and layering of programs

[@PhilosopherRex]

Really great interview!

[@sathvikudupa1668]

About time. Thank you

[@steffansood6007]

This is it.. Thankyou Jeremy sir for fastai

[@user-lk6rk8hb3d]

Very inspiring. Jeremy is a great person!

[@CesarTreetops]

we need a round 2 sith Jeremy!

[@bearwolffish]

2:58 Couldn't agree more. Co author of one of my favorite books, the Pragmatic Programmer is another.

[@nikeshnaik5516]

Great inspiration

[@atraps7882]

im bout to begin the fast ai course. here to see one of the people behind it.

[@TradeStream]

I never comment. BUT... now Rogan is on Spotify I can see you sky rocketing... enjoy the ride and I deeply thank you for sharing all of these insightful people. Be well.

[@sardorabdirayimov]

Adding current video to recommendation list 😇

[@NoNTr1v1aL]

Absolutely amazing video!

[@NaumOne]

This should have much more views. The secret of this interview is 0% hype in Jeremy words

[@hashkenhabib]

"99% of all A.I research is basically useless". Aaah. I see he is a man of culture as well.

[@eugenekim4523]

Is there a link to the paper Lex researched on transfer learning?

[@Tondadrd]

I feel I learned so much!

[@williamramseyer9121]

Incredible interview. Jeremy Howard seems like someone interested in the truth with a great mind and a big heart. I feel humbled when I imagine how much knowledge these two men have. My comments on AI displacing workers. This is a problem wrapped up in the history of market economies and technology. The theory: Competition leads to technological advances (to reduce costs) which leads to deflation (cheaper to make things and provide services) which culminates in a depression (which eliminates inefficient industries, companies and jobs). Then it starts over. New industries and new jobs. Until the Great Depression of the 1930s. Hitler and WWII interrupted that depression, and governments after WWII resolved to never allow deflation and depression again. So, they have stimulated debt increases to re-inflate economies (most currency is in the form of debt) and offset technological deflation. Unfortunately, cheap debt leads to economic disparity (the wealthy can borrow at low interest rates but the poor can’t borrow at all), financial bubbles, and “zombie” companies. I hope that some of you out there, who are smarter than me, can solve the following problems: 1) how to distribute the wealth created by technological advances without destroying market competition (if we have no depression re-set creating new jobs, then people who no longer needed in the economy need help from the government to survive and additional assistance to live a rich and creative life without jobs); and 2) how to unwind the massive buildup of debt without either depression or hyperinflation (both of which eliminate debt but also cause extreme social pain), or war. Good luck! William L. Ramseyer

[@arindamsarkar9009]

Wow.This one is really nice.Please bring Jeff Dean.

[@vikashkumar994]

Very enlightening conversation.

[@_ramen]

The lower the Lex Fridman podcast episode, the closer you get to machine code.

[@MrDangar11]

Thanks for sharing this kind of inspiration and knowledge for Z generation from Russian, our professors didn't bother about sharing they knowledges because they bother only about their position at the university and they won't accept that someone better then them in new programming field! From all students from Russian: We appreciate your work for free knowledge, most of us very smart persons but we can't afford teaching at your universities!

[@xiaokourou]

Spaced repetition for the win! Can't believe they both are into it. That's part of what my app is about

[@Nova-Rift]

best guest so far

[@marloncajamarca2793]

Totally agree with lex at 2:58!

[@zhongzhongclock]

Delphi, I recalled it

[@bell1095]

Could you pls open the subtitles ?

[@milantripathi720]

Besst podcast for Artificial Intelligence(Deep Learning)

[@MrMehrd]

Cuda language is hard Active learning labeling techniq is underrated, fluent in transfer learning 46:00 50:00 1st train with small images, then teach with the actual image size 59:00 read unpublished papers,10x faster ,less epochs

[@PhilippeLarcher]

"Image Nette" translates as "crisp image"

[@cmag8924]

Finally a fellow k, J, and F# lover :)

[@josephbertrand5558]

Love the Large bottle of ADVIL in behind the guest...for vodka hangovers?

[@javierfernandez6327]

I find the history interesting - particularly for what not to do - But looking at architecture from the ground(registers) and how today's dominant languages scale up and each use the prior. This is assembler, c, c++, java, c#. I developed in many of the old languages and they all had interesting features, but they all lacked completeness and connection to the metal. This is the elegance behind c/c++ they keep you connected to what is happening on the chip. Programming without understanding of CPU architecture is like an autonomous machine without sensors.

[@TeMp3rr0r]

Yes! TPUs are harder to program and very slow to load up data! Someone telling the truth finally.

[@danellwein8679]

thank you for this … I am 64 .. but fastai .. here I come ..

[@leon120897]

Pure gold!

[@danwilks7263]

Yeeeees!! Thank you

[@kozepz]

Thanks.

[@JohanKarlsson]

Super insightful

[@goldfish8196]

Bravo!

[@thenoorer]

2:50 Lex:" what's your favourite instrument?", Jeremy:"Saxophone", Lex: "SEX" . that escalated quickly

[@Hexanitrobenzene]

Oh, come on, it's "sax".

[@allenculbertson8170]

God bless you and thank U

[@synonym0puns]

Jeremy Howard seems to be proud he is going in a direction opposite to google and big companies by not scaling models through scaling hardware and data... This has tones of when computer programmers opted not to optimize code instead they would just wait for moore's law to release better hardware in the future. I wonder which approach will win out Innovations in algorithms? or Scaling up Data and Compute

[@ataru4]

I wonder what Jeremy thinks of the Julia language. The team have been putting more ficus on deep learning the past yr and for dara processing it would seem a better fit than Swift.

[@jamesoneill4571]

Great talk, but to say that his paper with Sebastian Ruder was the first introduction of successful transfer learning in NLP is just not true. "Natural Language Processing (Almost) from Scratch " showed success in 2011 and few papers before 2011 also

[@larslowesjosund2751]

Same with the statement that Enlitic was the first medtech company focusing on deep learning; at least there’s the Korean company Lunit that was founded a year before Enlitic.

[@Hexanitrobenzene]

Interesting, what would Jeremy say about Haskell in terms of elegance/cleanness and Julia in terms of balance between flexibility and speed ?

[@suchalooser1175]

Jeremy is awesome!!!

[@linchenpal]

That beauty of science...

[@lsfhieber]

Lex! When can I interview you?

[@owensoft]

I understand his frustration with modern programming languages that focus on tools rather than being compact and useful.