Jeremy Howard: Deep Learning Frameworks - TensorFlow, PyTorch, fast.ai | AI Podcast Clips

This is a clip from a conversation with Jeremy Howard from Aug 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. You can watch the full conversation here: • Jeremy Howard: fast.ai...
(more links below)
Podcast full episodes playlist:
• Lex Fridman Podcast
Podcasts clips playlist:
• Lex Fridman Podcast Clips
Podcast website:
lexfridman.com/ai
Podcast on iTunes:
apple.co/2lwqZIr
Podcast on Spotify:
spoti.fi/2nEwCF8
Podcast RSS:
lexfridman.com/category/ai/feed/
Note: I select clips with insights from these much longer conversation with the hope of helping make these ideas more accessible and discoverable. Ultimately, this podcast is a small side hobby for me with the goal of sharing and discussing ideas. For now, I post a few clips every Tue & Fri. I did a poll and 92% of people either liked or loved the posting of daily clips, 2% were indifferent, and 6% hated it, some suggesting that I post them on a separate KZbin channel. I hear the 6% and partially agree, so am torn about the whole thing. I tried creating a separate clips channel but the KZbin algorithm makes it very difficult for that channel to grow unless the main channel is already very popular. So for a little while, I'll keep posting clips on the main channel. I ask for your patience and to see these clips as supporting the dissemination of knowledge contained in nuanced discussion. If you enjoy it, consider subscribing, sharing, and commenting.
Jeremy Howard is the founder of fast.ai, a research institute dedicated to make deep learning more accessible. He is also a Distinguished Research Scientist at the University of San Francisco, a former president of Kaggle as well a top-ranking competitor there, and in general, he's a successful entrepreneur, educator, research, and an inspiring personality in the AI community.
Subscribe to this KZbin channel or connect on:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman

[@lexfridman]

This is a clip from a conversation with Jeremy Howard from Aug 2019. New full episodes every Mon & Thu and 1-2 new clips or a new non-podcast video on all other days. If you enjoy it, subscribe, comment, and share. You can watch the full conversation here: kzbin.info/www/bejne/gGe7lINqf7R7o80 (more links below) Podcast full episodes playlist: kzbin.info/aero/PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 Podcasts clips playlist: kzbin.info/aero/PLrAXtmErZgOeciFP3CBCIEElOJeitOr41 Podcast website: lexfridman.com/ai Podcast on iTunes: apple.co/2lwqZIr Podcast on Spotify: spoti.fi/2nEwCF8 Podcast RSS: lexfridman.com/category/ai/feed/

[@seriouscoder1727]

Rust is future of deeplearning,isn,t it?

[@clapton79]

I wish I was on a level where I can say "I am limited by python" LOL

[@DominicBrissette]

I totally did not expect to hear him mention Swift

[@EhKurd]

I'm not sure what Jeremy is thinking. Swift doesn't support Windows and probably never will; I'm not even sure how good the Linux support is. He said it himself at the end: Apple doesn't care. I think Swift is just an awful choice especially considering FastAI's goal of making AI more accessible. Accessibility is the complete opposite of what Apple stands for.

[@eastriverlee5296]

Sub that is not true, swift 5.3 was just released and windows will be officially supported in the comming weeks.

[@DragonKidPlaysMC]

Plus Swift is now open source.

[@brunospasta]

Tensorflow code is quite the mess. The whole tree structure of your model makes so much more sense in pytorch.

[@sayantanmazumdar3]

When TF engineers made their libraries, they did not expect it to be used by millions of people all around the world one day.

[@waqasaps]

"The whole model makes sense in PyTorch" beautiful words

[@tristunalekzander5608]

Who needs TensorFlow when you have assembly

[@juliansuse1]

Assembly is for pussies, I train my own biological neural network!

[@jonathancauchi6457]

😅😂

[@caiomatheus817]

@@juliansuse1 Personally, I don't like biological neural networks. They are too bised and half of the time they output gibberish.

[@Scott-fd7zi]

@@juliansuse1 only takes 8 billion years

[@lividpudding8565]

Who needs assembly when you can move electrons with your bare hands

[@aigagror]

I've been using TF 2.0 for a while and after hearing this, I'm switching to PyTorch. I didn't know it was x10 slower...

[@TheMazyProduction]

Eddie Huang I think he was specifically talking about eager executions in TF.

[@binmosa]

This is a very quick decision :D

[@amdenis]

Eddie Huang That would be a HUGE mistake! I develop in TF, FAI, Pytorch and Matlab, and for most things beyond eager experimentation and prototyping, or poorly written TF 2.0 function code, TF is on par with most things out there- and much better supported in several critical areas for GPU/TPU training. Now, the “best” choice depends on use case, and whether training, inference, maintainability, deployment, scaleability or something else is your bottleneck or overriding priority; but dropping Tensorflow at this time, is like dropping Java in the 90’s- a huge mistake. Also, forget about what he said about being able to migrate over to “pretty much any other library” within a few days. We work with a lot of new as well as seasoned AI and data science people, and it takes a seasoned AI person weeks to start to truly get a decent grip of how to use a modern, complex framework on a professional level (i.e. knowing the landscape of how to semi-optimally use that framework)- and we can definitely tell the difference in the development time and quality of the work done by a professional, let alone a newer AI developer, who has worked in something like TF for several months versus a few years. It takes several thousand hours of commitment in any of the major frameworks to begin to truly stsrt to master it. You can do great work well before that, but the real curve is long. The most important thing is to choose a good framework that (1) supports what you plan to do; (2) choose one that has legs, and will stay at least relevant and ideally dominant- as that will ensure a solid base of support on many levels. Tensorflow, especially with Google and Chris Lattner (the developer of Swift) definitely fits that bill. So, it is more important choose well and then go deep and long with a major framework first. Things like training time, inference performance, eager execution performance and such will be taken care of by the industry, as they have to be competitive to survive and thrive- and if you don’t think that Google has and will continue to make a major long term commitment of top resources, you’re mistaken.

[@undisclosedmusic4969]

It is not. Tensorflow 2.0 is a fantastic, extremely fast framework without any kind of regression behind PyTorch. I don’t even understand that statement from Howard...

[@thedeathstar420]

It's pretty good if you don't use eager execution.

[@digital_down]

I’ve been noticing lately that Pytorch seems to be gaining popularity and Tensorflow is getting less so. Maybe just me though, idk.

[@shulavkarki4253]

Which one would u prefer,?

[@wildreams]

Yes.

[@wildreams]

@@WW-to5rc Why?

[@billy818]

Industry is tensorflow research is pytorch. Trying to put pytorch into production is a hassle.

[@forloop7713]

@@billy818 which one is for good for hobbyists

[@tylersnard]

I'm happy to hear that Jeremy shares my opinion of Tensorflow.

[@awiewahh]

The only thing keeping me around TF is its compatibility with large TPUs. If PyTorch can start nicely interfacing with something like a V3-512 then that'd be great.

[@beyondmeaning]

Thanks Lex, very informative

[@adityaranigaon]

If you want advise as a new student, go to 6:16

[@CapitanFantasma1776]

Gosh, Lex. I was just thinking. What are you doing with all this information you are receiving? The great minds of the modern era.

[@Raven-bi3xn]

I wonder if Tensorflow customizable features has changed anything in tf vs torch equilibrium.

[@fgfanta]

Super-informative, thank you!

[@aniruddhadatta925]

I learnt almost everything I know about deep reinforcement learning is due to reading code in Pytorch .....thanks Pytorch

[@fragnet7673]

I used FANN a lot. Wich framework can do the same in an equal easy way?

[@vigneshrb1626]

With the introduction of keras API tensorflow coding has become a lot easier and accessible

[@MsFearco]

Yeah. Tf might be slow but in all honesty a beginner will see no difference. A 6 hour training for me in tf is still a 5.5 hour training in torch... 10% faster != 10x...

[@viniciussantana8737]

@@MsFearco you can take advatange of free gpu and tpu with tensorflow.

[@kushagrachaturvedy2821]

I agree. It's great for beginners to cut their teeth on but with Pytorch I feel you get a much better idea of what you're doing and your knowledge is tested and increased

[@SpellboundHaven]

Sir Jeremy Howard always reminds me of Gale Boetticher from Breaking Bad.

[@TheReidNantes]

A Practical Deep Learning for Coders course focusing on Swift for TensorFlow would be amazing!

[@normandblais5219]

After listening to that, I need some Advil.

[@JousefM]

Right behind Jeremy :D

[@justinmasters220]

With a nice cold bottle of Essentia water.

[@seydoudia7828]

A great example of product placement ;)

[@wolfisraging]

I love MXNet more than any DL framework by far. It's the fastest framework I've ever used. Also it comes with both imperative(useful for research) as well as declarative (useful for production and deployment) paradigm. Not just that you can convert your code written in dynamic graph to static graph in just one line. Also...... It's got more pretrained models than any other framework. I'm glad that I made my switch to MXNet.

[@StEvUgnIn]

The problem is that the last version running on Windows is too ancient

[@StEvUgnIn]

Support for CUDA is also limited if you choose to run it in a container

[@ciyoritana1740]

Anyone can answer me? What deep learning framework should i install if my laptop is core i3 and my VGA is intel and nvidia 820m? Well, i want to work on my thesis but unfotunately i just realize that my laptop is not too good to train with some framework. Or anyone can suggest some advice for what should i do?😭

[@mihirbhatia9658]

Just try using keras/tensorflow/pytorch in Google Colab. You can get quite a bit of work done using their GPUs. It's free upto a certain level.

[@codecampbase1525]

Google collab

[@andreinedelcu2005]

Why would you execute it with tf eager activated? I would use that only for debugging until I get a stable version. Then disable all sugarcoating and stay with the static graph that is so well optimized in TF 1.x

[@temporarychannel4339]

The Advil in the background explains a lot

[@maxitube30]

can someone reassume what they say in video? I'm not english first language and i understood small piece of what they sayd. I understood that python is limited,and tensorlow is a mess. But,if he created fast ai in python,and he already saying that can be slow,what is the purpose of fastai? is fast? it can use speed of gpus?

[@Trepetsky]

Jax, very flexible, but also fast, faster than PyTorch. And you can still run on TPU. Yes, now support for TPU has been added to PyTorch, but it was recently done, I tried it, it's still raw.

[@CapitanFantasma1776]

Man, I was interested in Python until now.

[@amdenis]

He was correct then, but now Tensorflow kicks butt. Although we have used several of the major AI frameworks, 90% of what we and most others do in AI are more and more best done with TF. This video is a few years old, and it has only become more the case.

[@proweiqi]

"Swift does dumb things" and "Python isn't going to cut it". Julia it is! Please just invest your OSs time in Julia!

[@karigucio]

My experience usually was that splitting the definition of a computation from its execution is beneficial. Can someone shed light on why it wasn't the case with tensorflow? And how is tf.eager slow, when - given I understand correctly - the python overhead may only apply to the construction of a computation graph, which is not that crucial?

[@michealhall7776]

I didn't notice upspeak until watching Joe organ. The upspeak is strong here

[@wolfisraging]

By now, I think a lot have been changed.

[@udaylunawat2163]

What do you think? Has tensorflow overcome the disadvantage?

[@kirtipandya4618]

Even though TF is slower, it is okay as TF Lite is more useful but thing is TF is very difficult to debug. but TF is difficult to debug. I started with TF but later switched to PyTorch. Maybe Lex can improve that .

[@rahulsingh7508]

PyTorch is a beast.

[@jamespaz4333]

Sometimes when you cage the beast, the beast gets angry.

[@amdenis]

Mostly great information, but it seems like he was speaking about 4 year old TensorFlow and Keras about 2 years ago. We use 12 different frameworks, 4 on a regular basis. Many of the supporting libraries and extensions are able to be used across most of the frameworks in fairly equivalent form. The appropriate development frameworks and supporting libraries really depends on the specific problem space, requirements of the delivered solutions and other factors. Swift and various ancillary language environments, including Node.js (used correctly), enables amazing productivity and performance. I totally do NOT agree with starting on Fast and PyTorch, as it is like when they taught Pascal because it was a better learning language with fast ramp-up, even though it was built with a limited professional carry-forward in mind. The basis of PyTorch is somewhat self-limiting, and even Jeremy asserts the same in this and many videos. I have generally found that the best tool at the time is the one you know best that can solve the given problem. If that is the case, why not learn the most robust long-term viable solution set first, and become familiar with secondary environments like PyTorch along the way? That way you get the best possible long-term reusability of skills and are buying into an architecture that is not edging towards marginalization or end-of-life. You can and many should learn Py-based frameworks along the way, while they are building a development skill set that they can grow with for many years.

[@chaidaro]

Ouch! 3:58

[@ApPillon]

I have tried both tf and pytorch, tf was easy to get in but hard to progress, I like and use pytorch these days

[@aniruddhadatta925]

Almost all Reinforcement Learning libraries are written in pytorch now .....I think it says something

[@aryaakhil8147]

I like the honesty. Hard to come by such people these days in the era of political correctness.

[@averageengineeer]

Any comments on Julia and it's future in machine learning ecosystem ?

[@pythgoreansassistant2789]

Tensorflow's eager execution is what bugs me. It limits what you can do in calculating gradients & layers.

[@L4rsTrysToMakeTut]

Does anyone use Julia programing language?

[@proweiqi]

Yeah. Flux.jl is more awesome than pytorch. Just needs more development!

[@chrisrackauckasofficial]

Yes, it works great for me to write non-standard neural networks from scratch for research purposes.

[@knightvertrag]

Use C bro.

[@alefratat4018]

What are the C Deep Learning Frameworks out there ?

[@keedt]

@@alefratat4018 write your own, bro

[@alefratat4018]

@@keedt Nevermind, I found one that is really good

[@quangho8120]

@@alefratat4018 PyTorch does have a C++ interface tho

[@bmoore813]

Im assuming this is based on tensorflow 1 and not tensorflow 2

[@devawratvidhate9093]

Why you have not change u r suit in during quarantine ....!

[@amarthapa9768]

Isn't Swift from Apple? Why not take Kotlin which has support for native. It's open source too.

[@aayush_dutt]

Why Swift, why not Rust or Go?

[@datonefaridze1503]

Pytorch is much simpler and more powerful rather than TF.

[@sleeping4cat]

Umm! Tensorflow provides much more customisation than PyTorch. PyTorch is okay, if you're doing Engineering but in terms of experimentation Tensorflow is a gem!

[@MattUebel]

RIP Swift for Tensorflow

[@sigmaroll9802]

A new vista

[@nbansal]

Even after 3 years, Swift for Tensorflow hasn't caught up 😅

[@sibonelongobese8639]

This guy is honest. TensorFlow is still difficult to use, especially when you are a Data scientist with no solid programming background.

[@MultiMediaUploads]

How are you a data scientist without a solid programming background?

[@alvin3171997]

MultiMediaUploads probably using just GUI data visualisation tool

[@udaylunawat2163]

Just say programming is hard.

[@imdadood5705]

I am an aspiring Data Scientist. I have invested a lot of time in coding. I have a seen a lot people telling that TensorFlow doesn’t help us understand what happens behind the scene. But PyTorch, the model is much more transparent. I am not an expert in both the libraries. But if this is true, I’d pick PyTorch.

[@ayushtiwari8857]

Swift for Tensorflow is officially closed. Kind of disheartening tbh :/

[@blackswaneleven]

TF 2.0 still suxx compared with PyTorch

[@pixel7038]

I feel sad he is working on Swift. Switch to flutter where the phone market shines :)

[@satoshinakamoto171]

eww swift..