ResNets are back baby!!! ...and they're gone | Machine Learning Monthly March 2021

Machine Learning Monthly covers the latest and greatest (but not always the latest) in machine learning of the previous month. This month we see things like multimodal modelling (modelling multiple sources of data types such as images combined with text) and self-supervised learning taking centre stage.
Machine Learning Monthly March 2021 - zerotomastery.io/blog/machine...
EfficientNetV2 - arxiv.org/abs/2104.00298
Learn:
Beginner-friendly Machine Learning - dbourke.link/mlcourse
TensorFlow for Deep Learning - dbourke.link/ZTMTFcourse
Connect elsewhere:
Web - dbourke.link/web
Twitch - / mrdbourke
ArXiv channel (past streams) - / @danielbourkearxiv2821
Get email updates on my work - dbourke.link/newsletter
Timestamps:
0:00 - Intro/hello/dancing
0:54 - My work from the past month
2:21 - Why is machine learning hard? By Zayd Enam
4:13 - Software 2.0 by Andrej Karpathy
4:32 - Start with less fancy networks by Victor Sanh
6:50 - Airbnb’s Multimodal model for images, text and structured data
10:16 - Learning from Videos by Facebook AI
13:10 - Self-supervised Learning: The dark matter of intelligence by Yann LeCun
15:48 - SEER: A new state of the art for self-supervised computer vision
18:39 - Paper of the month: ResNets are back baby!!!
21:55 - ...and they’re gone, welcome EfficientNetV2
24:03 - Super-powered Speech tools: Coqui & SpeechBrain
26:03 - The Deep Bootstrap Framework
30:27 - Summary/outro/more dancing
#machinelearningmonthly #machinelearning

[@mrdbourke]

Two weeks ago: ResNets are back baby!!! Yesterday: EfficientNets are back baby!!! It never ceases to amaze me how fast this field moves...

[@programmerrdai]

lol

[@programmerrdai]

True

[@yangxiang6094]

Good stuff again dude! read those paper and it was amazing, BTW thanks for all the sharing and passion you give to our community, I found a job as a data scientist in a Chicago startup unicorn recently , while back in 2019 I knows nothing about DS..then I found your channel while searching for ML help online.. keep learning and never stop!

[@ftmftm7627]

You are wondeful man! Your ml journey from zero to hero is inspiring ❤

[@adler3001]

Happy Birthday Sam 🙏

[@mrdbourke]

I showed Sam this and he said Thank you Erik! 🎊

[@mateusfernando5847]

Great work mate! Happy bday Sam🎉

[@mrdbourke]

Thank you Mateus! Just showed Sam, he says thank you too

[@mehdizobiri9267]

Gosh i love this series of ML Monthly , keep it up Dan

[@mrdbourke]

Plenty more to come!

[@sergey902]

Happy Birthday, Sam!

[@mrdbourke]

Just showed him, he says thank you Sergey!

[@joshcummins3916]

Happy Birthday Sam. Much love to you and your family.

[@mrdbourke]

Thank you Josh! I’ll pass the well wishes on!

[@roughr4044]

Happy related Birthday Sam!

[@mrdbourke]

Just showed him, he said thank you!

[@RevenueSharePartner]

*Happy Belated Birthday Sam, What are you planning to do, are you following your daniel's footstep, you're lucky to have him as a brother*

[@user-to7dp2uc3o]

Hi! your videos are always great and useful

[@mrdbourke]

Hey hey! Thank you for the kind words

[@easonlyon2217]

Amazing work, you are going to make my FYP project pass with flying colours!

[@mrdbourke]

All the best Eason! You’ve got this

[@JesseMathiu]

Happy birthday sam !🥳🥳

[@mrdbourke]

Just showed him, he says thank you Jesse!

[@programmerrdai]

Happy b-day Sam!

[@mrdbourke]

Just showed him this, he says Thank you Ranuga!

[@ayushchakraborty4406]

Happy belated birthday Sam!!🎉🎉🎉🙌 A day late😅 Also great vid as always mate!!

[@mrdbourke]

Woohoo! Thank you Ayush! I’ve passed the message onto Sam

[@basavarajpatil9821]

Happy Baday Sam

[@gero8049]

Hey. Whats that trackpad that you use?

[@danielniels22]

hello sir, I am new to your channel. I am here to ask pretty forward, what do you think about the Machine Learning Job Opportunities? Is it that scary to apply it? Considering hard technical skills from various programming language to math skills... Do you really need to have PhD? Well sorry for asking this... But do you have a video where you ever talked something about this? thank you...

[@kaushikhegde9384]

Hi Daniel. When is the new tensorflow course coming to udemy? Cant wait for it. Really enjoyed your Machine learning course earlier.

[@mrdbourke]

Hey Kaushik! Thank you for the kind words, as for Udemy release, see here: github.com/mrdbourke/tensorflow-deep-learning/discussions/34

[@kaushikhegde9384]

@@mrdbourke Will wait eagerly :D

[@thedailyepochs338]

Hey Daniel, how's your brother doing, I knew his background was finance related,how has he been able to combine machine learning and finance together. Btw Happy birthday sam

[@mrdbourke]

Hey Nifesimi, he’s going really well in his accounting/auditing role & trying to squeeze in more data science learning when he can (the job itself takes a lot of his time) PS I showed Sam this message and he says Thank you!

[@thedailyepochs338]

@@mrdbourke Yeah i am going through the exact same thing here too, i work at Pwc , I hope we both accomplish our goals

[@HarshithMK]

Self supervised learning sounds super cool!!

[@mrdbourke]

I agree, it definitely does, I’ll have to make a video on it

[@adityakane5669]

@Daniel Bourke Thanks man, this was damn useful gg

[@mrdbourke]

Thank you Aditya! Glad you enjoyed

[@egehurturk6209]

Have you heard about Graph Neural Networks (GNNs)?

[@mrdbourke]

I have heard about them but only really know the name of them rather than what their use case is

[@adityatiwari6247]

Do you use power bi?

[@mrdbourke]

I don’t

[@deepblender]

I was surprised by the popularity of EfficientNet as it was seemingly actually being used. For me, they were always way too slow. Apparently, this is something that should be fixed or at least improved with V2.

[@mrdbourke]

Oh really? What did you find worked better for you instead? I remember seeing a fair few George Hotz streams saying they were using EfficientNetB4 on their comma.ai cameras

[@deepblender]

@@mrdbourke For me ResNet architectures were always a lot faster, for training from scratch, transfer learning and most importantly for inference. To be clear, I am only training on my GPU, but it still made me wonder whether I am doing something wrong and whether there is a way to get faster depthwise convolutions.

[@lucyledezma709]

Hello Daniel !. I am a new suscriber!..

[@mrdbourke]

Yo Lucy! Welcome to the channel :)

[@lucyledezma709]

@@mrdbourkethanks!!! I really like data science, and also your videos;

[@mrdbourke]

amazing to hear! Let me know if there’s anyway I can help

[@lucyledezma709]

@@mrdbourke data science is an area in which I would like to specialize; I am receiving a Bachelor of Information Systems; and I think your videos will help me a lot! thanks and greetings from Argentina

[@zshan101992]

ml monthly Like usual. That drake in the beginning 👍😄

[@mrdbourke]

Haha glad you liked that reference!

[@gundavenkatashanmukhasaina8474]

Official code for EfficientNetV2 haven't released right? By the way thumbnail is 😂👌.

[@mrdbourke]

Ya I believe it isn’t out yet But I’d like to see more code before papers And thank you, I’m a fan of the thumbnail too!

[@alexchan5643]

the resnet revisited team be like 👁💧👄💧👁

[@mrdbourke]

I’m trying to imagine the feelings...

[@justinknott6217]

Hey Daniel I sent an email about your video on Titanic comp ... do you have the same email ... haven't got a reply yet ... I would appreciate some insight

[@mrdbourke]

Yo legend! You’re not looking hard enough! The link to the code is in the description: github.com/mrdbourke/your-first-kaggle-submission/

[@maksymmanziuk4108]

The EfficientNet is dead. Long live the EfficientNet

[@mrdbourke]

😂

[@MrAI_Rohan]

Add time stamps!

[@mrdbourke]

I knew there was something missing! Added!

[@MrAI_Rohan]

@@mrdbourke Thank You! ❤️

[@Naolo1608]

first

[@mrdbourke]

Winner winner chicken dinner!

[@thedailyepochs338]

@@mrdbourke lol I wish I had your enthusiasm to be honest