Groups, Depthwise, and Depthwise-Separable Convolution (Neural Networks)

Patreon: / animated_ai
Fully animated explanation of the groups option in convolutional neural networks followed by an explanation of depthwise and depthwise-separable convolution in neural networks.
Animations: animatedai.github.io/
Intro sound: "Whoosh water x4" by beman87 freesound.org/s/162839/

[@sajanphilip8221]

Please don't stop making videos. They are of great help. Thank you for your efforts.

[@yousrakateb2383]

Please continue make such amzing videos....they really helped me

[@leonardommarques]

You got a new subscriber. You are 3b1b of AI. Thanks for existing.

[@crazykidsstuff]

This is amazing! There is a lot of great material out there, and your channel is a really solid and valuable contribution to that. Thanks a batch!

[@danirubio2847]

This was an incredible video. You can see the the amount of work and dedication; and you explain really good! Thanks, please keep on doing this videos

[@mynameisvahe]

One of the best channels on DL I’ve seen so far. Please publish more!

[@yanickpalmers]

absolutely love these videos! doing gods work

[@katuz9366]

Hey, I want to thank you for spending time making great content animated! I’ve been using depth wise for a time and it has been always a little hazy

[@CBhattacharjee]

This is great stuff. Please continue to make more, you are saving new scholar lives here!

[@linkernick5379]

This is so underestimated channel, will share it as much as I can. Thank you, Mr AI Animator!

[@Mars-xm7uz]

Your work is truly amazing, please keep enlighting us!

[@user-ty6qc5dg8i]

I just wanted to say a huge THANK YOU for all the incredible animations you've been creating. Your work has been a game-changer for me, making complex concepts so much easier to understand.

[@hashimsharif1542]

By far the best explanation of depthwise-separable convolutions I found! This is a service, thanks!

[@slime67]

amazing! as an AI researcher I missed these videos back in the days when I studied convolutions, hope they'll bring more understanding to the people just coming to the field!

[@SajjadZangiabadi]

Absolutely loved the way the instructor used animations to explain concepts like Groups, Depthwise, and Depthwise-Separable Convolution. It made understanding the topic so much easier and engaging. Keep up the great work!

[@janphiliprichter96]

Incredible video! Brilliant visualisations and perfect explanation. Keep it up

[@neeleshgupta3491]

Amazing video, so well explained and to the point.

[@ShadeAKAhayate]

Thanks, the explanation of this mechanic is exceedingly lucid.

[@mingwen3854]

Very intuitive to understand, thank you.

[@homakashefiamiri3749]

The best conv layer visualization so far👍 Thank you for your great work💥💥

[@j________k]

Great stuff... the algorithm should give your content more attention!

[@assthera]

Bro its amazing, continue please !!

[@alexandrostsagkaropoulos]

That was so intuitive. Thanks for that!

[@matheuscardosomoraes5641]

Congratulations for amazing class

[@lakshaysethi7445]

Great Work. I am a Master's student in ML and I your animations are really helpful in understanding this concept!! Thanks a lot.

[@josephpark2093]

You are a freaking saint. I gotta sub for the effort you put in.

[@happyTonakai]

This is so great!

[@petyap7600]

Finally understood. Thanks. Really helpful videos.

[@user-qs2di6lj1r]

Thank you for making videos like this.

[@alexgamingtv7118]

Truly awesome!

[@himanshu1373]

thankyou very much brother this video means a lot for people like me 😍

[@user-iz7hg1iu9s]

this animation really helps me , thanks!

[@VlZlA]

Great video! Keep up the good work

[@pa7235]

great material!

[@naasvanrooyen2894]

Thanks! Best explanation ever

[@dimitrisspiridonidis3284]

amazing job !!!

[@xXMaDGaMeR]

Great work, thank you

[@revimfadli4666]

Fantastic explanations, even though I understand the paper diagrams, this makes it super clearer. Would you cover cascaded/DenseNet someday?

[@AnonymousIguana]

Fantastic!

[@user-bh2cn3zf3y]

Great video... keep it goining..Thanks a lot

[@thivuxhale]

this is too cool to handle!!!!

[@guyindisguise]

Another great video. Can't wait for you to go into animating Transformers!

[@darmstadtbeste4590]

awesome thanks!

[@joaomartinho4845]

so so good

[@DanielTorres-gd2uf]

Nice job!

[@100deep1001]

underrated channel

[@dennervasconcelosrodrigues1212]

Thank you

[@martinhladis1941]

Excelent!!

[@grayboywilliams]

Thank you!

[@xXMaDGaMeR]

ok visualize transforms next please, Vision Transforms would be nice.

[@kevalan1042]

yes, a good visualization of transformers would be great

[@revimfadli4666]

Also graphnet

[@sk7w4tch3r]

Thanks!

[@dandoe2461]

great video

[@user-jz5vv1pw1t]

Thank you very much for your sharing. It helps me a lot. I would really appreciate it if you could add subtitles

[@pi5549]

Do remember in future vids to invite viewers to smash the like button, as it improves your ranking as per the Algorithm. I just realised I watched half a dozen of these without hitting it.

[@ayeletgarber3697]

תודה!

[@sharjeel_mazhar]

Please sir, also make visualizations like these on RNN, LSTM and most importantly Transformers. Would be really thankful to you. And also, your videos on CNN are just gems in the ocean of youtube.

[@-mwolf]

Thanks! Could you also cover convolutions with processing audio?

[@abdulrahmankhalaf2982]

Awesome job, I have a quesion out of the box, how you are did this work? which programs used in this video to produce it?

[@valovanonym]

Great vidoe! Your website will be a very usefull ressource. May I ask you what tool you are using for creating these animations?

[@animatedai]

I'm using Blender and relying heavily on the Geometry Nodes feature.

[@mathieugalle391]

Great work, thanks ! Maybe FPNs next time ? :-D

[@adrienkin]

From your example, it could be nice to give the number of computations as example of +/- 9x faster :)

[@troyhan3741]

I want to know how can you make this video , what tools of software you used ?

[@sinkseeker]

Hi Animated AI, for clarification, are the stacks of cubes in the first 30 seconds of the video feature maps? Also, how exactly did the depth increase as we get into the deeper layers? Based on my understanding, the lecture you provided was focused more on maintaining the depth while increasing its efficiency. I hope to hear from you soon! Your work is great!

[@animatedai]

That's correct, they're the feature maps which are the inputs/outputs of the layers. The depth of a feature map is equal to the number of filters in the convolutional layer that created it. So the depth increases that you're seeing are simply layers that have more filters than the number of features in their input. Let me know if that isn't what you meant by your question. This video shows the depth staying the same in a depth-wise separable convolution, but you can still depth-wise separate a layer that increases the number of filters and get the performance benefits. You can just take the input depth and use twice (or some other multiple of) that value for the filter count in both the depth-wise and point-wise convolutions.

[@sinkseeker]

@@animatedai I see, I see. So if the input is an RGB image, and the first convolutional layer uses 5 filters, then the depth of the feature map will be 5. If that feature map goes to another convolutional layer with 5 filters, will the output contain a feature map with a depth of 25?

[@animatedai]

In that example, both outputs would have a depth of 5 because both layers have a filter count of 5. My video on filter count might help you visualize the relationship there: kzbin.info/www/bejne/j4SxfYCEo9GBrZo These videos are both part of this playlist on convolution: kzbin.info/aero/PLZDCDMGmelH-pHt-Ij0nImVrOmj8DYKbB

[@sinkseeker]

@@animatedai Hi animatedAI! I'll check the link out. I hope I'll get it afterwards haha. Thanks for sharing!

[@Antagon666]

Soo the output has the same number of channels as the input? Or can you modify that by 1x1 convolution at the end ? Also doesn't this double the required storage for feature maps ?

[@animatedai]

In practice, it probably doesn't make a huge difference where you increase the number of channels. You could increase the channels in the depthwise convolution as long as you wanted the output channel count to be a multiple of the input. EfficientNet actually increases the number of channels with an extra pointwise convolution before the depthwise convolution. Yes, it increases the storage required during training in TensorFlow and PyTorch. Post-training, you don't necessarily need to keep around all the intermediate feature maps. So whether or not it doubles the required storage is dependent on the library (if any) that you're using for deployment.

[@commanderlake7997]

It should be noted that this will not scale well with tensor cores and may even be slower.

[@a.h.s.2876]

❤

[@meguellatiyounes8659]

Can we convert trained standard Convnets to depth wise ones ?

[@animatedai]

You could theoretically separate any kernel into a depthwise-separated one. But you'd need a lot more filters in the depthwise convolution, so the result would be about the same performance. The performance improvement comes from training the network to take advantage of depthwise-separated convolutions.

[@martinm8179]

These videos are excellent, but I suspect your ability to discern adjacent colors on a color wheel greatly outpaces mine. I have to pause and stare back and forth between blocks. It would be nice it were easier to see. Tools like Viz Palette can help pick better colors for data visualization.

[@animatedai]

I appreciate your feedback! I could rant for hours about how hard it is to pick colors :). I have two clarification questions: 1) Which part of the video did you pause to stare back and forth between blocks? 2) Which feature of Viz Palatte do you think would have helped pick colors for that instance?

[@cardinalblues7121]

please be more productive . Your videos are amazing

[@ucngominh3354]

hi

[@user-gu2fh4nr7h]

geefs , not jiffs

[@xXMaDGaMeR]

ok visualize transforms next please, Vision Transforms would be nice.