Visualizing activations with forward hooks (PyTorch)

[@amerel-samman9929]

Only person to explain it properly. Make more videos please!

[@mildlyoverfitted]

Thanks for your support:) Planning to make these videos every week.

[@mohitlamba117]

Wonderful! I have gone through Hooks method by fastai and thought that's the best explanation. I was wrong. This one is.

[@mildlyoverfitted]

Glad it was helpful!

[@haswanthaekula7656]

Excellent video!! Never knew you can use forward hooks this way. Keep these unique and awesome videos coming.

[@mildlyoverfitted]

Thank you! Appreciate it:)

[@ayankashyap5379]

I love your channel, its exactly what I was looking for!

[@mildlyoverfitted]

Thank you! Glad you like it:)

[@vijayabhaskar-j]

Great video! Glad someone uploaded unique content.

[@mildlyoverfitted]

Thank you! Appreciate it:)

[@vishalgoklani]

thanks for sharing this video, it was very useful! It would also be useful to make a video on the register_backward_hook() so we could visualize gradient flow.

[@mildlyoverfitted]

Thank you! Great idea!

[@markomitrovic4925]

Thank you for the video, very helpful!

[@mildlyoverfitted]

Happy you found it helpful!

[@tirthasg]

What font, and color theme are you using? Looks really nice!

[@aarshchaube1705]

I'm actually running out of ram when I try to get activations , ofc the number of samples is higher and so is the image size(224 x 224). Is there any efficient way ?

[@mildlyoverfitted]

Interesting, I never had to deal with out of memory issues in this context. Can you run a standard forward pass without any hooks? Just FYI, there is a new feature in TorchVision that can be used as an alternative to forward hooks: pytorch.org/blog/FX-feature-extraction-torchvision/

[@TheGanner1]

thank you so much

[@amitpandita8971]

Liked and attached hook to your videos ;)

[@mildlyoverfitted]

Awesome thank you!

[@talha_anwar]

How to analyze these histogram

[@mildlyoverfitted]

Great question! Personally, I use them to verify that the distribution of activations is reasonable. Let's say centered around zero and with some reasonable standard deviation. You can also create histograms of gradients to get an idea of whether the network is still learning or whether it is stuck.

[@vishalgoklani]

@@mildlyoverfitted It would be useful to have a video showing the histogram of gradients so we could directly measure learning.

[@jimmiemunyi]

Thank you!

[@mildlyoverfitted]

You're welcome!