Here is how Transformers ended the tradition of Inductive Bias in Neural Nets

[@hieunguyentranchi947]

This is gold I hope it gets the ATTENTION it deserves

[@avb_fj]

Thanks!! More attention will surely TRANSFORM this channel! 😂

[@TP-ct7qm]

Awesome video! This (together with the last two videos) is one of the best explanations of Transformers I've seen. Thanks and keep it up!

[@karthikdulam9830]

This video needs to be more popular!

[@AI_ML_DL_LLM]

the gist of this video: 4:29, a great job, thanks

[@JasFox420]

Dude, you are a treasure, keep it up!

[@amoghjain]

wowww! what a great explanation! helps knit so many individual concepts together in one cohesive knowledge base!! thanks a lot for making this video and all the animations!

[@avb_fj]

Thanks!!

[@ZenTheSapo]

Incredible series, helped me a lot.

[@IdPreferNot1]

Came here after interest from one blue three brown. It's clear you've got a great explanation style... plus you were earlier ;). Hope your channel following builds to match your outstanding quality.

[@avb_fj]

Welcome! Thanks a lot for the shoutout!

[@matiasalonso6430]

Congrats !! Awesome channel !

[@avb_fj]

Thanks!

[@SuebpongPruttipattanapong]

Thank you so much!

[@GabrielAnguitaVeas]

Thank you!

[@user-yr9sf2yr3n]

great, great video.

[@sahhaf1234]

I really dont like to leave you a like. Instead, I want to leave you one hundred likes.. Unfortunately google limits me to one...

[@avb_fj]

Thanks!! Super appreciated!

[@user-wm8xr4bz3b]

at 2:33, you mentioned that self-attention is more biased, but at 2:54 you also mentioned that self-attention reduces inductive bias?? Sorry but i'm a bit confused.

[@avb_fj]

Self-Attention indeed reduces inductive bias and adopts a more general learning framework. At 2:33, I am asking a question: "IS Self-Attention more general or more biased?" And then I continue with "I'll argue that Self-Attention is not only more general than CNNs and RNNs but even more general than MLP layers".

[@axe863]

Should have a hybrid structure ...... 😅

[@avb_fj]

This is true. Many computer vision research has now moved into adding elements of CNNs/inductive bias into the Vision Transformer architecture, for example - Swin Transformers.