The Battle of Polynomials | Towards Bayesian Regression

In this tutorial, I explain the process of building models to fit a dataset using various degrees of polynomials.
I then compare the predictions made by these various models and investigate why a more capable model (one with a higher degree polynomial) overfitted and explain the various remedies to tackle it by making use of more training data points and performing regularization.
The tutorial provides the various derivations & mathematical justifications behind the usage of the error function (sum squared) but then argues that this view of regression is very limiting and if there is an alternative view to modeling that explains these formulations better.
#
PRML - (available for free, thanks to Dr. Bishop) - www.microsoft....
The (python) source code used to reproduce the experiment from the book can be found here in the form of colab/jupyter notebook -
colab.research...
or, for read-only view see this
ksachdeva.gith...
#linearregression
#ridgeregression
#overfitting

[@bharadwajreddy7840]

This is an amazing series, please continue doing videos for rest of "pattern recognition and machine learning" book

[@KapilSachdeva]

Will do.

[@KapilSachdeva]

Feeling guilty now :( …. Apologies for the delay.

[@KaranYadav-hw9yo]

Yeah..this is really helpful...I was struggling to get sense of these equations but then this video has very systematically explained the concepts!!!

[@KapilSachdeva]

@@KaranYadav-hw9yo thanks 🙏… glad you found it helpful!

[@Pruthvikajaykumar]

This channel is gold mine seriously. Thank you Sir!

[@KapilSachdeva]

🙏

[@adesiph.d.journal461]

Please do more from PRML! A whole generation of us need it :) Thank you!

[@KapilSachdeva]

Do you mean beyond Bayesian Regression and Inference? … I am trying to base most of the tutorials in this series using examples and plots from PRML provided it makes sense. For example, the latest (Markov Chains) in PRML is not covered to my satisfaction.

[@adesiph.d.journal461]

@@KapilSachdeva So here is how I see it, A lot of Advanced ML courses abroad require you to have prerequisites in the form of PRML or the MML book. At least if someone wants to understand research papers a lot of them assume students know chapters from PRML like Kernel Methods and Approximate Inference (which become important in Generative models, VAEs). Your videos are super nice so if you are working on a series on these lines I believe undergraduates can start understanding the subject in more depth and also start contributing and try their hand in research. Definitely, if you feel find other books to make more sense for a particular topic that is the way forward, but its just the confidence it gives students knowing they covered the prerequisite book with the reputation of PRML feels good.

[@KapilSachdeva]

All fair points. Thanks, your suggestion makes lot of sense and see the value in it. Let me ponder over it on how to execute on this great suggestion ! 🙏

[@adesiph.d.journal461]

@@KapilSachdeva Thank you! Really looking forward!

[@KapilSachdeva]

🙏

[@gender121]

Really a treasure of knowledge expressed in very simple way….a help to learn Bayesian treatment. Please complete the series and more videos are expected.

[@KapilSachdeva]

🙏

[@mathewjones8891]

This was a wonderful tutorial. Your graphics, style, clarity and humour are very helpful. Looking forward to the rest of the series. Specifically, I'm interested in using Bayesian prediction on my own data.

[@KapilSachdeva]

🙏

[@johnjohnston3815]

I just found your channel today and I am glad I did! Great stuff

[@zgbjnnw9306]

This video is so helpful for me after I finished my regression class. I didn't under the overfitting concept. Now I do! Thank you!

[@KapilSachdeva]

🙏

[@hariharanvenkatraman3074]

such a gem of the playlist. Every topics are explained clearly. Please do more videos with the same book❤❤❤

[@KapilSachdeva]

🙏

[@sanjaythorat]

Very well explained. Please keep up the good work!

[@KapilSachdeva]

🙏

[@Dev-zr8si]

This is amazing, I hope you make more. Thank you.

[@KapilSachdeva]

🙏… there are 10 more videos in this playlist called “Towards Bayesian Regression”

[@sertacderya3775]

Very helpful videos, thank you very much!

[@KapilSachdeva]

🙏

[@unimatorx]

That is super useful! Fantastic work :)

[@KapilSachdeva]

🙏

[@cerioscha]

Great video, thanks !

[@KapilSachdeva]

🙏

[@aarjavik123]

Thanks for excellent lecture

[@KapilSachdeva]

🙏

[@drewburns7805]

Thank you. Very helpful! But I think there is a mistake in saying the weights vector is a row vector. I believe it should be a column vector for t = Xw

[@KapilSachdeva]

🙏 thanks for pointing it out. You are correct. I will add a note in the description.

[@gender121]

Can you please guide me whether weight vector is column vector or row vector. It is creating confusion in multiplication. Thanks in advance for the great series.

[@medihazukic5382]

Amazing lectures, thank you so much! I was wondering at 12:07 the partial derivatives of E wrt w_i (in the right panel), shouldn't those be the partial derivatives of ys and not Es?

[@KapilSachdeva]

No it is E, the error function. Our goal is to minimize the error.

[@ankurgupta8060]

How to create this type of video. Just curious its spot on with black background and animation is enough to serve the purpose. Any lead is appreciated 👍. Thanks for the video and the content.

[@KapilSachdeva]

🙏 PowerPoint with black theme and morph transitions and animations. Not much more than that.

[@KapilSachdeva]

@@hyperadapted sometimes in latex but mostly in PowerPoint :)