2. Overfitting and underfitting (2/2)

[@ankoosh]

Preparing for my end semester paper on Machine Learning and I really have to say, this whole playlist is awesome. Thank you for explaining each and every thing in lucid language.

[@LuisRIzquierdo]

Thank you so much for such a nice comment. I'm very glad you found them useful. You made my day!

[@lazytocook]

why would u expect the test error to be very high when the variance is high and the training error is low?

[@LuisRIzquierdo]

The short answer would be that variance is a component of the test error, so when variance is high, generally you can expect the test error to be high. A more elaborate, intuitive and informal explanation is the following. Recall that if the variance is high, this means that our fit is very sensitive to small fluctuations in the training set. This means that if we changed the training set, we would generally get a very different model that would predict differently for any specific instance. Variance in this context is actually the variance of our estimations. If we have high variance, our estimations (for the same input) vary a lot if we change the training set. And any of these training sets is, in principle, equally valid, so any of our (different) estimations are equally legitimate... but they are widely different. This suggests high test error.