Ali Ghodsi, Lec [2,1]: Deep Learning, Regularization

[@Nakameguro97]

Excellent exposition! This is the first place I have found on YT that explains (through a derivation) why regularization terms are ADDED to the objective function.

[@quirkyquerty]

starts at 9.47

[@siddharthsvnit]

10:05 start here

[@sobitregmi31]

At 34:23 why does expectation drops while summing over m points

[@alisadeghi1370]

ممنونم استاد

[@nazhou7073]

thanks very very much, professor

[@sachinvernekar6711]

time: 33:26 How can co-variance be 0?

[@ivishal1990]

Covariance of independent terms is 0. Because the expected valueof XY i.e. E[XY]=E[X] * E[Y] if X and Y are independent. You can see it by looking at the formula of covariance and it gets zero. Intuitively covariance measures how 2 random variables effect each other(in a broad sense) and if they are independent then it becomes 0... Hope that helps...

[@sachinvernekar6711]

co-variance of independent variables = E[(X-mean(x))(Y-mean(y))] will be zero. Point to note is at 33:26, the equation is : E[(y0 - f) (f^ - f)]. Here f is not mean(y0) and f is not mean(f^), hence can't be 0.

[@sachinvernekar6711]

But the equation is not exactly co-variance. If you are convinced about it being 0, could you please post the solution?

[@priyamdey3298]

@@sachinvernekar6711 E[(yo - fo)(fo_hat - fo)] = E[yo*fo_hat] - E[yo*fo] - E[fo*fo_hat] + E[fo*fo] 1st term: yo* E[fo_hat] = yo*fo (bcoz yo is a constant, and expected value of fo_hat should be fo) 2nd term: E[yo*fo] = yo*fo (both are determinstic, not random) 3rd term: E[fo*fo_hat] = fo*E[fo_hat] = fo*fo 4th term: E[fo*fo] = fo*fo 1st term cancels with 2nd, 3rd one cancels with 4th = 0

[@ShahFahad-ez1cm]

watched for 41 mins but havent understood the motive of this lecture except constant derivations

[@charliean9237]

CS prof trying to do some stats... :P