This is the best explanation of gmm

i am watching this in 2025, and this is super helpful, thank you for making this channel!

Forgot this was an old video

very good explanation! thank you

Sir, if any vector is by default a column vector then the dot product equation should be "Theta Transpose x," which gives a scalar y; otherwise, it will give a matrix. I think it is a typo mistake. If your assumption is all are row vector, then it is fine.

Thank you for the knowledge!

Bit tough to follow without any visualiations the relation to k-means was intuitive, as Gaussian Mixture Models essentially group the inputs as being sampled from k number of gaussians... thanks

why f(x,b)=sign(x.x-b) is a circle?

Thank you! This video is so much better than all the garbage machine learning click bait videos.

at 11:00 what if the blue point is in between the two red points? I don't think a straight line can shatter them in that case. Or did I just misinterpret the entire thing?

Thanks for wonderful explanation Do you share slides?

great video even after 10 years! thanks! :)

By far the best explanation on GMM and specially the EM algorithm.

00:01 Gradient boosting is a method of converting a sequence of weak learners into a very complex predictor. 01:31 Gradient Boosting in a nutshell 02:56 Ensembling improves predictions by combining weak learners 04:30 Ensembles in Gradient Boosting add complexity gradually for better fit 05:52 Gradient Boosting involves adjusting predictions to reduce error 07:14 Gradient Boosting involves fitting models to error residuals 08:56 Explaining the importance of step size alpha K in Gradient Boosting 10:20 Gradient boosting uses weighted sums of regressors for better predictions. Crafted by Merlin AI.

Thank you for this video, helps alot!

Very clear and straightforward while containing all the necessary contents to understand the concept!

Thank you so much so much sooooo much

4:07 MLE for sigma-hat should be X by X-transpose (outer product) not X-transpose by X (inner product)

Oh my G. After 5 years of confusion, I finally understood Lp regularization! Thank you so much Alex!

This is not a very good explanation at all. There's WAY too much theorem dumping with difficult-to-parse variables all over the place, and a big lack of tangible examples. I don't know what other people see in this video.

Exceptionally clear explanation of the use of EM with Gaussian Mixture Models

Nice video , this is what I dig in youtube , an actual concise clear explanation worth any paid course

Thanks :)

学习

At 4:32, if x and μ are row vectors, [x-μ] should also be a row vector. Then how to multiply (Σ^(-1))* [x-μ]? Since the dimension of (Σ^(-1)) is 2*2, and the dimension of [x-μ] is 1*2.

The best explanation I ever see. Hope can talk a bit about how to derive the equation.

Prof. Ihler, could you provide the reference for hard EM (in the last slide)? Thx!

thanks for the explanation

Bootstrap-Aggregation. So helpful when you're trying to learn every AWS Machine Learning tool!

Very good nuance here.

excellent presentation

please publish more videos, professor Ihler!

Very few videos online give some key concepts here, like what we're truly trying to minimize with the penalty expression. Most just give the equation but never explain the intuition behind L1 and L2. Kudos man

Very well explained. Thank you!

Thank you very much for the amazing video! I have a couple of questions. 1. Would shatter be the right criteria to define VC dimension when classifying the data to the right class is also important? As an example of the circle classifier example, you will be able to shatter two points but you will not be able to classify the two points correctly as shown in the video. 2. One parameter with lots of power: Would it be possible to share a complete example for this case or any reference literature if you have any?

Excellent explanation!! (one correction for the mistake: the outer product was written as the inner product in the slides

Beautiful!

This video was so hard to follow and watch (too wordy and very little pauses) but I’m thankful anyway, since eventually I got to understand it by thinking about it and rewatching a few times.

Simply best explanation!!

Beautiful

Next time, I'd love it if you included the effect lambda has on regularization, including visuals!

Wow, that was such a great explanation. Thank you.

Hi Alexander Ihler, i watched plenty of videos, and this explanation is the best I found. Thank you!

Could you explain more about the sum of the vectors in your notations for the maximum likelihood estimates at the minute 1.45? As far as I have noticed, there has been only one data set, namely one x vector. Thus, what actually are you summing up with j indices? Cheers.

At the minute of 1.34, the maximum likelihood estimates formula has 1 over N coefficient. On the other hand, at the minute of 3.13, there is 1 over m coefficients. We know that N and m is the total number of values in the sums, but what is the reason you used different notations as N and m. Is it just to seperate univariate and multivariate cases while they keep their definitions (or meaning)? Also, the j values in the lower and upper limits of sum sembols are not so clear in this notation. Should we write j=1 to j=m or N for instance?

One more question about the example at the minute of 4.24, you said independent x1 and x2 variables. Independendent of what??? As far as I see, you can have 2 univariate formula like in this example, but when you combine them to see the combined likelihood, you have to have a mean vector in size of 2 and Sigma matrix iin size of 2x2. That's always the case, right? The size of the mean vector and the Sigma matrix look like defined by the number of combination of x values. Is that right? I saw another example somewhere else, you can have L(μ=28 ,σ=2 | x1=32 and x2=34) for instance to find the combined likelihood at x1=32 and x2=34, and he uses only one mean and sigma for both. REF:kzbin.info/www/bejne/ep-Zk2yceK6Ipq8&ab_channel=StatQuestwithJoshStarmer

I'd like to know how you call your x value for univariate caseü or x value set for multivariate case in your Gaussian distribuitons? Do you name them as "data set" or " variable set"? Also, what makes the mean value size same as the x data size? Thanks in advance. Should we think that we create one mean average for every added x data point in our data set? That's why we average them when we find the best estimated value in the end.

In the formula at the minute 2.11, when you find the inverse of a Sigma matrix in the exp(...) , do you use unit matrix method, any coding , or some other method? Cheers.

At 5.23, you should have said (x-mu) transpose.

This should be mandatory viewing, before being assaulted with the symbolic derivations!