Linear discriminant analysis (LDA)

Linear discriminant analysis (LDA) - simply explained

Рет қаралды 52,736

Күн бұрын

See all my videos at www.tilestats.com/
In this video, we will see how we can use LDA to combine variables to predict if someone has a viral or bacterial infection. We will also compare LDA and PCA (07:50), discuss separation (11:50), the math behind LDA (14:40), and how to calculate the standardized coefficients (21:00).

Пікірлер: 54

@hopelesssuprem1867 6 ай бұрын

the best explanation in all internet. Thank you a lot

@nikeforo2612 3 жыл бұрын

Thanks for sharing. This is amazingly crafted and easy to follow. I finally understood the math behind LDA. Brilliant!

@tilestats 3 жыл бұрын

Thank you! That's great to hear.

@mrgomelonsolaris 2 жыл бұрын

Very clear explanation!

@KayYesYouTuber 7 ай бұрын

Beautiful explanation. Thanks

@44hyt Жыл бұрын

Excellent video!!!

@maydin34 2 жыл бұрын

Great explanation.

@ratnakarbachu2954 2 жыл бұрын

really you are genius and your help always remembered until my last breath.

@tilestats 2 жыл бұрын

Thanks for your comment!

@user-wt6oe8nm7y 5 ай бұрын

I like your video always with subtitles so i can better understand what are you saying

@saifh.al-nimer2887 2 жыл бұрын

Thank you so much for this very useful video.

@tilestats 2 жыл бұрын

Thank you!

@gwendolyneortiz4152 5 ай бұрын

I went to the MANOVA video, and it said I needed to first understand the LDA video so now I am here, and now it says I have to go to the PCA video hehe

@rekhapriya2178 7 ай бұрын

Your videos have been amazing, can you please upload content related to Bayesian analysis

@iwwyl847 2 жыл бұрын

Hello! great video, however, I have a question/need clarification: is LD1 found at 18:19 the actual line, on which when projected upon best seperates the two classes? And then the following calculations in the video is the data being projected on the line, or is this line found somewhere else? So to clarify; im looking specifically for the line that best seperates the two classes

@tilestats 2 жыл бұрын

The discriminant axis is given by the first eigenvector. If you like to plot the line on the centered data (unrotated data), plot the first eigenvector (0.15 0.989 or 0.11 0.7, both will give the same slope/direction of the line). Data points should project on this line.

@paolopanlaqui9887 2 жыл бұрын

Hello! I would like to ask what the null hypothesis for LDA is or DA (Discrimination Analysis) in general? And also if you would know what the hypothesis is when MANOVA and DA together is used?

@tilestats 2 жыл бұрын

LDA or DA has no null hypothesis because it does not compute any p-value, in comparison to MANOVA. When you use LDA you can use some sort of validation method to check its performance. Have you seen my second video about LDA and the video about MANOVA?

@danialb9894 Жыл бұрын

Thank you for the video. Could you please make an extended video for LDA? I mean videos like what you did for PCA

@tilestats Жыл бұрын

Have you seen this one? kzbin.info/www/bejne/b4TYeniVqax5Y8k

@ahmadalmomani1553 Жыл бұрын

Thank you very much for these beneficial videos. However, you mentioned that you used software to calculate the Eigenvectors. Please recommend which software to use (the simpler, the better)? Again, thanks a lot for your videos

@tilestats Жыл бұрын

Hi I use R, which is free! You can compute the eigenvectors in R by: mat=matrix(c(2.05,14.15,0.16,0.96),2,2) eigen(mat) Note that I also have videos on how to understand and calculate the eigenvectors by hand: kzbin.info/www/bejne/b3S3YZ2kmtJnrK8 kzbin.info/www/bejne/gKXGf5hjYsumr6M

@spp626 4 ай бұрын

Hello sir, can you please explain why first eigen vector is considered for LDA?

@silesoul Жыл бұрын

Hello, thank you for the excellent explanation! So a variable contributes more to the groups' separation when it has a high weight. How do we interpret the negative weights of the variables (in LDA)?

@tilestats Жыл бұрын

Hi Use the absolute values for interpretation, which means that values far away from zero are important for separation.

@silesoul Жыл бұрын

@@tilestats Thank you !

@jimjohnson357 2 жыл бұрын

In your calculation of matrix W, when the sizes of the classes or groups is UNEQUAL, what are the variables n_1 and n_2? (i.e. in the equation shown at 15:43). Also, thanks so much for these videos (and the PCA ones). Well explained, with good examples and you did it in half the time everyone else takes!

@tilestats 2 жыл бұрын

Thank you! n1 and n2 are the sample sizes of group 1 (group A) and group 2 (group B). In this example, you should set n1=6 and n2=6. Note that you should use the same formula at 19:22 if you like to normalize the weights when you have an unequal sample size.

@jimjohnson357 2 жыл бұрын

@@tilestats Thanks very much... you guessed what my second question was going to be and answered it too! Looking forward to working my way through all your videos when I have some extra time.

@NN-if9lr 2 жыл бұрын

@@tilestats what if we have 3 or more classes? should the denumerator still n1+n2 - 2? or it change to n1+n2 - 3 and so on?

@tilestats 2 жыл бұрын

@@NN-if9lr Yes but n1+n2+n3-3

@NN-if9lr 2 жыл бұрын

@@tilestats oh i forgot the n3, thank you for the reply.

@sergeypigida4834 Жыл бұрын

Could you please check if the legend colors for bacterial and viral are correct on the figures (e.g. at 6:21)? With scikit-learn I get low values for bacterial and high values for viral data transformed samples. Therefore, viral data points, transformed with LDA, should be above the bacterial ones.

@tilestats Жыл бұрын

What values did you get for alpha 1 and alpha 2, using scikit?

@sergeypigida4834 Жыл бұрын

@@tilestats [-0.3627029 -2.3969302] import pandas as pd from sklearn.discriminant_analysis import LinearDiscriminantAnalysis import matplotlib.pyplot as plt data = pd.DataFrame({ "infection":["Viral","Viral","Viral","Viral","Viral","Viral", "Bacterial","Bacterial","Bacterial","Bacterial","Bacterial","Bacterial"], "crp":[40.,11.1,30.,21.4,10.7,3.4,42.0,31.1,50.,60.4,45.7,17.3], "temp":[36.,37.2,36.5,39.4,39.6,40.7,37.6,42.2,38.5,39.4,38.6,42.7]}) X, y = data[["crp","temp"]], data["infection"].ravel() lda = LinearDiscriminantAnalysis(n_components=1, store_covariance=True) X_t = lda.fit(X, y).transform(X) colors = ["red", "green"] plt.figure() for color, cls in zip(colors, lda.classes_): plt.scatter(["LDA"]*len(X_t[y==cls]), X_t[y==cls], alpha=0.8, color=color, label=cls) plt.legend() plt.show()

@sergeypigida4834 Жыл бұрын

@@tilestats Hi Andreas, I managed to get the same picture as you. To do this I used the eigen solver (by default sklearn uses singular value decomposition). lda = LinearDiscriminantAnalysis(solver="eigen", shrinkage=0.003, n_components=1, store_covariance=True) Thank you for your

@tilestats Жыл бұрын

Great!

@MariaMartinezGarcia-ky6ru 8 ай бұрын

I greatly appreciate the video. I only have one question: Does this linear discriminant analysis approach rely on the Bayesian, Fisher or some other approach?

@tilestats 8 ай бұрын

Well, the original discriminant analysis method was developed by Fisher, but note that LDA is not used to make any inference about a population (no p-value is calculated).

@MariaMartinezGarcia-ky6ru 8 ай бұрын

@@tilestats Thank you for your response. That being said, I would like to know to which specific category or approach the method presented in the video could be associated. This is considering that its primary function is to classify known data, without conducting broader inferential analyses. Could you provide additional information on how this particular approach would be classified?

@tilestats 8 ай бұрын

I would say that it is a classification method. Have a look at this video where I group LDA with other methods. kzbin.info/www/bejne/jn69kGOXodSJd8U