Linear discriminant analysis (LDA) - simply explained

[@hopelesssuprem1867]

the best explanation in all internet. Thank you a lot

[@nikeforo2612]

Thanks for sharing. This is amazingly crafted and easy to follow. I finally understood the math behind LDA. Brilliant!

[@tilestats]

Thank you! That's great to hear.

[@快來快]

I like your video always with subtitles so i can better understand what are you saying

[@ratnakarbachu2954]

really you are genius and your help always remembered until my last breath.

[@tilestats]

Thanks for your comment!

[@KayYesYouTuber]

Beautiful explanation. Thanks

[@gwendolyneortiz4152]

I went to the MANOVA video, and it said I needed to first understand the LDA video so now I am here, and now it says I have to go to the PCA video hehe

[@mrgomelonsolaris]

Very clear explanation!

[@maydin34]

Great explanation.

[@44hyt]

Excellent video!!!

[@danialb9894]

Thank you for the video. Could you please make an extended video for LDA? I mean videos like what you did for PCA

[@tilestats]

Have you seen this one? kzbin.info/www/bejne/b4TYeniVqax5Y8k

[@rekhapriya2178]

Your videos have been amazing, can you please upload content related to Bayesian analysis

[@saifh.al-nimer2887]

Thank you so much for this very useful video.

[@tilestats]

Thank you!

[@spp626]

Hello sir, can you please explain why first eigen vector is considered for LDA?

[@hongdao9349]

How to calculate score from to matrix

[@silesoul]

Hello, thank you for the excellent explanation! So a variable contributes more to the groups' separation when it has a high weight. How do we interpret the negative weights of the variables (in LDA)?

[@tilestats]

Hi Use the absolute values for interpretation, which means that values far away from zero are important for separation.

[@silesoul]

@@tilestats Thank you !

[@MariaMartinezGarcia-ky6ru]

I greatly appreciate the video. I only have one question: Does this linear discriminant analysis approach rely on the Bayesian, Fisher or some other approach?

[@tilestats]

Well, the original discriminant analysis method was developed by Fisher, but note that LDA is not used to make any inference about a population (no p-value is calculated).

[@MariaMartinezGarcia-ky6ru]

@@tilestats Thank you for your response. That being said, I would like to know to which specific category or approach the method presented in the video could be associated. This is considering that its primary function is to classify known data, without conducting broader inferential analyses. Could you provide additional information on how this particular approach would be classified?

[@tilestats]

I would say that it is a classification method. Have a look at this video where I group LDA with other methods. kzbin.info/www/bejne/jn69kGOXodSJd8U

[@ritiksuri7248]

Can you please provide the dataset on which you worked

[@tilestats]

The data set is the same as shown in the video.

[@paolopanlaqui9887]

Hello! I would like to ask what the null hypothesis for LDA is or DA (Discrimination Analysis) in general? And also if you would know what the hypothesis is when MANOVA and DA together is used?

[@tilestats]

LDA or DA has no null hypothesis because it does not compute any p-value, in comparison to MANOVA. When you use LDA you can use some sort of validation method to check its performance. Have you seen my second video about LDA and the video about MANOVA?

[@sergeypigida4834]

Could you please check if the legend colors for bacterial and viral are correct on the figures (e.g. at 6:21)? With scikit-learn I get low values for bacterial and high values for viral data transformed samples. Therefore, viral data points, transformed with LDA, should be above the bacterial ones.

[@tilestats]

What values did you get for alpha 1 and alpha 2, using scikit?

[@sergeypigida4834]

@@tilestats [-0.3627029 -2.3969302] import pandas as pd from sklearn.discriminant_analysis import LinearDiscriminantAnalysis import matplotlib.pyplot as plt data = pd.DataFrame({ "infection":["Viral","Viral","Viral","Viral","Viral","Viral", "Bacterial","Bacterial","Bacterial","Bacterial","Bacterial","Bacterial"], "crp":[40.,11.1,30.,21.4,10.7,3.4,42.0,31.1,50.,60.4,45.7,17.3], "temp":[36.,37.2,36.5,39.4,39.6,40.7,37.6,42.2,38.5,39.4,38.6,42.7]}) X, y = data[["crp","temp"]], data["infection"].ravel() lda = LinearDiscriminantAnalysis(n_components=1, store_covariance=True) X_t = lda.fit(X, y).transform(X) colors = ["red", "green"] plt.figure() for color, cls in zip(colors, lda.classes_): plt.scatter(["LDA"]*len(X_t[y==cls]), X_t[y==cls], alpha=0.8, color=color, label=cls) plt.legend() plt.show()

[@sergeypigida4834]

@@tilestats Hi Andreas, I managed to get the same picture as you. To do this I used the eigen solver (by default sklearn uses singular value decomposition). lda = LinearDiscriminantAnalysis(solver="eigen", shrinkage=0.003, n_components=1, store_covariance=True) Thank you for your

[@tilestats]

Great!

[@iwwyl847]

Hello! great video, however, I have a question/need clarification: is LD1 found at 18:19 the actual line, on which when projected upon best seperates the two classes? And then the following calculations in the video is the data being projected on the line, or is this line found somewhere else? So to clarify; im looking specifically for the line that best seperates the two classes

[@tilestats]

The discriminant axis is given by the first eigenvector. If you like to plot the line on the centered data (unrotated data), plot the first eigenvector (0.15 0.989 or 0.11 0.7, both will give the same slope/direction of the line). Data points should project on this line.

[@ahmadalmomani1553]

Thank you very much for these beneficial videos. However, you mentioned that you used software to calculate the Eigenvectors. Please recommend which software to use (the simpler, the better)? Again, thanks a lot for your videos

[@tilestats]

Hi I use R, which is free! You can compute the eigenvectors in R by: mat=matrix(c(2.05,14.15,0.16,0.96),2,2) eigen(mat) Note that I also have videos on how to understand and calculate the eigenvectors by hand: kzbin.info/www/bejne/b3S3YZ2kmtJnrK8 kzbin.info/www/bejne/gKXGf5hjYsumr6M

[@jimjohnson357]

In your calculation of matrix W, when the sizes of the classes or groups is UNEQUAL, what are the variables n_1 and n_2? (i.e. in the equation shown at 15:43). Also, thanks so much for these videos (and the PCA ones). Well explained, with good examples and you did it in half the time everyone else takes!

[@tilestats]

Thank you! n1 and n2 are the sample sizes of group 1 (group A) and group 2 (group B). In this example, you should set n1=6 and n2=6. Note that you should use the same formula at 19:22 if you like to normalize the weights when you have an unequal sample size.

[@jimjohnson357]

@@tilestats Thanks very much... you guessed what my second question was going to be and answered it too! Looking forward to working my way through all your videos when I have some extra time.

[@NN-if9lr]

@@tilestats what if we have 3 or more classes? should the denumerator still n1+n2 - 2? or it change to n1+n2 - 3 and so on?

[@tilestats]

@@NN-if9lr Yes but n1+n2+n3-3

[@NN-if9lr]

@@tilestats oh i forgot the n3, thank you for the reply.

[@upanshisharma2313]

hey can you tell me how you calculated 0.11 and 0.70?

[@andreasjansson2366]

It is explained later on in the video…

[@ig55549]

Thanks! This video is so helpful! Coffee on me :)

[@tilestats]

Thank you!

[@sunitharamcse]

How you assign alpha1 and alpha2 value?

[@tilestats]

Do you mean how they are calculated? That is explained from 11:30.

[@azizullah6360]

how you computed alpha1 and alpha2?

[@tilestats]

That is explained from 14:40 and forward.

[@morecharacterswithamix9067]

The visuals and the commentary are not in sync.

[@azizullah6360]

how to calculate covariance matrix of W?

[@tilestats]

Can you refer to the time point in the video?