UMAP: Mathematical Details (clearly explained!!!)

[@statquest]

To learn more about Lightning: github.com/PyTorchLightning/pytorch-lightning To learn more about Grid: www.grid.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@mohamedarebi7419]

Can't wait to get all the nitty gritty details :)

[@statquest]

BAM! :)

[@hansenmarc]

I have to agree with you. Spectral embedding does sound very cool!

[@statquest]

bam!

[@henriquefantinatti4601]

Please, make videos about time series. Would be amazing seeing you talk about it. Love the videos!

[@statquest]

I'll keep that in mind.

[@nourelislam8565]

Amazing!.... we actually need a video explaining the different normalization methods used in scRNA seq analysis especially SCTransform.. Appreciate your support, Thanks

[@statquest]

Thank you!

[@dataanalyticswithmichael8931]

Finally, i like mathematical things and this video popped up in my recomendation

[@statquest]

Hooray!

[@mohamedamr8081]

Hey Josh, thanks for the videos and the great content! I was wondering if you can make a video about causal inference, that would be great (for me lol), thanks.

[@statquest]

I'll keep that in mind.

[@utkarshtrehan9128]

Level: God Level! Could we have a video on Factor Analysis!

[@statquest]

I'll keep that in mind! :)

[@venkateshmunagala205]

Great Video. Thanks Josh

[@statquest]

Thanks!

[@mariapelaez3758]

Hi Josh, I saw both of your videos for UMAP, but I have a doubt regarding how you did the low-dimension graph. I knw you mention spectral embedding an what I know from that is that you calculate the Laplacian of the graph and then get the eigenvalues and eigenvector and the coordinates of the new dimension will be the values of the eigenvector for the lowest eigenvalue (ignoring zero eigenvalues). But, when I try that for your data, I am not able to get the values that you showed, so, I wanted to know if you did something different. Also I wanted to know if for more than 1 dimension, I will use more than 1 eigenvector right? For the 2d case the x-y coordinates will be the values of the first and second eigenvectors of the lowest eigenvalues. Thanks

[@statquest]

To be honest, I just drew the low-dimensional graph in a way that I thought would best highlight how UMAP works, rather than stay faithful to how spectral embedding would have projected the points. In other words, I completely ignored spectral embedding when I drew the low-dimensional graph and only took pedagogical aspects into consideration. I'm sorry if this caused confusion. :(

[@AHMADKELIX]

Hi Josh, thank for your explained

[@statquest]

My pleasure!

[@cahbe6108]

Thank you for the great video! One question: What happens if there are multiple closest neighbors with the same distance? Then there will be multiple similarity scores = 1. Then changing sigma might not help to get close to log2(num_neighbors) for the sum of similarities.

[@statquest]

I'm not sure what the technical details are exactly, but I would guess it simply finds the value for sigma that gets the sum closest to the ideal value. It doesn't have to be exact.

[@amalnasir9940]

Hi Josh, Spectral clustering is an interesting topic. I hope you cover it one day!

[@statquest]

I'll keep that in mind! :)

[@QwakeRunner]

Hi Josh! would you mind showing us how this can be done via Python! Would be very happy to buy a template script from you and support your great work!

[@statquest]

That's a good idea and I'll keep that in mind.

[@anitat9727]

@@statquest I'd love this too

[@AyushSharma-tg5co]

sir I'm looking forward to the data analyst domain, so pls tell me which playlist of yours I should follow to get the stats part done?

[@statquest]

You can just start at the top of this page and work your way down: statquest.org/video-index/

[@ass_im_aermel]

Hi Josh! Is it possible, that you make some videos for time series related topics like: serial correlation or the box jenkins method? And thank you for all the videos you made in the last years. They are awesome :-)

[@statquest]

I'll keep those topics in mind.

[@ieserbes]

Hello Josh, Amazing explanation. Thank you so much. Where does 2.1 (lower dimensional distance between a and b ) come from? I am a bit lost there.

[@statquest]

If you look at the number line that the points are on, you'll see that point 'b' is is at about 1.8 and point 'a' is at about 3.9. Now we just do the math: 3.9 - 1.8 = 2.1. Bam.

[@ieserbes]

@@statquest Hooray 😄 Thank you Josh.

[@grinps]

waiting for the BAM!

[@statquest]

BAM! :)

[@xavisolersanchis7145]

Hi, thank you very much! Just one question. In your case you could compute the initial distances between the data points as euclidean distances because you are only working wuth two features. How are they computed when you have much more features, do you always start with euclidean distances??

[@statquest]

The Euclidean distance works for more than 2 features, en.wikipedia.org/wiki/Euclidean_distance so there's no problem adding more features. That said, if you wanted to use a different distance metric, it would probably be OK.

[@xavisolersanchis7145]

Thank you very much! I also realized there is a little error in the video. It is the part that you say the result seems strange to you. To compute the Symmetrical Score they don't take what you call the "Similarity scores". What they do is to iterate over the y nearest neighbours and compute the distance between x and y as the maximum value between 0 and the distance between x and y minus the distance to the nearest neighbour of x; all of this divided by the previously learnt sigmas. Then they exp-1 this dist to get a Similarity score between x and y (saved as the probability of this element in the fuzzy set). Once you have all this fuzzy set you apply the t-conorm you mention over this Similarity scores to have the Symmetrical score. I hope this is helpful to you, and I also hope not being wrong hehe. Thank you very much!@@statquest

[@Pedritox0953]

Great video!

[@statquest]

Thanks!

[@alexmiller3260]

Hey, Josh! Can you make video(s) about likelihood and MLE for an unknown distribution, if it can't be easily approximated or it's impossible to approximate? Because everyone talks about well-known distributions, but literally nothing about working with something unknown. Of course it would be better, if you decided to make a playlist with everything about unknown distributions, but couple of videos is also OK

[@statquest]

I'll talk about this topic when we cover Bayesian statistics. That said, even when the distribution is unknown, the central limit theorem ( kzbin.info/www/bejne/j3LPe3Z7ea1lq7s ) results in a known (gaussian) distribution. And that means it doesn't matter what the original distribution is.

[@ouryly1541]

Amazing!! Thank you

[@statquest]

Thank you too!

[@conlele350]

Hi Josh, could you please take some time to give some explanation about Markov chain decision process and it application in ML and how do we code it in R. Thanks

[@statquest]

I'll keep those topics in mind.

[@yeah6732]

Thank you!!

[@statquest]

bam! :)

[@3Mus-cat-tears]

Hi Josh! Thank you for the info! It's really helpful. What to do if I have zeros or NAs in my dataset? I couldnt find anything on imputation before UMAP on Google :(

[@statquest]

there might not be a UMAP specific imputation method, so if you just search for imputation methods in general, you might find something that works.

[@lukesimpson1507]

Hi Josh I was wondering how you feel about using some stills from your channel to explain these types of plots prior to displaying them? This would be done I'm an educational setting and I would credit the channel and provide a link if that is okay?

[@statquest]

As long as you provide the link, it's fine with me.

[@lukesimpson1507]

@@statquest Thank you! Keep up the great videos!

[@ChocolateMilkCultLeader]

The day you do Hilbert Curves will be a lot of BAMMMMMSSSSSSS

[@statquest]

I'll keep that in mind! :)

[@muriloaraujosouza462]

In "making the scores symmetrical" part, the operation is done even if the scores are already symmetrical. For example, the score from A->B was the same as the score from B->A. And the score from A->C was the same as the score from C->A. Any ideas why it is necessary to perform this operation in already symmetrical scores?

[@statquest]

Probably not and it is possible that efficient implementations avoid the extra math.

[@miguelcampos867]

Would be great talking about normalizing flows

[@statquest]

I'll keep that in mind.

[@miguelcampos867]

@@statquest thanks. In fact that’s the main reason of why I am watching all your videos of likelihoods, gaussian distribution etc jajaja (they are great btw)

[@harryhamjaya]

Hello could you bring Spectral Embedding topic, and I have a question regarding UMAP how is it possible that UMAP is faster than T-SNE, in which part does it beat the t-sne? Since logically T-SNE moves more point in a time compared to UMAP(?) Correct Me If I Wrong and BTW we are hoping the new updates ❤

[@statquest]

I talk about why UMAP is faster than t-SNE in my other UMAP video here: kzbin.info/www/bejne/m3-TqHmwd6ZnicU

[@markmalkowski3695]

Excellent! (mr burns style)

[@statquest]

Thank you!

[@meichendong3434]

Love it!

[@statquest]

Thank you!

[@veenakumar2384]

how do i know which is the best curve plotted against similarity scores (Y axis) and the cluster points wrt distances (X axis)?

[@statquest]

When the sum of the similarity scores = log(num_nearest neighbors).

[@buffetCodes]

BAM!

[@statquest]

:)

[@naman4067]

Nice

[@statquest]

:)

[@annaluizavicente]

Hi!! Could you, please, make a video about PLSDA? Thanks

[@statquest]

I'll keep that in mind.

[@annaluizavicente]

@@statquest Thanks! You are the best

[@ridwanwase7444]

Hey,do you know pranab k sen? He is Bengali and im Bengali too! Im an undergrad of statistics

[@statquest]

Nice!

[@THEMATT222]

Noice 👍

[@statquest]

:)

[@THEMATT222]

@@statquest :D