UMAP Dimension Reduction, Main Ideas!!!

[@statquest]

To learn more about Lightning: github.com/PyTorchLightning/pytorch-lightning To learn more about Grid: www.grid.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

[@EthanSalter3]

This is such perfect timing, I'm supposed to learn and perform a UMAP reduction tomorrow. Thank you!

[@statquest]

BAM! :)

[@Dominus_Ryder]

You should buy a couple of songs to really show your appreciation!

[@evatosco-herrera8978]

I just found this channel. I'm currently doing my PhD in Bioinformatics and this is helping me immensely to save a lot of time and to learn new methods faster and better (I have a graphical brain so :/) Thank you so much for this!!

[@statquest]

Good luck with your PhD! :)

[@codewithbrogs3809]

After three days of coming back to this video, I think I finally got it... Thanks Josh. When I'm in a place to support, I will

[@statquest]

Bam!

[@codewithbrogs3809]

DOUBLE BAM

[@aiexplainai2]

I can't appreciate how much this channel helped me - so clearly explained!!

[@statquest]

Thank you very much! :)

[@terezamiklosova104]

I really appreciated the UMAP vs t-SNE part. Thanks for the video! Really helpful when one tries to get the main idea behind all the math :)

[@statquest]

Thank you very much! :)

[@smallnon-codingrnabioinfor3792]

I totally agree! The part starting at 16'10 is worth to look at back! Thanks a lot for this great and simple explanation!

[@kennethm.4998]

Dude... Dude... You have a gift for explaining stats. Superb.

[@statquest]

Thank you!

[@JulietNovember9]

New StatQuest always gets me amped. High yield, low drag material!!!

[@statquest]

Awesome!!!

[@markmalkowski3695]

This is awesome, thanks for explaining UMAP so well, and clearly explaining when to use! Love the topics you’re covering

[@statquest]

Thank you!

[@abramcadabros1755]

Wowie, I can finally learn what UMAP stands for and how it reduces dimensionality AFTER I analysed my scRNA-seq data with it's help!

[@statquest]

BAM!

[@akashkewar]

Not sure if I can hold my breath for long enough before the video starts, Amazing work!! @StatQuest

[@statquest]

Thanks!!

[@offswitcher3159]

Great Video, Thank you! You are with me since first semester and I am so happy to see a video by you on a topic that is relevant to me

[@statquest]

Awesome!

[@user-hg4jk2q]

This will help me greatly for my MS project.

[@statquest]

Good luck!

[@dexterdev]

I was waiting for this. thank you. best dimensionally reduced visual explanation out there.

[@statquest]

Thank you very much! :)

[@VCC1316]

I'd love to see a cross-over episode between StatQuest and Casually Explained. Big bada-bam.

[@statquest]

:)

[@rajanalexander4949]

Great video; especially liked the echo on the full exposition of 'UMAP' 😂

[@statquest]

:)

[@saberkazeminasab6142]

Thanks so much for the great presentation!

[@statquest]

Glad you enjoyed it!

[@user-mv3im2fi4f]

Ele explica como se eu fosse uma acéfala. Só assim eu entendi, obrigada!

[@statquest]

Muito obrigado! :)

[@siphosakhemkhwanazi6042]

The intro made me to subscribe😂😂

[@statquest]

bam! :)

[@dataanalyticswithmichael8931]

Nice esplanation, i want to use this as my references for my projects

[@statquest]

Bam! :)

[@brucewayne6744]

Amazing video!! Hope there is a statquest on ICA coming soon :)

[@statquest]

One day...

[@meenak722]

Thank you very much!

[@statquest]

You're welcome!

[@agentgunnso]

Thank you so much!!! Love the sound effects and the jokes

[@statquest]

Glad you like them!

[@danli1863]

I must say this channel is amazing! I must say this channel is amazing! I must say this channel is amazing! Important things 3 times. :)

[@statquest]

TRIPLE BAM! :)

[@davidhodson6680]

Adding a comment for the cheery ukelele song at the start, I like it.

[@statquest]

Thank you! :)

[@THEMATT222]

New video!!!! Very Noice 👍

[@statquest]

BAM!!!

[@floopybits8037]

Thank you so much for this video

[@statquest]

Most welcome 😊!

[@shubhamtalks9718]

Yayy. I was waiting for it.

[@statquest]

bam!

[@samuelivannoya267]

You are amazing!! Thanks!!!

[@statquest]

Thank you!

[@emiyake]

PaCMAP dimension reduction explanation video would be very appreciated!

[@statquest]

I'll keep that in mind.

[@MegaNightdude]

Great content. As always!

[@statquest]

Thank you!

[@whitelady1063]

Best comment section in KZbin Also now I get why people on office won't stop praising you BAM!

[@statquest]

Thank you! :)

[@Pedritox0953]

Great video!

[@statquest]

Thanks!

[@gergerger53]

Great video (as always). You might want to calm it down with the BAMs though. It used to be quirky and fun but having them literally every minute or two is a bit much and forced. Your video creation skills are seriously awesome. I wish I had even half your skills at making these concepts accessible for the YT audience. 👏

[@statquest]

Noted

[@abdoualgerian5396]

With this amazing explanation way, please consider doing a Deep TDA quest starting with the paraparapepapara funny thing instead of the songs

[@statquest]

Noted

[@junaidbutt3000]

Hey Josh, Great work as always, this StatQuest came at a great time for me because I've been looking into UMAP myself. I had a few questions apologies if they're covered in the mathematical details video: 1. Is there an additional constraint for the curve used to compute the high dimensional similarity score to make the scores what they are? In the example where you computed the distance of points B and C relative to A, you had 1.0 and 0.6. This is because the scores must sum to 1.6. But why not 1.3 and 0.3 or 1.59 and 0.01? Is there an additional consideration which locks them to be 1.0 and 0.6? 2. Will there be an explanation about spectral embedding? This may be outside of the scope of the video but I thought I'd ask! 3. Could you please check my understanding for what is happening when we move point D closer to point E? The discussion starts at 14:48 in the video. As I understand it, moving D closer to E (we want this) also moves D closer to C (we don't want this). So we compute a tradeoff and find that the cost of moving D closer to C is lower than the benefit of moving D closer to E. Therefore we move D to E. Is this correct? If so, is there an equation or rule that allows us to quantify this such that we can determine the exact distance to move D closer to E? I suspect that most of these will be included in the mathematical details follow up video but I thought I'd ask just in case they aren't.

[@statquest]

1) You'll see the answer to this in the follow up video. However, to give you a head start - the similarity score for the closest point is always 1, and this limits what the score for the second point can be (since only have 2 points as neighbors). 2) Unfortunately I'm not going to dive into spectral embedding (not yet at least!) 3) You're understanding is correct and you'll see the equation that makes this work in the follow up video (which will be available very soon!)

[@kiranchowdary8100]

ROCKINGGGG!!!! As always.

[@statquest]

Thanks!

[@nbent4607]

Thank you!!

[@statquest]

You're welcome!

[@RelaxingSerbian]

Your little intros are so silly and charming! ^_^

[@statquest]

Thank you!

[@veronicacastaneda6274]

Hey! I love your videos! Can you do one on Weighted correlation network analysis? I share your videos with my friends and we want to learn about it :)

[@statquest]

I'll keep tat in mind.

[@sumangare1804]

Thank your the explanation! If possible Could you do a video on HDBSCAN algorithm

[@statquest]

I'll keep that in mind.

[@92marjoh]

Hey Josh, Your videos have made my learning curve exponential and i truly appreciate the videos you make! I wonder, have you ever considered making a video about Bayesian target encoding (and other smart categorical encoders)?

[@statquest]

I'll keep that in mind.

[@jatin1995]

Perfect!

[@statquest]

Thank you!

[@cytfvvytfvyggvryd]

Thank you for your terrific video! If you got time, could you made a relevant video about densMAP? Again appreciate your wonderful work! Thank you!

[@statquest]

I'll keep that in mind.

[@MinsangKim-n1z]

Hello Josh, thank you so much for the amazing video! I have a question about the mapping consistency of UMAP. In the video, UMAP can keep mapping consistency (meaning that the mapping does not change over the iteration) when we map the projected points on low-dimensional plane based on high-dimensional similarity score, unlike to t-SNE. My question is, it doesn't necessarily mean the final visualization result would be consistent for all time, right? Because since there is randomized sampling, I don't think the final result would be consistent. I tried it using umap-learn lib and the result was also inconsistent. I'm not sure I explained well on my question but please feel free to tell me if there's any ambiguous points. Thank you and have a nice day :)

[@statquest]

The only way to get the exact same graph every time is to set the random seed right before you use UMAP. Although it has less randomness than t-SNE, it still has some randomness.

[@AU-hs6zw]

Thanks!

[@statquest]

bam! :)

[@ashfaqueazad3897]

It will be great if you do some videos on sparse data if you get the time. Would love it. Thanks.

[@statquest]

I'll keep that in mind.

[@paulclarke4548]

Great video! Thank you!! Do you have any plans to clearly explain Generative Topographic Mapping (GTM)? I'd love that!

[@statquest]

Not right now, but I'll keep it in mind.

[@LazzaroMan]

Love you

[@statquest]

Thank you!

[@spenmop]

Your videos are awesome! Makes things so much clearer! But I have a couple of questions: How do you handle the situation where a point has many identical points (ie. high-dim distance = 0)? How to calculate sigma_i? For example, if k = 10, but 7-8 of the neighbours are duplicates with Dij = 0, then sigma_i is undefined. Do I de-duplicate the data first and then add it back in at the end? And symmetrizing: Wij' = Wji' = Wij + Wji - Wij x Wji, yes? But aren't Wij and Wji only calculated for neighbours of i and j? What happens if Wij exists, but Wji does not? Do I add i as another neighbour of j's? (but then j would have more than k neighbours) I'm so confused.

[@statquest]

To be honest, I would just try UMAP out and see what it does. It could treat duplicate points as a single point or do something else.

[@grace6228j]

Thanks for your amazing video! I am a little bit confused, it seems that UMAP is able to do clustering (based on the similarity scores) and dimensionality reduction visualization at the same time, why do researchers usually only use UMAP for visualization?

[@statquest]

That's a great question. I guess the big difference between UMAP and a clustering algorithm is that usually a clustering algorithm gives you a metric to determine how good or bad the clustering is. For example, with k-means clustering, we can compare the total variation in the data for each value for 'k'. In contrast, I'm not sure we can do that with UMAP.

[@andreamanfron3199]

i just love you

[@statquest]

Thanks!

[@cssensei610]

can you cover Locality Sensitive Hashing, and do a clustering implementation in PySpark

[@statquest]

I'll keep that in mind.

[@Friedrich713]

Great quest, Josh! First time I noticed the fuzzy parts on the circles and arrows. What tool are you using to make the slides? Looks damn fine!

[@statquest]

Thanks! I draw everything in keynote.

[@AkashKumar-qe5jk]

Great video!!! One query: What characteristics of the features/dataset we would be analyzing when we choose a smaller value of neighbors? Same question with larger values?

[@statquest]

The number of nearest neighbors we use does not affect how the features are used. The features are all used equally no matter what.

[@hiankun]

The big picture is ❤️ 😃

[@statquest]

You got it! BAM! :)

[@mericknal8752]

echoing UMAP part is amazing 😂

[@statquest]

Thanks! :)

[@wlyang8787]

Hi Josh, would you please make a video about DiffusionMap? Thank you very much!

[@statquest]

I'll keep that in mind.

[@lamourpaspourmoi]

Thank you! Could you do one with self organizing maps?

[@statquest]

I'll keep that in mind.

[@franziskakaeppler5602]

Thank you for this great video. I have a question at 8:21min: Why are the similarity scores 1,0 an 0,6? Could they as well be e.g. 0,9 and 0,7?

[@statquest]

I'm sorry for the confusion. There's an important detail that I should have included in this video, and not just the follow up that shows the mathematical details ( kzbin.info/www/bejne/oKXLZZ57q69mhpo ): the nearest point always has a similarity score of 1.

[@franziskakaeppler5602]

Thank you:)

[@Dominus_Ryder]

StatQuest please do a UMAP tutorial in R next!

[@statquest]

I'll keep that in mind. However, I'm doing the mathematical details next.

[@indolizacja9829]

Have you considered comparing UMAP and Concordex? :)

[@statquest]

Not yet.

[@rajankandel8354]

12:44 why does UMAP decides to move point e farther from b? Is it because similarity score is zero

[@statquest]

At 12:44 we move 'b' further from 'e' because they were in different clusters in the high dimensional space.

[@gama3181]

Hi-dimentional BAAAMM!

[@statquest]

I love it! BAM! :)

[@ranjit9427]

Can you make some videos on recommender systems??

[@4wanys]

complete list for recommender systems kzbin.info/aero/PLsugXK9b1w1nlDH0rbxIufJLeC3MsbRaa

[@statquest]

I hope too soon!

[@leamon9024]

Hello sir, would you cover a dimension reduction technique which uses hierarchical or k-means clustering if possible? Thanks in advance.

[@statquest]

I'll keep that in mind.

[@pranilpatil4109]

But we can we seperate those clusters? We need cluster centroids for that.

[@statquest]

UMAP isn't a clustering method, it's a dimension reduction method. If you want to find clusters, try DBSCAN: kzbin.info/www/bejne/iHW9hpeIiKmCpc0

[@rajankandel8354]

13:27 how do you derive t distribution fit

[@statquest]

That question, and other details, are answered in the "details" video: kzbin.info/www/bejne/oKXLZZ57q69mhpo

[@alexlee3511]

Complicated dataset you referring to is the dataset that cannot be explained by one or two PC?

[@statquest]

yep

[@prashantsharma-sr5dl]

how did the low dimensional plot came just after the similarity score?

[@statquest]

At 4:14 I talk about how the main idea is that we start with an initial (somewhat random) low dimensional plot that we then optimize based on the high dimensional similarity scores.

[@joejohnoptimus]

How does UMAP identify these initial clusters to begin with?

[@statquest]

You specify the number of neighbors. I talk about this at various times, but 17:18 would be a good review.

[@flc4eva]

I might have missed this, but how does UMAP initializes a low-dimensional graph? Is it randomized as done in tSNE?

[@statquest]

This is answered at 16:43

[@user-of6ev3ej8z]

I have a question. After moving d closer to e, do we still consider moving d to c? Or, would c be moved to d? The direction in the video confuses me.

[@statquest]

When we move 'd', we consider both 'e' and 'c' at the same time. In this case, moving 'd' closer to 'e' and closer to 'c' will increase the neighbor score for 'e' a lot but only increase the score for 'c' a little, so we will move 'd'. For details, see: kzbin.info/www/bejne/oKXLZZ57q69mhpo

[@Chattepliee]

I've read that UMAP is better at preserving inter-cluster distance information relative to tSNE, what do you think? Is it reasonable to infer relationships between clusters on a UMAP graph? I try to avoid doing so with tSNE.

[@statquest]

To be honest, it probably depends on how you configure the n_neighbors parameter. However, to get a better sense of the differences (and similarities) between UMAP and t-SNE, see the follow up video: kzbin.info/www/bejne/oKXLZZ57q69mhpo

[@samggfr]

Concerning distance information, initialization and parameters are important. Read "The art of using t-SNE for single-cell transcriptomics" pubmed.ncbi.nlm.nih.gov/31780648/ and "Initialization is critical for preserving global data structure in both t-SNE and UMAP" dkobak.github.io/pdfs/kobak2021initialization.pdf

[@ammararazzaq132]

As PCA required correlation between features to find new principal components, does UMAP approach require correlation between features to project data onto lower dimensional space?

[@statquest]

no

[@ammararazzaq132]

@@statquest So we can still see clusters even when data is not correlated?

[@statquest]

@@ammararazzaq132 That I don't know. All I know is that UMAP does not assume correlations.

[@ammararazzaq132]

@@statquest Okay thankyou. I will look into it a bit more.

[@ali-om4uv]

How does umap know which high dimensional datapoint belongs to which cluster?

[@statquest]

The similarity scores.

[@TheEbbemonster]

Seems very convoluted compared to K-means or hclust.

[@statquest]

UMAP uses a weighted clustering method, so that points that are closer together in high-dimensional space will get higher priority to be put close together in the low dimensional space.

[@juanete69]

But how do you "decide" that a cluster is a distant cluster? PS: I guess you consider a point as a distant point if it's not among the k neighbors.

[@statquest]

correct

[@juanete69]

@@statquest But do you keep "adding" new points to the cluster if they are within the k neighbors of the next point, and so on? Or in order to define the cluster you only consider the k neighbors of the first point?

[@statquest]

@@juanete69 We start with a single point. If it has k neighbors, we call it a cluster and the neighbors to the cluster. Then, for each neighbor that has k neighbors, we add those neighbors and repeat until the cluster is surrounded by points that have fewer than k neighbors.

[@AHMADKELIX]

Permissionntomlearn sir

[@statquest]

:)

[@sapito169]

i think he will sing all the video XD

[@statquest]

:)

[@TJ-hs1qm]

auto-like 👍

[@statquest]

bam!

[@connorfrankston5548]

Thanks, I appreciate the information. However, I think your videos would be easier to watch with a reduction of the "bam" dimension.

[@statquest]

Noted!

[@dummybro499]

Don't say bam....!! It irritates

[@statquest]

noted

[@maburwanemokoena7117]

@@dummybro499 Double Bam!!!

[@ghazalehgolmohammadnezhadk5307]

@@statquestI like it though

[@ScottSummerill]

UMAP is a MESS. No thank you.

[@statquest]

noted