Introduction to Cluster Analysis with R - an Example

Provides illustration of doing cluster analysis with R.
R code: github.com/bkrai/Top-10-Machi...
Data file link and more on cluster analysis: • R Programming Live - L...
For citation as reference in a research paper, use:
Meshram, A., and Rai, B. (2019). “User-Independent Detection for Freezing of Gait in Parkinson’s Disease Using Random Forest Classification,” International Journal of Big Data and Analytics in Healthcare, Vol. 4, Issue 1, 57-72.
Rai BK (2017) “Feature Selection and Predictive Modeling of Housing Data Using Random Forest,” International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, Vol. 11, No. 4, 880-884.
Xiaoling, Lu., Rai, B., Yan, Z., and Li, Y. (2018). “Cluster-based Smartphone Predictive Analytics for Application Usage and Next Location Prediction,” International Journal of Business Intelligence Research, Vol. 9, No. 2, 64-80.
Topics
00:00 Read data file
00:45 Scatter plot
02:30 Data normalization
04:27 Calculate Euclidean distance
05:54 Cluster dendrogram with complete linkage
08:20 Cluster dendrogram with average linkage
08:52 Cluster membership
10:47 Cluster means
12:35 Silhouette plot
13:31 Scree plot
14:47 Non-hierarchical k-means clustering & interpretation
Cluster analysis is an important tool related to analyzing big data or working in data science field.
Machine Learning videos: goo.gl/WHHqWP
Becoming Data Scientist: goo.gl/JWyyQc
Introductory R Videos: goo.gl/NZ55SJ
Deep Learning with TensorFlow: goo.gl/5VtSuC
Image Analysis & Classification: goo.gl/Md3fMi
Text mining: goo.gl/7FJGmd
Data Visualization: goo.gl/Q7Q2A8
Playlist: goo.gl/iwbhnE
R is a free software environment for statistical computing and graphics, and is widely used by both academia and industry. R software works on both Windows and Mac-OS. It was ranked no. 1 in a KDnuggets poll on top languages for analytics, data mining, and data science. RStudio is a user friendly environment for R that has become popular.

[@factChecker01]

This is an excellent tutorial -- well presented and thorough. I followed along with my own application example (country healthcare per capita expenditure versus infant mortality rates of various types) and got very interesting results.

[@bkrai]

Thanks for comments and feedback!

[@ArcenisRojas]

Great tutorial. I really like how you stuck to explaining the steps through a practical application. Thank you for this.

[@bkrai]

Thanks for comments!

[@markshanks9142]

This is truly an excellent, clear and concise tutorial. You covered a lot of topics in a short amount of time. I will be watching your other videos. Well done!

[@bkrai]

Thanks for your comments and feedback!

[@sebastiansocianu5441]

5-star explanation. thank you! Very much recommended for beginners and intermediate R users. You got a new follower!

[@bkrai]

Awesome, thank you!

[@DineshKumarT1990]

Great tutorial!!...the way you explain is easy to understand...you should do more like this

[@bkrai]

Thanks for the feedback!

[@josebueno7602]

Please, how can I get the data utilities.csv? Thanks.

[@karoargote]

Really thank you so much!!! The best tutorial on this topic!!!

[@bkrai]

You're very welcome!

[@rarosification]

My goodness, this video is so complete, and clearly explained with details of the script... Thank you so very much... 100 points to you...!! You have a new fan...

[@bkrai]

Thanks :)

[@kanikalungani]

If i had a thousand likes you would have received them all sir. Love the way you have explained and covered the concepts

[@bkrai]

Thanks, I’ll consider it 1000😊

[@janelutken9818]

Thank you so much. This was easy to follow and I did my own analysis as we went along with almost no trouble. This was a breakthrough video for me.

[@bkrai]

You are welcome! For more detailed presentation, you may refer to: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@ramasamythirunavukkarasu6777]

Thank you so much Dr.B.Rai, I inspired your way of teaching even you in online, hopefully, every one enjoying your teaching

[@bkrai]

You are welcome!

[@stephenhobbs948]

Excellent explanation and code. I took the Johns Hopkins data science course, and clustering was part of the course. This video really helps explain the concept.

[@bkrai]

+Stephen Hobbs thanks 👍

[@archeops.]

Fantastic explanation! I followed along with a different dataset and it worked perfectly! Great work!!

[@bkrai]

Thanks for comments!

[@arnab_jana]

After a long time, I have seen such a good tutorial. Thanks, for your effort

[@bkrai]

+Arnab Jana Thanks for the feedback!

[@rosestube1233]

Thank you for this tutorial! it's amazingly easy to follow and thanks a lot for the script/file

[@bkrai]

+Roses Tube 👍

[@emiltsenov7853]

Hi Bharatendra, this is an excellent tutorial - the first one that worked for me. Great effort, keep up the good work!

[@bkrai]

+Emil Tsenov Good to know, thanks for feedback!

[@rupeshbharadwaj]

Great tutorial! You are really helping a lot of people like me, and the best part is- drama, background music etc are completely missing unlike many other tutorials. Also saw some bhojpuri songs :)...thank you sir!

[@bkrai]

Thanks for comments and feedback!

[@sarahroffe2142]

This is a brilliant tutorial which is easy to understand and follow.

[@bkrai]

Thanks for comments!

[@harikamacharla7005]

Wah!!! how could u explain it so well!! Great job.

[@bkrai]

Thanks!

[@saikrishna2589]

Thank you for wonderful explanation. Appreciate your help with these amazing videos

[@bkrai]

Thanks for your feedback!

[@fredpoole6373]

Great Video! Look forward to more videos!

[@bkrai]

Thanks for comments! For more machine learning videos you can use this link: goo.gl/WHHqWP

[@metalhealth14]

this is a really great detail thank you! I appreciate the detailed guidance into understanding and checking cluster membership

[@bkrai]

It's good to hear your feedback! Thanks

[@ahmetcandemir7032]

Very good tutorial ! impressively well explained. Thank you

[@bkrai]

You are welcome!

[@tradingtraveller05]

Thanks for such wonderful explanation. By the way, I was working on a similar dataset, and apply didnt work for me. Although I removed all character vectors, but still the numeric vectors were returning 'NA'. I applied sapply and it solved the purpose. Thanks again!!

[@bkrai]

Good to hear!

[@nafinks6081]

Excellent tutorial! very easy to grasp.

[@bkrai]

+Nafin Ks thanks for the feedback!

[@mwambakapambwe2382]

Fantastic presentation. Very helpful

[@bkrai]

Thanks for comments!

[@kapilrana1153]

Great Explanation! Thank you Sir For this Video Lecture I will be watching your other videos.

[@bkrai]

Thanks and welcome!

[@Aminah6623]

Wow. This was extremely helpful. Thank you.

[@bkrai]

You're very welcome!

[@liamhannah6325]

This was really helpful THANK YOU! Make more! I would love it if you showed us how to do Latent Class Analysis in R, its not obvious right now

[@bkrai]

Thanks for comments and suggestion!

[@kandreitapomen]

Great tutorial. Thank you very much!

[@bkrai]

+Kandreitapomen 👍

[@abdulkhader101]

You are a great teacher sir, you are really awesome

[@bkrai]

Thanks for comments!

[@saikatkar547]

thats really excellent explanation!

[@bkrai]

Glad it was helpful!

[@khushboobegwani1612]

Thank you so much sir for informative video. You really made it easy.

[@bkrai]

Thanks for your comments!

[@desisto007]

Thank you so much! Very well explained. I would like to ask you if I still can use the Euclidian distance to find the closest elements of a cluster center, even if I use a dimensionality reduction approach (such as PCA, T-sne) that uses probabilities to arrange clusters in 2 dimension before using K-means.

[@bassamal-kaaki3253]

Lovely explanation:) easy to absorb.

[@bkrai]

Thanks for comments!

[@omkarsingh6060]

Amazing...Really impressed

[@bkrai]

Thanks for comments!

[@txigual]

Thank you so much, very useful video.

[@bkrai]

Thanks for comments!

[@ssundaraju]

Very Informative, great slides and explanations. The delivery and presentation was good. I will be viewing other videos produced by Edureka. Some suggestions, show more examples. Present the limitations and god fit scenarios for K-means clustering.

[@bkrai]

Thanks for comments and feedback!

[@EduardoFrancoChalco]

Really great tutorial, thank you very much!

[@bkrai]

+Eduardo Franco Chalco 👍

[@EduardoFrancoChalco]

Would you please send me the scrip and data? email: [e-mail]

[@gulapakarthik3864]

This is really Amazing...Thank you so much 😎

[@bkrai]

You are welcome!

[@phediasdiamandis2441]

Great Video. Congrats

[@bkrai]

+Phedias Diamandis thanks for the feedback 👍

[@zhuziyan9454]

dear professor, I am so lucky to know you. could you also update full tutorial about using rmd and advanced model like hmm? Thank you and wish you have a great day

[@bkrai]

Thanks for the suggestion, I've added this to my list.

[@javzmaatsend3785]

Thank you, Very easy

[@bkrai]

You are welcome!

[@kishoreyarramshetty2930]

Good Job in explaining the content along with code..

[@kishoreyarramshetty2930]

can u provide us the link to download the dataset in this video to run the code.

[@bkrai]

Thanks for comments!

[@bkrai]

For data, there should be a link below this: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@alicelatimier3133]

Thank you so much for your amazing videos, everything is so clear and practical :) From a french research in cognitive science, I have one tricky question for you : i would like to find the best classifier/cluster analysis for repeated measures dataset (i.e., multiple repeated measures for one subject on the same features, as this is the case in experimental psychology research for example, or in longitudinal studies). Best

[@bkrai]

You can look into this link: kzbin.info/aero/PL34t5iLfZddvMPAl1TzHJ_GjQcD3s6w_Z

[@rinoypaultharu5071]

Great tutorial, it really help for my analysis. Im having some douts, in that while silhouette calculation, whether we need to check average silhouette value, or which value we have to check to find out the number of clusters. Please help me with that. In your analysis what is the silhoutte value for k=3, where it is showing on that plot? Second while calculating my Euclidean distance, i have 40 observations, so it is not showing complete rows of Euclidean matrix, so is there any other way to obtain the complete matrix

[@asifjeelani1215]

thank you sir, very well explained

[@bkrai]

Thanks for comments!

[@DeepeshSinghAndroid]

Hi Mr. Rai, great tutorial. Thanks for your effort. Just wanted to understand more about these 2 methodologies. Why and when we apply different methodologies i.e. K means and Hierarchy. It will be great help if you can make separate videos for the same. Also, as lots of people requested for data set and you have already uploaded to Dropbox, could you please share the link in your description for everyone's benefits. Thanks again :)

[@bkrai]

Initially we try all methods and finally choose the one that seems more meaningful for the dataset used. It's difficult to say which method will work best beforehand. Also thanks for your feedback and suggestions.

[@prashantmishra2094]

nice tutorial Sir. Keep making such videos

[@bkrai]

Thanks for comments!

[@vinzkyvijayaraj4035]

Thank you!

[@bkrai]

You're welcome!

[@biswadeepdas5528]

sir, it is quite good. I would really appreciate if you upload more videos .

[@bkrai]

+biswadeep das thanks for your feedback! I'll definitely create more such videos.

[@shivakazempour5603]

you are amazing!

[@bkrai]

Thx for comments!

[@stephravelo]

This is a very informative video. I hope you would have a repository github of your data so that we can play around with the script you used.

[@bkrai]

Here is the link: github.com/bkrai/Top-10-Machine-Learning-Methods-With-R

[@gayatritadla3844]

Thank you.

[@bkrai]

+Gayatri Tadla 👍

[@Masshiara]

Thanks! a lot!!

[@bkrai]

Welcome!

[@niv2419]

Hi! Thank you so much of making this blog! Can you please make a video on feature engineering in R? Thank you!

[@bkrai]

Here is the link: kzbin.info/www/bejne/jHalkqtojLKVe6M

[@AnchalSingh06]

Thank you for posting this video. It's helpful. I have (500,226) data . Please guide me to do Kmeans and Silhouette in R

[@maggief6653]

I´d like to change the dendogram position (Horizontal Plot), what package and function can I use?

[@hridayborah9750]

yes all your videos are helpful. Could you prepare a tutorial on machine learning in the tidy verse.

[@bkrai]

I've added it to list of future videos. Thanks!

[@DrMDarwish]

Thanks :)

[@bkrai]

welcome!

[@Nit1601]

THE BEST !!! Could you please advise, do we need to do anything else to normalize if we are dealing with Binary columns (0,1). Thanks !

[@bkrai]

We should exclude such variables.

[@tahzeebfatima3121]

Thanks for the informative video. May I please know how to deal with dichotomous variables along with continuous variables in the data if we want to include both in one cluster analysis, how do we do it please?

[@bkrai]

This link has more cluster analysis topics: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@betzthomas9693]

Thank you Sir for the tutorial.Please explain if there is any package is R to identify on what basis clusters are grouped from the data we provide.

[@bkrai]

Refer to the averages for each cluster and all variables.

[@santosacosta4645]

Thank you very much sir. Question: using Within group SS plot (min 14:39), isn't the optimal number of clusters 5? the variability from 4 to 5 seems very significant. Please let me know.

[@bkrai]

This data has only 22 companies. As we increase number of clusters, number of companies in some clusters becomes really small, to the extent that a cluster may contain just one company. So the choice of 'k' should also consider this aspect.

[@sapnapatil27]

Thanks.

[@bkrai]

Welcome!

[@azfersaeed1602]

Great video man! Thank you very much for posting :). Could you show cluster analysis using more than 2 variables?

[@bkrai]

+Azfer Saeed thanks for feedback! In the example we have cluster analysts with 8 variables. However for scatter plot we use two variables at a time.

[@azfersaeed1602]

+Bharatendra Rai You are correct...sorry for the incorrect semantics. At 2:15, you mention that broadly, there are 3 clusters but they are based only on 2 variables. Is there a way to create clusters based on more than 2 variables?

[@rezaamirahmadi6013]

Thanks , How can I use fuzzy k-means (FKM) to impute missing in R ?

[@manigandanv8531]

Thank you so much for explaining this . It would be really grate help if you could upload a video using bray curtis similarity.. b

[@bkrai]

Thanks for the suggestion, I'll keep it for future.

[@thejuhulikal6290]

sir please make the video on this K-mode also, that would be great to understand both topics and comparison

[@bkrai]

Thanks, I've added it to my list.

[@AdityaLandge1994]

nice tutorial

[@bkrai]

thanks for comments!

[@VenkateshDataScientist]

HAPPY NEW YEAR TO YOU AND YOUR FAMILY MEMBERS . Sir ,If you have time please upload support vector machine and Sentimental analysis .

[@bkrai]

A very happy new year to you and family too! I'll keep your suggestion in mind for next videos.

[@bkrai]

Here is the link to SVM: kzbin.info/www/bejne/poSYmIt7g8lml5Y&list=PL34t5iLfZddtII4ssT8FSUFP27fPYDEhY&index=25

[@kratostube1]

vERY GOOD !!

[@bkrai]

Thanks!

[@jalluravikiran4146]

good video.

[@bkrai]

Thanks!

[@jonathanrhein7553]

Hi Bharatendra, great video - really helpful! Everything goes well until the point of doing the scree plot, I am getting: > withinGroupSumOfSquares = (nrow(normNum)-1) * sum(apply(normNum, 2, var, na.rm=TRUE)) > for(i in 2:20) withinGroupSumOfSquares[i] = sum(kmeans(normNum, centers=i)$withinss) Error in do_one(nmeth) : NA/NaN/Inf in foreign function call (arg 1) > plot(1:20, withinGroupSumOfSquares, type="b", xlab = "Number of Clusters", ylab = "Within group SS") Error in xy.coords(x, y, xlabel, ylabel, log) : 'x' and 'y' lengths differ Can you help me? Thank you.

[@jonathanrhein7553]

someone has deleted my comment...

[@bkrai]

+Jonathan Rhein Not sure what's causing the error you got. May have something to do with data. I ran my data using the code you have, and everything seems fine.

[@bkrai]

+Jonathan Rhein I still see your previous comment.

[@sajidurrahmannafis8476]

Best tutorial in the internet. I have one question: why are using euclidean distance then again complete linkage? I thought we need one distance measurement technique. I will be really grateful if someone can clarify. My questions answer may help others also. Thank you.

[@bkrai]

You can refer to this more recent one: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@sajidurrahmannafis8476]

@@bkrai Thank you sir. I am a big fan of your teaching. I am also a research assistant in US. Thank you for your amazing lectures!

[@sajidurrahmannafis8476]

@@bkrai Thank you. I got the answer to my question from your new cluster video lecture.

[@bkrai]

Thanks for the update!

[@bkrai]

You are welcome!

[@mariaamithapennington3737]

Thank you so much for the tutorial. It is extremely helpful. But my question like the other is that it would have been very kind of you if you would have linked your data set too. Thanks!

[@bkrai]

You can get it from here: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@mariaamithapennington3737]

@@bkrai Thank you very much! Appreciate it! :)

[@bkrai]

You are welcome!

[@ramp2011]

Great tutorial. Thank you... How do you handle categorical variables for clustering? In this example looks like you removed the 1st column that happened to be a factor variable. Can you please post the data file used in the comments as well if possible? Thank you

[@bkrai]

Cluster analysis only works with quantitative variables. During the analysis you may note that we calculate distances, which we cannot do with categorical variables. But after finalizing number of clusters, you can plot dendrogram with a categorical variable to see if there is any obvious pattern or not. For data, send email id.

[@Jorge-vp7of]

you can use K-modes to do clustering with categorical data

[@medardkafoutchoni6511]

Thank you dear Sanchez. What about mixed data (i.e. including both numerical and categorical variables)?

[@vivekwilliam3370]

[e-mail]

[@tanmaygawade1068]

hello sir!! actually wanted to know how to perform clustering on PCA generated scores in r and how to compare the cluster size for both.

[@mfkalabdullah6966]

Sir, Do you have more videos on clustering? Also, can I contact you in the future regarding clustering because I'm doing a research using data mining clustering?

[@bkrai]

There is a playlist on clustering: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@mayankhmathur]

Great

[@bkrai]

Thanks!

[@rithishvikram1759]

nice explaination sir!!!!! thank you so much ....great respect ....sir if you would pls attach concern datasets with a video ...thank you once again

[@bkrai]

send your email id

[@rithishvikram1759]

[e-mail]

[@bkrai]

all set.

[@rithishvikram1759]

thank you so much sir

[@zhuziyan9454]

could you please explain why subtracting the first variable by [,-c(1,1)] rather than[,-1]? Thank you

[@bkrai]

Both work fine. You can use it if you need to remove more than one variable.

[@TusharLapani]

Thanks Bharatendra. Can you please upload video of how to performe clustering when the dataset has numbers of numerical attributes and categorical attributes. In this video you are eliminating categorical attribute. What would you have done if your dataset has 10 numeric columns and 8 categorical data. Appreciate your knowledge contribution.

[@bkrai]

+Tushar Lapani For cluster analysis you must have quantitative variables. You can use categorical variables after cluster analysis to see if they show any pattern with identified clusters and use it for characterizing the clusters.

[@tsmg6889]

thanks.

[@bkrai]

Welcome!

[@Guavarosa]

Please can you give me a hint? I want to give as input the initial centres for kmeans clustering. I just do not manage to select these points out of my dataset. Thank you in advance for your help!

[@bkrai]

Why do you need that? The algorithm should automatically take care of finding the best clusters.

[@Guavarosa]

@@bkrai Because I try to correlate my clusters to the physical problem. That is why I was wondering if I can give initial centres as in case of software Origin Pro. I appreciate your answer.

[@aks1008]

Sir how to remove multicollinearlity in cluster analysis as it is an unsupervised algorithm..there is no dependent variable..

[@bkrai]

Multicollinearlity is a problem only for regression models. For cluster analysis it not an issue.

[@sanjayh3897]

Excellent tutorial Bharatendra ! Do you have any example to share for Overlapping clustering - would appreciate it. Thanks !

[@bkrai]

There are 52 datasets where clustering can be applied in the link below: archive.ics.uci.edu/ml/datasets.html?format=&task=clu&att=&area=&numAtt=&numIns=&type=&sort=nameUp&view=table

[@rohanshetty1016]

Sir your video lectures are really awesome! Excellent Tutorial! Can you please share the csv file used for cluster analysis?

[@bkrai]

send me your email id.

[@maryamaziz5064]

would love to try it on my own

[@bkrai]

Thanks!

[@anigov]

Dear Sir..thank you for the time & effort that you have put in to make this wonderful video tutorial. I have a query. At 12:27 , how are the original average values displayed even though member.c is used which is obtained through a series of calculations using the normalised data? Why did not you use PCA to decide the no. of clusters for kmeans? Regards Aniruddh

[@bkrai]

In the 2nd aggregation line, note that I've used utilities. That's the reason we can display original values. In the 1st aggregation, z was used. Also, here focus was on clustering, so pca is not used.

[@anigov]

Thank you

[@deepaksingh9318]

A good tutorial , Could you please also tell us when should we go for Kmeans and When should we go for Hclust(I.E situations to select methods) 2. What do we mean when we say above average and below average (in Hclust) , i mean if the value is 1.05 so are we saying that sales in cluster x is higher 1.05 than average ?? a explanation will be appreacited.. REst everything is explained in a really simple way so Subscribing the channed :) Keep it up..

[@bkrai]

For more on clustering: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@niv2419]

Hello sir, as always your videos have been very helpful and thank you for this video too. Also, I wanted to know if there is a way to improve between cluster distance? If so can you please let us know? Thank You!

[@bkrai]

You can increase or decrease number of clusters and see which one improves between cluster distance.

[@jaatni64]

Sir... Please tell me that why i am getting different result when we run kmeans command twice or more

[@bkrai]

you can run set.seed(123) each time before running kmeans to get same result.

[@adityasharma2667]

Hello Sir, Just wanted to know, after running K-Means with 3 cluster, we found that there are two cluster which are overlapping, how to remove overlapping of these two clusters otherwise it would not be called as K-means clustering.Please correct me If I am wrong sir. Thanks

[@bkrai]

Some amount of overlap is common with k-means or any other clustering method.

[@shubhasmitasahani1738]

Hello Sir, do you have any video on latent class clustering in R? Please share...Looking forward.

[@bkrai]

Not yet, but I'm adding this to my list for future. For clustering related videos, you may refer to this link: kzbin.info/aero/PL34t5iLfZddvMPAl1TzHJ_GjQcD3s6w_Z

[@mattcotoia8749]

Can you tell us what each variable means in this dataset please?

[@nitinchan]

Sir, do you have any tutorials for time series clustering?

[@bkrai]

This playlist has many cluster analysis topics: kzbin.info/www/bejne/paXNiHaXgsiJl6M

[@thejuhulikal6290]

Hello sir, please upload a video on Qualitative comparative analysis!! thanks again sir

[@bkrai]

I've added it to my list, thanks!

[@machinelearningzone.6230]

HI Sir, How do we assign the clusters to new data points, like if we have a new data set but use the same model. Regards Gourab.

[@bkrai]

You can develop a prediction or classification model with cluster as independent variable.

[@machinelearningzone.6230]

@@bkrai Thank you . Do you mean that we can develop a classification model using the clusters labels as classes?if so,then how do we take into account the distance parameters like eucledian or taxi cab etc..

[@bkrai]

You don't need that as it is already baked into the clusters.

[@jyoti9426]

How to plot clusters if I already know the affiliations of the nodes?

[@bkrai]

Not sure about your question, but you may try this: kzbin.info/www/bejne/rX3YY2Rpf7CZpLM

[@sudhakarbabunynavarapu8133]

Could you please send the data files for the practice what datafiles used in the tutorial.

[@bkrai]

email id?