Stacking Ensemble Learning|Stacking and Blending in ensemble machine learning

[@atomicbreath4360]

First of all thanks for the video. Bagging: when we take different models and train them parallely with each getting sub-set if data from the total data and each model has high-variance and low bias. Boosting: same as above but the difference is instead of training them parallely ,output of one model is given as input to the other and each model should have high-bias and low-variance.

[@UnfoldDataScience]

Well said!

[@crazycurlzXD]

I've been struggling to understand this for quite a few hours now. Finally, got it. Thank you so much!

[@UnfoldDataScience]

Glad it helped!

[@hetal1926]

I am a new in this field and I was trying to understand this concept, refered many webpages and seen many videos. You explain Very nicely. I got it the concept.

[@UnfoldDataScience]

Thanks for watching.

[@TheMuktesh89]

That is very nicely Explained. Thank you, Sir.

[@tosinlitics949]

I love that you throughly explained the theory before you dove into the code. Great job!

[@UnfoldDataScience]

Thank you.

[@sridattu4467]

I beg my pardon....I was struggling with this technique Very clearly understood and the code n got executed!! Thanks a lot

[@UnfoldDataScience]

Thanks for watching Sri.

[@davidgao5351]

Great explanation of the concept. Thank you for also showing the python samples to really bring it home.

[@sharatainapur]

Hello Aman Sir, Thank you for the great video, simple explanation. Could you please elaborate on how the meta-model is built and used for the testing / real-test set? Like here, the meta-model uses Logistic Regression, right? How a logistic regression works to stack the results from base model?

[@ranajaydas8906]

Thanks a lot... Was struggling with this Stacking approach.... Now it's clear!

[@UnfoldDataScience]

Cheers Ranajay :)

[@bharatbajoria]

Bagging is Bootstrap Aggregation which is used primarily to reduce Variance , it uses CLT to do the same. Boosting improves the base learners by learning from mistake of the previous model using homogeneous weak learners, it helps in reducing Bias.

[@UnfoldDataScience]

Thanks Bharat.

[@saurabhdeokar3791]

In bagging , we make different subset of dataset by using row sampling and replacement and that subset we pass different model's to make prediction and at the end we combine or aggregate all of the model prediction.

[@FarhanAhmed-xq3zx]

Greatly explained💥👌

[@UnfoldDataScience]

Eid Mubarak Farhan. Tc

[@arrafihriday1333]

Laudable teaching. Learnt a lot.

[@sonalkapuriya4944]

crystal clear explanation

[@reflex2627]

Absolutely very good explanation , better than my professor

[@UnfoldDataScience]

Thanks a lot. your comments motivate me.

[@vaddadisairahul2956]

Bagging helps in reducing variance due to overfitting in decision trees and further to reduce bias, boosting is used. Hence, ultimately we achieve a model with low bias and low variance.

[@UnfoldDataScience]

Correct Rahul.

[@akshayshenoy7417]

Beautiful man got my concepts cleared you deserve more reach.

[@UnfoldDataScience]

Thanks a lot Akshay. Kindly share in the data science groups you are part of :)

[@nan8922]

Wow, fast and cleare, thanks.

[@UnfoldDataScience]

You're welcome!

[@sandipansarkar9211]

finished watching

[@preranatiwary7690]

Good content

[@UnfoldDataScience]

Thanks a lot.

[@phanikumar3136]

bagging is the process known as parallel computing and in this method, we can choose rows and columns with replacement and its example is randomforest....but in boosting it is a series computing and example xgboost.

[@UnfoldDataScience]

Correct. Thank you.

[@pankajnegi9278]

In bagging we take base learner models, with high variance and low bias,. Eg - random forest we typically take decision trees ( which are Fully grown to their max depth, with max_depth = None) as such decisions trees are high variance, models. The main aim of bagging itself is to reduce high variance of the overall/final model, In bagging we have bootstrap (row sampling and column sampling) and aggregation steps which helps to achieve low variance final model, Also every base learner is being train on a sample dataset not on whole data set, so every base learner is learning something unique or different from other base learners

[@mikohotori4276]

Thanks for your sharing.

[@UnfoldDataScience]

My pleasure

[@magesh10mano]

Good Explanation... Thank you

[@UnfoldDataScience]

You are welcome

[@Mars7822]

Nice class

[@UnfoldDataScience]

Thanks

[@tusharsalunkhe7916]

Thank you sir for this lecture. Want to know one thing... data which goes to Meta model consist of Independent variables and actual output value (Target variable Y) along with predictions from weak learners like LR, SVM, NN... so how does Meta Model use predictions from weak learners to predict the final output/ prediction? Is Meta model consider predictions from weak learners as additional independent variables(along with existing independent variables) and target variable as dependent variable and give final prediction ? Please help.

[@UnfoldDataScience]

Good question Tushar. The meta model will take predictions from weak learner as features.(no original feature)

[@tusharsalunkhe7916]

@@UnfoldDataScience Thanks for reply. So predictions from weak learners are taken as Independent variables and original target variable as dependant variable... Right?

[@samuelpradhan1899]

How to use stacking regressor models from sklearn and keras??

[@harshakrishna8259]

Thanks a lot bro! ... Helped a lot for one of my projects!!

[@UnfoldDataScience]

Welcome Harsha.

[@hansmeiser6078]

Simply wonderful!

[@UnfoldDataScience]

Many thanks!

[@SandeepSSMishra]

Can you make a separate video for Blending with detailed example and implementation without the libraries?

[@Kumarsashi-qy8xh]

Nice subject

[@UnfoldDataScience]

Thanks a lot :)

[@manjunambiar4954]

While executing the for loop, there is an error message. "type error:KNN not iterable" .How to solve this?

[@tolifeandlearning3919]

Good Explanation.

[@UnfoldDataScience]

Glad you liked it

[@christian.belmar]

excellent

[@UnfoldDataScience]

Thanks Chris

[@pranitflora9482]

Very well explained. Can you also explain KcrossK cross validations and go in dept of meta model.

[@ujjwala7286]

Thank you for explaining .Can you suggest which ensemble techniques is suitable for deep learning model for video classification task

[@UnfoldDataScience]

Yes

[@sangrammishra4396]

Love your study sir..

[@UnfoldDataScience]

Thanks Sangram.

[@sandipansarkar9211]

I am unable to locate this ipynb file in your respective google drive .Please guide

[@UnfoldDataScience]

I think this file is missing, I will try to find out and place it however I doubt it may be in my old laptop and difficult to recover.

[@sonambarsainya3747]

Hi Aman, thanks for your explanation. I have a question regarding deep learning models, Can we stack yolov4 (which I converted the .weight file into a .h5 file) and other CNN models like InceptionResnetV2+LSTM into one ensembling model for different classification with different data?

[@diniebalqisay2658]

I also have the same question...

[@seema5579]

Thnks 4 the video, sir, can i perform stacking between different CNN models and feature fusion in between these models

[@SandeepSSMishra]

Sir Namaskar. That code you did in Python is for stacking or blending, kindly say.

[@MegaBoss1980]

Can we do level 2 meta model. Also can we insert new training features in meta model?

[@Neerajkumar-xl9kx]

Thanks a lot , i am a beginner

[@UnfoldDataScience]

Thanks Neeraj.

[@eduardocasanova7301]

Hi Aman, thanks for your explanation! I have a question though - is regularization and ensembling the same? In the decision trees case we use the same techniques of bagging and boosting, so, if i'm regularizing am i implicitly ensembling and viceversa? Thank you!

[@talaasoudalial-bimany6605]

Than you very much Just I want to understand there are how many approaches when imlpimenting stacked ensemble learning I mean when we combine base learners to meta learner

[@UnfoldDataScience]

The way of implementing can be many depending on how u implement that in code however the internal logic remains same.

[@sadhnarai8757]

Very good Aman

[@UnfoldDataScience]

Thank you.

[@shivanshsingh5555]

@5:48 at this time, you said "and this training data goes to another model, called meta model." the way you pointing the finger and that what to said is not getting understood by me. For me these are very much imp so i goes to the depth of each n every word along with action. Please kindly sort my query please? and what is training model here after dividing 75 records into 80-20%....? if it is 80% (as i know) then y didn't you mentioned it....im confused

[@mohammadmoslemuddin7274]

I can share with you my understanding. First, we divide the 100 training examples into 75 Training and 25 Test examples. Then we divide the 75 Training examples into (80% Training and 20% Validation examples) i.e. 60 training and 15 validation examples. After that, we train the different base models on these 60 training examples and make predictions on the 15 validation examples. The predictions on the 15 examples become the input to our meta-model. Now we train the meta-model and test our accuracy on the initial 25 test examples. In short this Blending. When we follow the K-fold approach to divide the 75 Training examples to divide and train as 60 training and 15 validation examples, it is called stacking. Hope it helps. Happy learning.

[@UnfoldDataScience]

Thanks Mohammad and Shivansh for discussion.

[@chitramdasgupta3122]

Thank you! Keep making these videos.

[@UnfoldDataScience]

Will do Chitram. Your comments are my motivation.

[@vinushan24]

Thanks!

[@orchidchetiaphukan4658]

Clearly Explained.

[@UnfoldDataScience]

Thanks a lo for motivating me.

[@orchidchetiaphukan4658]

@@UnfoldDataScience Sir, Can you share the notebook of this tutorial?

[@sagaradoshi]

Hi Aman, Thanks for the video..I have one question When we have finished training the stacked model and now when we have test sample..will the test sample go through all the learners + metal model (i.e, SVM, Random forest, gaussian and logistic regression-meta model) or we will feed the test sample only to meta model (i.e., logistic regression in our case)?

[@stemicalengineer]

Also have this question in mind. Why is no one answering this?

[@anuragchandnani8037]

thank you

[@UnfoldDataScience]

You're welcome Anurag.

[@ling6701]

Thank you.

[@jyotireddy341]

Hi. Thank you so much for the video. Can you please guide on how to merge 2 BERT models together. Thanks for the help!

[@UnfoldDataScience]

Thanks Jyoti, will do

[@MAK335]

sir which ensemble technique we should choose? and when we should choose ? how do we decide that

[@UnfoldDataScience]

Bagging and Boosting are good. Decision will happen based on available resources , Data Size etc.

[@MAK335]

@@UnfoldDataScience nobody has made a single video on this on KZbin. you should definitely make a video on this topic !!!

[@svltechnologiespvtltd9181]

nice explanation, how can we do testing with test data set

[@UnfoldDataScience]

same way like normal ML

[@nasali5116]

is it possible if blending model sometimes lower accuracy that the initial model??

[@UnfoldDataScience]

Possible.

[@UnfoldDataScience]

Possible.

[@RinP3103]

Hi.. I am very much interested in ML concepts and try to build career in this, but I can see lots of mathematical derivations are there when trying to learn any new concept, also so many libraries are there, its quite difficult to get acquainted to all these, can you please guide how to actually learn all these so that can be understood well.

[@UnfoldDataScience]

Hi Rinky, Please watch my machine learning playlist once. Tell me if it boosts your confidence: kzbin.info/www/bejne/boGppWeAntNqeJI

[@MrXRes]

Thank you for the video Can this approach be usfull for semantic segmentation purposes? For example we have metamodel consists of UNet, Deeplab and FCN And metaclassifier FCN Is it going to get better result?

[@UnfoldDataScience]

Yes, we can try that, I am not 100% sure it will work though.

[@ASNPersonal]

Getting error with this code: # creating stacking classifier with above models stackingclf = StackingClassifier(classifiers=[myclf1, myclf2, myclf3], meta_classifier=mylr) Without error below code: # creating stacking classifier with above models stackingclf = StackingClassifier(estimators=[myclf1, myclf2, myclf3], final_estimator=mylr)

[@UnfoldDataScience]

may be this argument method is not taking "meta_classifier" due to version issue.

[@gauravkanu4486]

Thanks a lot @anugati saved my time!

[@sabeenak7159]

Thank you sir

[@UnfoldDataScience]

So nice of you Sabeena.

[@studywithme4275]

thanks

[@UnfoldDataScience]

You're most welcome

[@raphaeldayan]

thank you a lot!

[@UnfoldDataScience]

You're welcome raphael.

[@FlorieAnnRibucan]

Good day! May I request a link for a copy of your code, sir? Thank you

[@sandipansarkar9211]

finished coding

[@jagannathdas3991]

Sir stacking thoda aur clearly bolte... Blending achha tha..🙏🙏

[@UnfoldDataScience]

Feedback k liye Dhanyavaad. :) dekhne k liye bhi :)

[@MAK335]

iris is not binary classification it has more than 2 classes in target variable

[@UnfoldDataScience]

Yes, Correct it has three categories. Did I say 2, Thanks for pointing out.

[@MAK335]

@@UnfoldDataScience you didn't say 2 but you said iris is a binary classification dataset .....

[@zainabfatima9932]

Code for this one?

[@UnfoldDataScience]

I m searching for it however there is a possibility it may have it in my old laptop. I will try to find and upload.

[@Monty-hl1rb]

Sir ..Plz exam briefly... Not understanding 😟😟

[@riyosantoyosep1749]

please make subtitle

[@SESHUNITR]

Good job

[@UnfoldDataScience]

Thanks Sesha.