How to stack machine learning models in Python

[@DataProfessor]

👉 Watch this next: kzbin.info/www/bejne/pYOZaGOKrdybbpo (The Art of Learning Data Science - How to learn data science in 2021) ---------- 🌟 Download Kite for FREE www.kite.com/get-kite/? 🌟 Buy me a coffee www.buymeacoffee.com/dataprofessor 🌟 Subscribe to this KZbin channel kzbin.info 🌟 Join the Newsletter of Data Professor newsletter.dataprofessor.org

[@skylerfranklin7089]

@Kingston Dangelo thank you, I signed up and it seems to work :D I really appreciate it !

[@kingstondangelo4541]

@Skyler Franklin You are welcome xD

[@Dr_Insha_Altaf]

Sir can you please provide me its pseudocode???

[@TinaHuang1]

Wow this is great - was just thinking about this couple days back!

[@DataProfessor]

Awesome, thanks for tuning in Tina!

[@gguchristine]

I was literally thinking about how to do stacking and I saw your video in my subscription box haha thanks for the video!

[@DataProfessor]

Awesome, glad to hear!

[@remymakota1813]

@@DataProfessor Hello there, Is it possible to use Particle Swarm Optimization as part of the stacking models? If so, could you kindly show me how?

[@paulntalo1425]

Thank you professor, your contribution towards enablement for data scientists is unmatched in this year. Your my best channel towards full stack data science

[@DataProfessor]

Thanks Paul for the encouragement and support of the channel :)

[@taseersuleman7343]

Much awaited video 👍

[@DataProfessor]

Thanks Taseer!

[@mogamoga4474]

sir, why always logistic regression is used for stacking?

[@shahadsha8692]

logistic regression for classification tasks

[@andykim7654]

I’ve noticed that the outcomes from the random forest model and the stacking model are identical. Any thoughts?

[@stephentete1211]

Yes I was wondering also, the same for the svm model. @DataProfessor Could you please clarify this. Thanks!!

[@shubhamdandekar20]

Thanks for this video now i understand what stacking is.

[@DataProfessor]

Glad to hear that :)

[@taki7394]

you are great, prof !

[@SandraBabirye-t7d]

How are you able to obtain features of importance from the stacked model

[@SadTeddyBeer]

At the end of the video, when we print the final df with metric scores in it, we can see that the stack model is mainly inspired of the random forest classifier. Why not go to the decision tree who has 1 to all its scores ? It's because 1 is likely to be a biased and so the stack classifier doesn't take it in count ?

[@Julio_Zambrano]

Amazing! This video is super helpful! Thank you, Professor! :D

[@DataProfessor]

Thanks for watching and glad to hear that it's helpful 😊

[@Dr_Insha_Altaf]

@@DataProfessor kindly provide me its pseudocode...

[@mamacita5636]

Thank you! Quick question, why do you perform train-test split if you’re going to use cross validation? Wouldn’t the cross validation do the split ?

[@futureceltic00]

So if I were not to use StackingClassifier, this is basically me consolidating all predicted classes of the base models and using them as features for the meta model? If this is the case, then does stacking also detect overfitting from too many features used in the base model if the performance decreased on the final meta model?

[@muditarora9860]

it shows error at this line stack_model_train_accuracy = accuracy_score(Y_tr, Y_tr_pr) can not be continuous.

[@DataProfessor]

Can you try r2_score function instead (also import the r2_score function first), I think you're Y is a quantitative value therefore the error is saying "continuous". For accuracy_score function to work your Y has to be categorical (or discrete values).

[@gammetube3976]

Thank Dear. how we check Cv of the model and save the developed model and check the evaluation of the model?

[@ama016]

This is SO awesome! Laser eyes ML

[@DataProfessor]

Glad its helpful :)

[@tsunamio7750]

Could you make an example of this with Keras neural networks? This is a very specific issue when you wrap your DNN with KerasClassifier, where you must provide the new model as a function or something... instead of training it.

[@hareshk.mangtaniretrita1376]

Great video! Had a quick question for you. After training and testing the KNN algorithm, how are the metrics of the performance of the test set higher than that of the training set? Haven't we trained the model using the training set? I would expect the model to be more accurate when making predictions in the training set (which is seen data) as opposed to the test set (unseen data). Regards!

[@chadgregory9037]

LOL you are right.... something SUS going on here!

[@bedoe9684]

I dont have a theoretical answer. But it does happen that validation scores are greater than training ones. Might be because of the test split itself containing more favorable data (aka data that the model has learnt very well) Regards

[@gammetube3976]

great question! how we apply the feature extraction on this model?

[@nurmukhammad_30k]

Very nice explanation! Keep going!

[@connectrRomania]

Man your videos are awesome, keep up the good work. Thank you

[@DataProfessor]

Glad to hear! Thanks!

[@Kmysiak1]

Whats your logic for using log classifier final_estimator? How come you didn't tune your hyperparameters? Good clean code and well explained but could be better.

[@username42]

what about the cross validation of the model ? how do we do that in such stacking models?

[@DataProfessor]

There's a built-in CV option in the function of stackingclassifier and stackingregressor

[@username42]

@@DataProfessor cool so it is doing the cv before stacking the model or afterwards ? for instance in yourcase, is it gonna be cv the logistic regression model or the previous ones?

[@kholoodsh4616]

It shows error: ValueError: The estimator Sequential should be a classifier. at stack_model.fit(x_train,y_train) , how can I fix it ?

[@t.t.cooperphd5389]

Beautiful!

[@DataProfessor]

Thanks for watching!

[@sangnp]

Sir, Can you please answer me.Is it a 2 layers stacking ?

[@aimanjatt4]

I got error Y_train is not defined how can i fix this error plz tell me

[@Karenshow]

Can we use the stacking on Time Series Models??

[@yaminadjoudi4357]

thank you for this, is the stacking is the same concept as Modular neural networks MNNs please ?

[@aditiarora2128]

sir great explanation...but still i am confused about formation of dataset for meta learner...different blogs says different concepts of creating training dataset for meta learner. For example you have first training every model individually and and then stacked model again on same training testing set!!! But in many blogs I have seen that input training dataset for meta learner should be formed by combing Pred_prob of every base model and actual label. Plz clarify!!!

[@jaredking929]

Wouldn’t it be great if you could explain the strengths of each algo and show how they improved the models

[@mogamoga4474]

Sir, when I'm using my own data set, this line "X = data.drop('Activity', axis=1)s" is not working...showing invalid syntax

[@DataProfessor]

Hi, you have an extra "s" at the end of your syntax, please delete it.

[@mogamoga4474]

@@DataProfessor thanks sir, it worked.

[@mohak9102]

Please always keep up the great work

[@DataProfessor]

Thanks for the support!

[@chandu-mu2cg]

but how do we know which meta learner to choose??

[@priyadoesdatascience5141]

excellent video! I am going to use it in my model. I have just one question? How can it predict better than a single model? Is it because of the inputs from the different models?

[@DataProfessor]

It’s an ensemble of several classifiers, think of it like a team of judges helping to decide together. And yes it uses the predictions from individual classifiers to make a final single prediction

[@priyadoesdatascience5141]

@@DataProfessor Excellent sir thank you! If I want to tune the hyperparameters, should I do it individually for each model and then supply it in the meta algorithm?

[@DataProfessor]

@@priyadoesdatascience5141 Yes, exactly. The video shows the use of default parameters for the individual classifiers.

[@joaomaia2898]

Thanks for this.. The better way to make data science is making data science... Recomends a method to select models to input in stack? Only models performs better?

[@Мага123-о2о]

One of the most useful videos in my life! Sure wont be able to find more convenient explanation of models stacking. Thank you professor! But I have a question, is it possible to visualize feature importances after stacking?

[@chadgregory9037]

I don't think it makes sense to "look after stacking", because each model is like an independent set which relies on its own features... but the whole is just a sum of the parts.... So I guess technically speaking, if you want a convoluted way of evaluating feature importance, you could probably do some kinda stats based on each prediction, and which submodel was most accurate to that model, and then take like, the top 3 submodels, and compare features across them. I could be entirely wrong here, lol, but it's my experience that machine learning stuff is quite intuitive. So just by that I have a tendency to feel like it doesn't make sense to see a feature importance after stacking, since the stack relies on each individual model, and for any particular prediction one submodel might be superior to others. It's a very interesting thing to think about though!

[@transferlearning6983]

Tanks a lot for this helpful video, i was wondering on how we can use a loaded models(already pre-trained) as estimators ?

[@MegaBoss1980]

Hi. Can we do level 2 meta model? Any references? Also can we insert new training data in meta model? Any references if yes?

[@praveen2112]

Sir, Here in stacking how do we know our final estimator as logistic regressor???

[@ebenezeragbozo]

because we are dealing with a set of continuous values (i.e. the results from all the models combined)

[@praveen2112]

@@ebenezeragbozo it can be any regressor now sir such as random forest, linear regressor or decision tree regressor... How can we pick the best one as final estimator among them?

[@ahsanm.5040]

Dear Professor, Could you please help me write python code about how to stack PROPHET and SARIMA univariate regressor models to predict better

[@shreyanhce315]

hello sir thanks a lot I have a doubt however , if we choose to use our own csv , which column should be our y ?? SET

[@DataProfessor]

Hi, there's many solutions to this but the easiest is to set Y by using df.Y or df['Y'] (given that your Y variable is called "Y")

[@shreyanhce315]

@@DataProfessor thank you so much and please make more videos they are very valuable 🤩🤩🤩🤩🤩

[@khedirzakaria1916]

great vide, Why didn't we use hyperparameter?

[@vision3309]

sir, I am getting a error in every fit function like , knn.fit(X_train, y_train), when I am using my own data set. I am showing you the error. can please provide any solution for that. ValueError Traceback (most recent call last) in () 4 5 knn = KNeighborsClassifier(3) # Define classifier ----> 6 knn.fit(X_train, y_train) # Train model, x-> features y->classification of flowers 7 8 # Make predictions 2 frames /usr/local/lib/python3.7/dist-packages/sklearn/utils/multiclass.py in check_classification_targets(y) 196 "multilabel-sequences", 197 ]: --> 198 raise ValueError("Unknown label type: %r" % y_type) 199 200 ValueError: Unknown label type: 'continuous'

[@ashwinig8273]

hello sir its was wonderful video very informative sir can u please suggest me the best denoising network can we stack different denoising algorithms in same manner?

[@bambangSiswo]

Good post

[@boulaabimeher5891]

But why stack haven't the good result? , I think it was the same of RF results. So if it will take the best results why it's not 1 for all the metrics ? Thnx a lot

[@divyakarade4378]

Nice video..Thanks :)

[@DataProfessor]

You're welcome!

[@khedirzakaria1916]

great video