XGBOOST in Python (Hyper parameter tuning)

[@jaripeltola]

This presentation is the best overall view on the most important XGBoost model parameters I have seen.

[@DataMites]

Thank You

[@wanneesalkhayyali8386]

Indeed!

[@kmnm9463]

Hi Ashok, What I think is, there is no need to check for training accuracy. This is a redundant approach. The reason is the model is trained on the training data. So obviously the accuracy, whatever the hyperparameter tuning we do, is more likely to be close to 1.0. The better approach is to just focus on the test data. In real time scenario , for a problem statement, we would be feeding unseen data to the model and then fine - tune the hyper parameter. Thanks for the tutorial. Thanks from KM

[@DataMites]

Thank you

[@darrencr1987]

Just for discussion… I think the purpose for calculating training performance is to compare it with test performance and see if there is any overfitting, otherwise how would you know ? Also I don’t think accuracy is a good measure here, AUC might be a better one, just my 2 cents

[@prakharbaheti4055]

Great tutorial , exact and to the point.

[@DataMites]

Thank you!

[@satishb9975]

Thank you and excellent way with detailed elaboration, of each parameters for Hyper parameter tuning) explained very well, finally in got the topic of hyper parameter tuning concept

[@DataMites]

Thank you, Keep Supporting

[@dehumanizer668]

Nice one 👍🏼

[@DataMites]

Thanks

[@sugandhchauhan1900]

Great video. Could you help me fine-tune my model, please? I am getting really low training and testing accuracy?

[@DataMites]

How can I help you?

[@sugandhchauhan1900]

@@DataMites I have messaged you on LinkedIn 😊

[@carolinnerabbi965]

Very good explanation and test strategy, thanks!

[@DataMites]

Glad it was helpful!

[@nasifosmanshuvra8607]

Great explaplnation Sir! How can I provide batches of Images by using data generator for image dataset to Xgb classifier model to fit images and labels ??

[@DataMites]

Kindly refer this: github.com/bnsreenu/python_for_microscopists/blob/master/195_xgboost_for_image_classification_using_VGG16.py

[@PhenomenalInitiations]

Sir I want to use sotmax as objective, I have 4 dependent varaibles. How to make xgboost understand that there are 4 such variables? Pls reply.

[@DataMites]

Hi Sai Akhil Katukam, Thanks for your comment. If you want to use softmax and define the number of class in xgboost you need to put the following parameter while building the model... from xgboost.sklearn import XGBClassifier XGBClassifier(objective= 'multi:softmax', num_class=4,...)

[@arjungoud3450]

There is explanation of what they. Hoping you would a video in more detail

[@DataMites]

sure, will do that

[@gauravrajpal1994]

Very good explanation and test strategy, thank you so much sir

[@DataMites]

All the best

[@analuciademoraislimalucial6039]

Thanks Teacher. Love it explanation

[@DataMites]

You're welcome!

[@NextVersionOfYou]

Thank you. What's said regarding random state... true for regression problems as well?

[@DataMites]

Hi , yes Heshini

[@estebanbraganza1067]

Amazing video it would be better if you could use a different dataset so we can see the effects of the different parameters better.

[@DataMites]

Sure will do that since this is to explain you basic concept.

[@AkshayArbune]

Very helpful Video

[@DataMites]

Glad it was helpful!

[@welcomethanks5192]

WHy your digital pad can have pressure? my wacom intuos doesn't?

[@DataMites]

Can you reframe your question?

[@welcomethanks5192]

@@DataMites I mean you are writing something with a digital pad and the words you write can have different thicknesses. But my digital pad only works like a marker pen(all same thickness)...

[@DataMites]

@@welcomethanks5192 You will have an option to change the thickness

[@qazdata-science4420]

Amazing Tutorial!!!!

[@DataMites]

Thanks!

[@madhur089]

Thank you this helped in understanding

[@DataMites]

Glad it helped!

[@majorcemp3612]

Hi, what about gamma, don't you use it ? I think it's the only important missing here.

[@vikasrajput1957]

I guess since he is already using max_depth just 2-3, he doesn't need much of a pruning parameter for the trees, I guess. Your thoughts?

[@majorcemp3612]

@@vikasrajput1957 Surely, and you can tune parameters differently with gamma too, I just think in term of education he should mention it 😅😊

[@DataMites]

Please refer stats.stackexchange.com/questions/418687/gamma-parameter-in-xgboost

[@prakashaiml8423]

EXCELLENT EXPLANATION..

[@DataMites]

Thank you!

[@carlmemes9763]

Sir this problems also in the gradient boosting? Am i correct? If it in, we can do as you explained. If no, what have we da sir? Thank you sir, your videos are amazing ❤️

[@DataMites]

Hi, Thank you for your comment, can you clarify which problem you are trying to figure out?

[@carlmemes9763]

@@DataMites overfitting sir....

[@xolanijozi8375]

This is great.

[@DataMites]

Thank you

[@pradeepsharma30]

This is amazing stuff!!

[@DataMites]

Thank you!

[@kuox0005]

It appears that the target variable, y, is limited to nx1 array for making predictions using XGBOOST. Could the target variable, y, be a nXm, where m > 1, array ?

[@DataMites]

Yes possible, you can use multioutputregressor as a wrapper on xgboost

[@gauravverma365]

Such an informative video about the tunning of xgboost hyperparameter. My question is, can we extract mathematical equation for the input and output parameters. For instance, I have successfully applied Xgboost regression to predict y parameter using X1, X2, X3, X4 input parameters, now how can I get the xgboost's predicting equation between those input and output parameters. Please provide the information in this manner

[@DataMites]

No we cannot extract mathematical equation

[@avaolsen1339]

Thank you, Mr. Veda! This is really helpful. I have a question: is there an efficient way to tune these parameters automaticall?.

[@DataMites]

Hi Ava Olsen, you can automate the tuning of hyper parameter using python scripts. Or you can have a look in automl.

[@youmadvids]

@@DataMites Hi, what about GridCV?

[@allalzaid1872]

grid search cv

[@avaolsen1339]

But grid can is resource/time consuming. Is there an efficient way to do it?

[@wimavlogs6826]

can you do a full video of time series forecasting for any future prediction using previous data? (Using XGBoost)

[@DataMites]

We will definitely do in future. Thank you

[@rafsunahmad4855]

Is knowing the math behind algorithm must or just knowing that how algorithms works is enough? please please please give a reply.

[@DataMites]

"Hi Rafsun Ahmad, thanks for your comment. It is necessary to know the math and other background behind any algorithm so that you will have better idea on why and how that algorithm should be used."

[@rafsunahmad4855]

Thank you very much

[@vikasrajput1957]

increase learning rate makes the algorithm learn faster but at the cost of accuracy and does not dicrease the sensitivity contributed by a single point by a great amount, and thus does not generalises the model well and leads to overfitting in some cases

[@DataMites]

That is what convergence of algorithm means.

[@Nixterrex]

Thank you! Are the parameters for XGBClassifier similar for the XGBRegressor? I can look at the documentations on my own, but it’s late at night for me and I can’t sleep thinking about it but i also don’t want to get sucked back into my project (i fixate XD) and i need to sleep hahah… Thank you again though! The video really helped me. I’m only 3 months into learning data science with python so it feels good every time i finally piece things together.

[@DataMites]

Hi Niko Blanco, yes you can find some similar parameters in XGBClassifier and XGBRegression. Thank you

[@praveenk302]

What is min_child_weight and its significance?

[@DataMites]

Hi, please refer to this documentation. xgboost.readthedocs.io/en/latest/parameter.html

[@prajothshetty6848]

great video sir! straight & to the point explanation. sir where is the link to the code report or the repository?

[@DataMites]

We request you to pause video and type the code and will soon update the code in the description

[@davintjandra4226]

Hey, i ve got a question, say if i use a correlation matrix, and manually deselect the feature that are ambiguous(neutral), can i still put the col sample as 1?? Great tutorial man

[@DataMites]

Yes you can but check how you model performed.

[@2broke2code]

Starts at 14:50

[@mdfahd1795]

Keep it up bro

[@DataMites]

Thank you!

[@souptikmukhopadhyay6531]

If your train accuracy is 1 and test accuracy is 0.97 how can you say that the model is overfitted ? The model is clearly performing very well on the test data. What you can do is perform k-fold cross validation to be more sure that it gives high accuracy on various test sets .... But having high train and test accuracies is not overfitting, it means that the data is relatively simple for the model to learn.

[@DataMites]

Yes it could be a simple dataset. But we can validate this model using cross validation to see if model overfits.

[@planetscore]

What a chaos!

[@ltrahul1016]

nice

[@DataMites]

Thank you.

[@johnmasalu8703]

Fruitful and informative training, please share your email, for clarifications on some of the issues

[@DataMites]

"Hi John Masalu, Thanks for reaching to us. You can share all your queries and doubt here in the comment section, we will reply in the comment itself."

[@wangrichard2140]

perfect！

[@DataMites]

Thank You!

[@dineshpramanik2571]

please keep your microphone near your mouth...can't hear properly

[@aiinabox1260]

Training accuracy was 1 don't u think it's a overfit

[@DataMites]

Yes. Hyperparameter tuning will help to overcome that. But as said, this is a very small dataset.

[@shashankgpt94]

you could have chosen a better dataset

[@DataMites]

Hi Shashank Gupta, thank you for your suggestion but this dataset is working good for this task.

[@nassimbouhaouita1697]

the data was too easy for the model

[@DataMites]

Yes. This video is to focus on hyper parameters of XGBoost.

[@lextor99]

Better to make this on a real dataset, that's how this video could be better.

[@DataMites]

Aleksei, Do you mean a large dataset? The one used in this video is a real dataset, contributed by the University of Wisconsin in 1995. ref: archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Diagnostic)

[@lextor99]

@@DataMites, Yeah I mean something more realistic and more challenging.

[@DataMites]

@@lextor99 Sure.

[@nathan_falkon36]

it's enough for it's teaching proposal i think

[@wimavlogs6826]

can you do a full video of time series forecasting for any future prediction using previous data? (Using XGBoost)

[@DataMites]

Sure till that time keep checking our channel for more videos