AdaBoost in Python - Machine Learning From Scratch 13 - Python Tutorial

[@anudeep.20]

At 21:37, Is it 'error' or 'min_error' in the calculation of alpha? If it is 'error', then why are we calculating the 'min_error' ?

[@patloeber]

Thanks for this catch! It is indeed min_error that we should use : clf.alpha = 0.5 * np.log((1.0 - min_error) / (min_error + EPS)). Btw if you find errors you can double check with the code in my repository: github.com/python-engineer/MLfromscratch. Sometimes the code is a little bit more polished there

[@alitaangel8650]

This video is awesome, I've watched lots of tutorial videos and materials, and found that walking through the basic implementation of an algorithm with viewers is absolutely the best way to help them grasping the gist. Hope you can make more videos like this. Thank you.

[@patloeber]

thanks! I try to do a multilayer perceptron in a few weeks...

[@amrdel2730]

YEAH ITS A GREAT IDEA TO ALLOW THE LEARNER TO SEE FOR REAL THE STEPS OF FUNCTIONNING OF THE ALGORITHM IT IS A BETTER WAY TO UNDERSTAND IT AND TRY TO REPRODUCE IT ON YOUR OWN

[@nftk5413]

you have been a great help .the way you explain things simply is really great ,keep on. tnx alot

[@patloeber]

I'm glad you like it :)

[@SunilPatil-is6dn]

Make a video on bagging

[@syedsabeeth7996]

what about multi-class classification instead of binary? What changes should be made in that case?

[@Soninmike]

Why at 23:32: we divide weights (w) by the sum of updated w? As we saw on a formula, it should be the sum of non-updated w.

[@fedorlaputin9119]

потому что сумма весов должна быть равной единицы, а сумма только что обновленных весов не равна 1, поэтому мы их и масштабируем в интервал от 0 до 1

[@mohammadrahimpoor513]

Thank you for your informative Video!!

[@mosesmbabaali9381]

Is this a typo on 12:47? when [X_column < threshold] = -1 and also when [X_column>threshold] = -1? Isn't the else part supposed to be 1?

[@patloeber]

No this is not a typo. We start with an array full of 1s. And this here sets some values to -1 depending on the polarity. If you compare with the 2D plots I showed in the beginning, this basically tells us if the left or the right side from our decision boundary should be negative...

[@mosesmbabaali9381]

@@patloeber Would it be okay to have your notes for this tutorial?

[@minzhang7409]

Hi Python Engineer, I have almost watched your 3 tutorials. Thanks for your nice job! It helps me a lot. Are you interested in doing a series for algorithm?

[@patloeber]

Thanks for watching! For which algorithm?

[@minzhang7409]

​@@patloeber I mean computer science algorithms, such as BFS, DFS...... (By the way, will you continue to do ML tutorials? Such as the gradient boost, xgboost and so on....)

[@karmasince94]

Is this illustration similar to the calculation of Gini impurity for identifying weak learners?

[@arezu7382]

Hi, thanks for sharing it. how i can use AdaBoost for selecting feature? my data set is image and a file include features. i need your help

[@tulgaa1114]

bro i got question what is connection with your code between ddos attack detection?

[@manishgaurav84]

Hi Amazing explanation. I understand it takes lot of efforts to make these tutorials. I would really appreciate if you could help us with another tutorial for Gradient boost

[@patloeber]

i'll have a look at this

[@Swan584]

I get " predictions[X_column > self.threshold] = -1 IndexError: too many indices for array" when I try this code. Cant seem to figure out why

[@redhwanalgabri7281]

me, too

[@TheZombiebrainz]

did you ever figure this out?

[@amrdel2730]

canwe use adaboost with a weaklearner other than decisio stumps // like example svm weakened or neuralnet ?? if so can you show us an example

[@grimonce]

Going to do XGBoost from scratch video or article? :) That's quite enjoyable to watch. Awsome work!

[@patloeber]

Thanks! Yes it is on my list for the future

[@finderlandrs7965]

I see that many Adaboost approaches are very "hard coded" to the usage of decision stumps, and unfortunately cannot be applied to other cases. It'd be great if you could show us a more generic way to code it (for other types of weak learners). Cheers!

[@patloeber]

Thanks for the suggestion! I'll look into that

[@richkhid1298]

Great video. Can I use this tutorial to predict students' performance?

[@patloeber]

I don't know what your dataset is, but I guess you can

[@richkhid1298]

@@patloeber Am using a custom dataset and I can't figure how how to get the X and y variables from the dataset

[@richkhid1298]

@@patloeber Help will be appreciated...Thank you

[@revolutionarydefeatism]

@@richkhid1298 X is your features, like students' marks, their performance, etc. and y is the thing you want to predict, like the final GPA.

[@ivanong2857]

Can you use adaboost for arduino nano?

[@Ivanskiful]

It is possible to improve runtime alot by simply just removing those thresholds of the decision stumps which have a neighboring threshold that is the same sign, before running the greedy search. Why? Example where we have 4 thresholds: + - - + Splitting as following: + | - - + is obviously better than: + - | - +

[@ray811030]

could you add gradient boosting

[@HuyNguyen-fp7oz]

great video

[@patloeber]

Thanks :)

[@hamzahal-qadasi1771]

If you made your algorithm with class 0 and class 1, it would be easier to understand. But anyway thank you for this informative video.

[@souviktewary8913]

Hi I have a question, I am using adaboost, but before that on a separate program I have already made several decision stumps which I want to use in adaboost algorithm, I have stored them all together on a csv file, so that I can use it as a list. My question is here you are using prediction values as -1 or 1 for the adaboost training part, what if the predict_proba value aquired from the decision tree be used in adaboost? example my aquired decision tree list consist of predict proba values of class 0 and class 1 (such as for patient 1 being in class 0 predict proba value is 0.3 and the same patient being class 1 is 0.7)

[@patloeber]

why not just convert your 0s to -1s? np.where(x==0, -1, x)

[@ccuuttww]

why flip the error

[@fedorlaputin9119]

did you get the answer?

[@redhwanalgabri7281]

('Accuracy:', 0)