Gradient Boosting and XGBoost in Machine Learning: Easy Explanation for Data Science Interviews

[@emma_ding]

Many of you have asked me to share my presentation notes, and now… I have them for you! Download all the PDFs of my Notion pages at www.emmading.com/get-all-my-free-resources. Enjoy!

[@SanuSatyam]

Thanks a lot. Can you please make a video on Time Series Analysis? Thanks in Advance!

[@anand3064]

Beautifully written notes

[@PhucHoang-ng4vh]

just read out loud, no explanation at all

[@emmafan713]

I am confused about the notation, so h_i is a function to predict r_i and r_i is the gradient of the loss function w.r.t the last prediction F(X). so h_i should be similar to r_i why h_i is similar to gradient of r_i

[@Heinz3792]

I believe there is an error in this video. r_i is the gradient of the loss function w.r.t. the CURRENT F(X), i.e. F_i(X). The NEXT weak model h_i+1 is then trained to be able to predict r_i, the PREVIOUS residual. Alternatively all this could be written with i-1 instead of i, and i instead of i+1. TLDR: Emma should have called the first step "compute residual r_i-1", not r_i. And in the gradient formula, she should have written r_i-1.

[@jennyhuang7603]

For 5:10, why the MSE delta r_i is Y-F(X) instead of 2*(Y-F(X))? or is the coefficent doesn't matter?

[@zhenwang5872]

I usually watch Emma's video when I doing revision.

[@annialevko5771]

Hi! I have a question, how does the parallel tree building work? Because based in the gradient boosting it needs to calculate the error from the previous model in order to create the new one, so I dont really understand in which way is this parallelized

[@shashizanje]

Its parallelized in such a way that , during formation of tree , it can work parallel....means it can work on multiple independent features parellaly to reduce the computation time....suppose if it has to find root node, it has to check information gain of every single independent feature and then decide which feature would be best for root node...so in this case instead of calculating information gain one by one, it can parallely calculate IG of multiple features....

[@MahdiShayanNasr]

An excellent video

[@aaronsayeb6566]

there is a mistake in the representation of algorithm. the equation for ri, L(Y, F(X)), and grad ri = Y-F(X) can't hold true at the same time. I think ri= Y-F(X) and grade ri should be something else (right?)

[@kandiahchandrakumaran8521]

Excellent video Many thanks. Could you kindly make a video for time to event with survival SVM, RSF, or XGBLC?

[@jet3111]

Thank you for the very informative video. It came up at my interview yesterday. I also got a question on time series forecasting and preventing data leakage. I think it would great to have a video about it.

[@Leo-xd9et]

Really like the way you use Notion!

[@emma_ding]

Thanks for the feedback, Leo! I tried out a bunch of different presentation methods before this one, so I'm glad to hear you're finding this platform useful! 😊

[@elvykamunyokomanunebo1441]

Hi Emma, I'm struggling to understand how to build a model on residuals: 1) Do I predict the residuals and then get the mse of the residuals? What would be the point/use of that? 2) Do I somehow re-run the model considering some factor that focuses on accounting for more of the variability e.g. adding more features(important features) which reduce mse/residual? Then re-running the model adding a new feature to account for remaining residual until there is no more reduction in mse/residual?

[@poshsims4016]

Ask Chat GPT every question you just typed. Preferably GPT-4

[@Heinz3792]

It's important to understand what the residual is. The residual is a vector giving a magnitude of the prediction error AND the direction, i.e. the gradient. Thus, regarding your questions: 1) we predict the residual with a weak model, h, in order to know in what direction to move the prediction of the overall model F_i(X) so that it is reduced. We assume h makes a decent prediction, and thus we treat it like the gradient. 2) we then calculate alpha, the regulation parameter, in order to know HOW FAR to move in the direction of the gradient which h provides. I.e., how much weight to give model h. Minimizing the loss function gives us this value, and keeps us from over or undershooting the step size.

[@objectobjectobject4707]

Okay subscribed !

[@wallords]

How do you add L1 regularization to a tree???

[@ermiaazarkhalili5586]

Any chance to have slides?

[@NguyenSon-ew9wn]

Agree. Hope to have that note

[@emma_ding]

Yes! Download all the PDFs of my Notion pages at emmading.com/resources by navigating to the individual posts. Enjoy!

[@nihalnetha96]

is there a way to get the notion notes?

[@MacadamMarcus-y1x]

Miller Christopher Clark Brian Hernandez Daniel

[@RichardLentz-g5k]

Clark Maria Lopez Gary Lopez Christopher

[@riswandaayu5930]

Hallo Miss, thankyou for the knowledge, Miss can I request your file in this presentation ?

[@ellazeaustin1404]

Rodriguez Timothy Taylor Christopher Perez Gary

[@JamesaGray-b1l]

Lee Angela Garcia Richard Walker William

[@MacadamMarcus-y1x]

Jones William Martinez Melissa Brown Patricia

[@SrabontyAfrin-f8p]

Smith Mary Clark George Johnson Ronald

[@BuckleLynn-v7s]

Jackson Carol Smith Eric Lopez Kenneth

[@MillDonald-g3p]

Thomas Jessica Harris Cynthia Wilson Timothy

[@khguhxjddVudhijo]

Lopez Jennifer Clark Joseph Martin Matthew

[@patrickgilbers3160]

Clark Kevin Taylor Sandra Lee Thomas

[@BurneJonesClaire-b1v]

Perez Paul Williams Donna Williams Paul

[@EdwardHernandez-l4z]

Thomas Scott Lewis Paul Lewis George

[@RanjsaBiswase]

Thomas Kevin Brown Sarah Martin Thomas

[@LisaRamos-y4s]

Jones Matthew Brown Jennifer Williams Angela

[@richardpaterson3888]

Garcia Sandra Davis Jose Walker Jennifer

[@ExGuess-w1i]

Anderson Mary Clark Richard White Nancy

[@GraceElla-f2x]

Lee Ruth Martinez Robert Jones Linda

[@SimonMaria-y7r]

Smith Lisa Rodriguez Eric White Kenneth

[@rosalindwalkman5742]

Lewis David Thomas David Davis Deborah

[@WendellSaddler]

Anderson Deborah Young John Davis Laura

[@JustinGreek-b4g]

Lewis Mary Gonzalez Linda Moore Barbara

[@EisenhowerMarshall-q4j]

Clark Paul Harris Karen Moore Shirley

[@DawnIngram-e8x]

Lee Amy Young Donna White Sarah

[@MaxPenelope-w4j]

Anderson Betty White Steven Smith Gary

[@PitmanBoyd-m9z]

Johnson Amy Anderson Kevin Miller Susan

[@HarrodQueena-p8o]

Lee Barbara Lewis Helen Clark Christopher

[@BettyBarry-u2m]

Davis Linda Davis Sarah Gonzalez Dorothy

[@Susan-l5n7d]

Taylor Elizabeth Taylor Mary Davis Michelle

[@JeffreyWest-p2u]

Taylor Donald Lee Anna Taylor Mary

[@faisalsal1]

She just read the text with zero knowledge about the content. U no good.

[@TnikusaMoni-i7p]

Wilson Angela Rodriguez Sharon Brown Ronald

[@MimiKnight-n7u]

Moore William Clark Christopher Wilson Ruth

[@ShirleyWheeler-j2o]

Robinson William Hernandez Dorothy Anderson Jason

[@KarenPerez-w9d]

Taylor Jeffrey Moore Cynthia Lopez Anthony

[@jacquelinecroftoon2071]

Miller Helen Garcia Charles Gonzalez Sarah

[@TruckJob-t5h]

Hernandez Jessica Lee Barbara Davis Anna

[@CharlesJackson-i7q]

Jackson Kimberly White Timothy Thompson Gary

[@harrisongibbse7663]

Jones Jessica Brown David Hernandez Michael

[@StoweLauren-t1c]

Martinez Jessica Lee Donna Young Richard

[@agathakeatase9243]

Gonzalez Shirley Walker Brian Harris James

[@GuntherPatrick-o3u]

Lee David Lewis Matthew Harris Shirley

[@marisafraser3404]

Lopez Mark Walker Larry Wilson Donald

[@marisafraser3404]

Rodriguez Michelle Harris Paul Lewis William

[@LewisHughes-d2o]

Martinez Barbara Miller John Davis Mark

[@SaimirBogdani-t6j]

Gonzalez Jason Brown Barbara Hall Ronald

[@Melissa-zi4xq]

Hall Robert Taylor Carol Brown Shirley