How to use Feature Engineering for Machine Learning, Equations

[@leonardsmith9870]

Hi Jeff. I've recently subscribed and I honestly have to say you have the most comprehensive and easy to understand guides out there. Not to mention the fact that whenever there is an update to something, you make a new video explaining how to work with it. I tried getting in to machine learning just over a year ago and nobody at the time was able to actually explain anything apart from "download this, download that, if it doesn't work oh well" and would just go through the official tutorials without actually explaining how to do anything on your own. Your channel alone has given me the motivation to get started again and thank you so much for doing what you're doing!

[@HeatonResearch]

Hello Leonard, thank you for the kind words. Glad the content is helpful, and yes, it is a lot of work keeping everything up to date.

[@HarrysKavan]

Just wanted to leave a thank you Mr Heaton. I'm currently working on my bachelor thesis and your videos are a great help. Much appreciation.

[@HeatonResearch]

Happy to help! Thank you for the note.

[@jameswilliamson1726]

I read over your thesis comparing types of feature engineering vs machine learning models. Great stuff! Thx.

[@HeatonResearch]

Thanks!

[@jameswilliamson1726]

@@HeatonResearch Would standardizing or normalizing the input features give you better results? That one ratio had such a wide range.

[@HeatonResearch]

@@jameswilliamson1726 I will often standardize/norm after applying these techniques. The techniques I use here are really to capture the interaction between underlying features. Then standardization/normlization on top solves range concerns.

[@ShashankData]

I've been following you for months, thank you for the free, well explained content!

[@HeatonResearch]

Thanks!!

[@SebastianHolt-t6f]

This is incredibly intuitive! Thanks

[@khaledsrrr]

Feature Engineering Explained! 😍 This is likely the best explanation on YT. Thx 🙏

[@sheikhakbar2067]

I like Jeff's approach of giving us the big picture of he is talking about!

[@HeatonResearch]

Thanks!

[@germplus]

Fabulous explanation. In the early stages of my course ( MSc AI & Data Science ) and I find your channel very helpful. Thank you.

[@amineleking9898]

Such a practical and helpful video, many thanks professor.

[@nicolaslpf]

Amazing video Jeff ! The only thing you didn't tell us is if you then drop the source features to avoid collinearity or you just leave them along with the new features you created .... Or you perform PCA, VIF or Lasso after it to chose what to do?.... I loved the video concise and super useful!

[@daymaker_trading]

This video and presentation is amazing. Thank you SO MUCH!! All the best!

[@lakeguy65616]

excellent video of real practical use!

[@juggergabro]

At last, not another Data Science hijacker trying to prove themself on YT... Thank you.

[@StevenSolomon-jb3zi]

Very insightful. Thank you.

[@MLOps]

Super helpful! much appreciated!

[@HeatonResearch]

Glad it helped!

[@akramsystems]

This looks really fun to do!

[@korhashamo]

Awesome. Great explanation. Thank you 🙏

[@DeebzFromThe90s]

Hi Jeff, what concepts should I look into to understand "Weighting" better? For instance at 9:41, you mention that if one values food more they might square it. Someone might cube it, someone might multiply it or add a coefficient of 2 or 5. These are all subjective. For weighting when it comes to features in the stock market or econometrics (my specific application), one might have a feature that is GDP or inflation. I know for a fact that change in GDP (slope) and change in the change in GDP (slope of slope i.e., acceleration) are pretty important. My first problem, is that I found these two (change in GDP and GDP acceleration) simply through guess and check, and research papers. Is there a better method to this? Or should I focus on automating 'guess and check'? Secondly, sometimes the GDP features or inflation related features vary in importance to participants in the stock market. Perhaps right now (as of Oct 2022) investors might place more emphasis on inflation related features and so I might multiply inflation features by coefficient of 2 or square it. How would one deal with dynamic weighting? Or a simpler problem might be, how do you objectively select for weighting? EDIT: I have come up with an idea, to add a coefficient to GDP or inflation based on social media mentions (sentiment), for instance. Thoughts on this and weighting in general? Thanks so much! Love the video by the way!

[@felixlucien7375]

Awesome video, thank you!

[@yongkangchia1993]

Really valuable content that is clearly explained! keep up the great work sir!

[@bingzexu7259]

When we do feature engineering, are we expecting that the new feature has a high correlation with the predicted values?

[@HeatonResearch]

Yes for sure, so you must keep that in mind when evaluating feature importance. Generally, I leave the existing features in and let the model account for that (though some model types perform better with correlating fields removed).

[@sandeepmandrawadkar9133]

Thanks for this great information

[@jonnywright8155]

Love the energy!!!

[@HeatonResearch]

Thanks! I also went a little crazy on video editing too. lol

[@mohammed333suliman]

Great, thank you.

[@HeatonResearch]

You are welcome!

[@SAAARC]

I found this video useful. Thanks!

[@programming_hut]

💛✌️ Thanks

[@HeatonResearch]

You're welcome 😊

[@heysoymarvin]

this is amazing!

[@liquidinnovation]

Thanks, great video! Any examples on using the shap package to additively decompose regression r^2 using shapley values?

[@gauravmalik3911]

very informative

[@jhonnyespinozabryson8241]

Very thanks for sharing

[@HeatonResearch]

My pleasure

[@sumitchandak6131]

Thia is really great and something out of box. Can you please provide similiar techniques for NLP as well

[@Shkvarka]

Awesome explanation! Thank you very much! Best regards from Ukraine!:)

[@hannes7218]

great job!

[@HeatonResearch]

Thanks!

[@Oliver-cn5xx]

Hi Jeff, would you have a link to your paper and the kaggle notebook that you showed?

[@HeatonResearch]

Oh yeah, I should have linked that. I added it to the description, here it is too: arxiv.org/pdf/1701.07852.pdf

[@Oliver-cn5xx]

@@HeatonResearch Thanks a lot!

[@jamalnuman]

Very useful

[@lehaipython9242]

How should I perform Feature Engineering on anonymous variables? I cant put my domain knowledge on them

[@jifanz8282]

Informative video as always. +1 like for my professor 👏

[@HeatonResearch]

Thanks Jifan!

[@youngjoopark4221]

I am novice. The model would figure out that relationship, then creating a new feature by dividing, multuplying something is worthy to do??

[@ramiismael7502]

Can you try all different possible method to do this.

[@Jeffben24]

Thank you :)

[@ali_adeeb]

thank you so much!!

[@Knud451]

Thanks! Why would you e.g. square variables to make them more dominant in the model? Wouldn't the model just put more weight on them by themselves? Unless its because you want to make a nonlinear scaling of that variable. On a side note, isn't BMI a good example of poor feature design... 😀

[@avithaker]

Would love to see a link to your paper?

[@HeatonResearch]

Sure! Should have linked in the description. arxiv.org/abs/1701.07852

[@avithaker]

Thank you!

[@johncaling6150]

I dont remember if i asked this already if I did sorry but it would be great if you could do a tutorial about mxnet/gluon. It is a advanced library that is good for advanced things.

[@HeatonResearch]

Currently researching Gluon for such a video.

[@johncaling6150]

@@HeatonResearch Nice.

[@johncaling6150]

@@HeatonResearch I always have a hard time getting it installed. You install guides are the best!!!!

[@taktouk17]

Please show us how to customize StyleGan2 to for example generate a babyface or change the gender of someone in the image

[@HeatonResearch]

Yes thinking about how to do something with that.

[@brandonheaton6197]

Can you address Sutton's Bitter Lesson as it applies here?

[@HeatonResearch]

Kind of the limit of the Bitter Lesson, as time approaches infinity is that any program can be written by a random number generator, if we have enough compute time, and a way to verify correctness. I think the cleaver algorithms are always filling in the gap before massive compute is able to perform this operation on its own. However, I still see Kaggles won on feature engineering, so I tend to assume that it is still a needed skill. At least for now.

[@marineprost990]

7

[@SuperHddf]

thank you! :)