Scikit-Learn Model Pipeline Tutorial

[@GregHogg]

Take my courses at mlnow.ai/!

[@TheCsePower]

Thanks Greg. This made me realise how non-standard my code is. I learnt: - Use copy or deepcopy and not assignment. - Always perform preprocessing on the train and test separately. - sklearn pipelines have nothing to do with ETL pipelines from Data Engineering. - sklearn transfers have nothing to do with NLP Transformers. - sk elarn estimators have nothing to do with Statistics estimators.

[@GregHogg]

Super glad you got some useful pointers!!

[@crepantherx]

Keep Posting Greg, I am Data Analyst by profession and your video certainly helps a lot

[@GregHogg]

That's awesome! Thank you 😄

[@hansenmarc]

Great stuff! I’m curious why you used FunctionTransformer instead of ColumnTransformer, which could run the two scalers in parallel? Also, since FunctionTransformer is stateless, the documentation says that fit just checks the input rather than actually fitting the scaling parameters. Doesn’t that lead to data leakage since applying transform to test data won’t use parameters learned from fitting on the training data?

[@kyleGrealis]

thanks, Greg. really good explanation and structured example. this makes it easy to create a template for easy reuse!

[@brandonn8166]

Just out of curiosity, is there a reason you don't use train_test_split to get X and y values?

[@NikitaShilyaev]

yes, why he uses X_train for train_predictions instead of another dataset X_valid

[@AmitabhSuman]

A very practical video, that I came across on Pipelines. Thank you for this video!

[@GregHogg]

Awesome that's great to hear. You're very welcome ☺️☺️

[@ilanyutsis9653]

When you do the StandardScaler().fit on the dataframe, what is the meaning of this operation? what is happening?

[@alexrook5604]

I undstand what you are doing here but I have two questions that I think would be helpful and would make it easier to follow along and replicate you steps. 1) Where did you get the data. I can't the california_housing dataset that is already in the train/test form. 2) Why not use scikit-learn tooling rather than doing it yourself? Like you could have used train/test split or pipelines (or column transformer... or similar stuff). That just has me confused.

[@JJGhostHunters]

Great tutorial! I use the MinMaxScaler with the option to scale from -1 to 1 instead of 0 to 1 when I am dealing with values that can be positive and negative. Seems to be fine, but I may need to reconsider going forward. I have never noticed any issues though.

[@rahiiqbal1294]

This was very helpful, thank you :)

[@JJGhostHunters]

I would love to see a tutorial that covers using pipelines with multilayer perceptron models (MLPs), CNNs and LSTMS.

[@lythien390]

Thank you Greg! It's a great video!

[@GregHogg]

Glad to hear it!

[@TheFrankyguitar]

Thanks for this amazing video! Would that work also with a statsmodels model?

[@GregHogg]

Thanks so much!! And I'm not sure, haven't tried :)

[@junaidlatif2881]

How to transform y variable and then fit model. And after how to reverse transform for the scatter plotting

[@marcofogale9719]

Perfect explanation. Thanks a lot

[@GregHogg]

Very welcome 😁

[@talyb7383]

Thanks for the great tutorial! what do I need to change to create a pipeline for an image classification model? like the cifar10 model?

[@GregHogg]

Well, everything. You probably won't be using scikit for that. And you're very welcome!

[@talyb7383]

@@GregHogg I didnt explained myself clearly... I want to create a pipeline that receives a trained cifar10 model an also make preprocessing on the e data set ? so I cant use your way?

[@Nadia-db6nb]

Thanks for the great tutorial. Can you make a video on how to combine multiple feature selection methods and feature extraction using python?

[@fabio336ful]

Did you say pipelines doesn't function for classifications problems? Min: 1:07

[@GregHogg]

Does, not doesn't

[@fabio336ful]

@@GregHogg thanks 🙏🏼

[@adriandiazNY]

Great Video!

[@GregHogg]

Thank you Adrian!

[@krzysztofzaucha3592]

nice video Greg

[@GregHogg]

Thanks so much!!

[@nabanitadasgupta]

Thank you for the video!

[@00SeijiHan00]

TYSM bro really appreciate this

[@GregHogg]

Very welcome!!

[@tareq8109]

Bro can you show how to make youtube and any video downloader make by python

[@juampaaa90]

awesome ty

[@allanmachado2011]

Thank you!

[@Supernyv]

Awesome !

[@GregHogg]

Thank you!

[@m18293]

Can you share this notebook?

[@GregHogg]

dang i think i lost it, sorry

[@AceOnBase1]

Bro you literally just copied this out of a textbook lmao but I respect the grind.

[@MrAhsan99]

you are ❤

[@GregHogg]

❤️

[@johnspivack]

Too confusing. Too many tangents, doesn't cover the main idea clearly. Downvoted.

[@GregHogg]

Well I upvoted it to counter you

[@n8trh]

What tangents? This video was not only to the point from the start, but it also went into depth with useful examples. If you thought those were tangents, I recommend watching again, maybe with more care this time.