Understanding Pipeline in Machine Learning with Scikit-learn (sklearn pipeline)

Often in Machine Learning and Data Science, you need to perform a sequence of different transformations of the input data (such as finding a set of features or generating new features) before applying a final estimator. Pipeline gives you a single interface for encapsulating transformers and predictors to simplify the process. Since transformers are usually combined with estimators for preprocessing, using pipeline in scikit-learn can be really useful. To be precise, Pipelines sequentially apply a list of transformers and a final estimator. Therefore, the purpose of the pipeline is to assemble several steps that can be cross-validated while setting different parameters. In this video, we discuss the Python implementation of Pipeline using a polynomial regression example.
#Transformer #Estimator #Pipeline

[@AnkitGupta005]

Short and crisp. Thank you!

[@fabianaltendorfer11]

that's a great introduction to pipelines! Thanks

[@DrDataScience]

Thank you!

[@kianaliaghat7740]

thanks for your short, useful introduction! it helped me a lot

[@DrDataScience]

Thanks for the comment.

[@Hajar1992ful]

Thank you for this useful video!

[@gabrielmarchioli4669]

Great video. Helped me a lot

[@aszx-tv4pq]

HI there, very happy with this channel could you explain a bit simpler what is pipeline part!

[@maxwellpatten9227]

This is excellent. Thank you

[@DrDataScience]

Thanks!

[@muhammadjamalahmed8664]

Love your tutorials..

[@DrDataScience]

Thanks!

[@sebacortes8812]

muchas gracias saludos desde chile!!

[@DrDataScience]

Gracias!

[@hiba8484]

Thanks, its really helpfull

[@DrDataScience]

Thanks for watching this video!

[@nachoeigu]

I have a big one question: What is the difference of build a Machine Learning application with Pipeline and to build a machine learning application with a OOP technique? I see that it is the same.

[@DrDataScience]

Everything in Python is defined as a class so we use OOP all the time. Pipeline provides a nice flexible way to combine multiple transformers and an estimator.

[@burakakay6632]

Thank you :=}

[@DrDataScience]

You are welcome!

[@adiver_]

hello As you have imported polynomial features and transformed the independent variable(X_train) for it be fitted in a polynomial regression then why did you put linearregression() as the estimator in the last tuple of the list?? shouldn't you have use polyfit function or something else? NOTE: I am a beginner here , so the doubts can be silly.

[@DrDataScience]

Good question! We have already created all the polynomial terms that we need, i.e., x, x^2, x^3, etc. Thus, we can now view this as a linear regression problem with respect to the "new/artificial" features.

[@adiver_]

I appreciate your reply , it cleared exactly what i was asking. Thanks @@DrDataScience

[@adiver_]

@@DrDataScience one more thing I need to ask if you can spare some time, I have seen people do parameter scaling using StandardScaler() before polynomial features and estimator in a Pipeline argument, so is the scaling a necessary step or we can skip it??

[@rishidixit7939]

Why are all arrays converted to column matrices while applying sklearn

[@DrDataScience]

Because each column corresponds to a feature or attribute of your data set. Thus, the number of elements in that column vector is equal to the number of samples.