Machine Learning Pipeline In Python | How to run pipeline in python machine learning

Machine Learning Pipeline In Python | How to run pipeline in python machine learning
#MachineLearningPipelineInPython #UnfoldDataScience
Hello All,
My name is Aman and I am a Data Scientist.
About this video:
In this video, I talk about step by step process of implementing machine learning pipeline in python. I talk about how to use sklearn pipeline module to implement machine learning pipeline in python. Below questions are discussed in this video:
1. Machine learning Pipeline in python
2. How to run pipeline in python machine learning
3. How to use sklearn pipeline in python
4. Python machine learning pipeline
5. Machine learning pipeline tutorial
About Unfold Data science: This channel is to help people understand basics of data science through simple examples in easy way. Anybody without having prior knowledge of computer programming or statistics or machine learning and artificial intelligence can get an understanding of data science at high level through this channel. The videos uploaded will not be very technical in nature and hence it can be easily grasped by viewers from different background as well.
Join Facebook group :
groups/41022...
Follow on medium : / amanrai77
Follow on quora: www.quora.com/profile/Aman-Ku...
Follow on twitter : @unfoldds
Get connected on LinkedIn : / aman-kumar-b4881440
Follow on Instagram : unfolddatascience
Watch Introduction to Data Science full playlist here : • Data Science In 15 Min...
Watch python for data science playlist here:
• Python Basics For Data...
Watch statistics and mathematics playlist here :
• Measures of Central Te...
Watch End to End Implementation of a simple machine learning model in Python here:
• How Does Machine Learn...
Learn Ensemble Model, Bagging and Boosting here:
• Introduction to Ensemb...
Build Career in Data Science Playlist:
• Channel updates - Unfo...
Artificial Neural Network and Deep Learning Playlist:
• Intuition behind neura...
Natural langugae Processing playlist:
• Natural Language Proce...
Understanding and building recommendation system:
• Recommendation System ...
Access all my codes here:
drive.google.com/drive/folder...
Have question for me? Ask me here : docs.google.com/forms/d/1ccgl...
My Music: www.bensound.com/royalty-free...

[@froilanemeliano6551]

U definitely know what beginners want to see. Thank you so much for this sir.

[@MissWhite21]

I have been looking for this in a point to point explanation that was done by you in an approximately 10 mins videos that others are taking hours to explain. Very impressive and informative. Please keep up the good work. This going to be very helpful to all the DS Aspirants like me! 🔥🔥🔥

[@UnfoldDataScience]

Thanks Sweta.

[@mehediazad1780]

thanks for making it simple to understanding this concept.

[@cvrbcheppali8214]

Well presented with good Explination

[@ajaykushwaha-je6mw]

you break the problem in such a simple chunks that I have to words to Thank and appreciate your effort.

[@UnfoldDataScience]

You are most welcome Ajay.

[@vishnukumarcheruku134]

Thank you aman!

[@sandipansarkar9211]

finished watching

[@saharrohani5136]

Excellent tutorial. Thank you so much.

[@UnfoldDataScience]

You're very welcome Sahar.

[@ppsheth91]

Thanks Sir for such an easy explanation for Machine Learning pipelines.!

[@UnfoldDataScience]

You are most welcome Prayag.

[@rabbilbhuiyan5666]

Excellent resources and thanks for publishing such useful video among us !

[@UnfoldDataScience]

So nice of you Rabbil. Thanks for watching and motivating me.

[@achumohan5908]

Thank you so much Aman!! explaining the concept at simple and best 🙂

[@UnfoldDataScience]

Happy to help Anchu

[@dukefler]

Great video, Aman... Simply Amazing... Thanks a lot...

[@UnfoldDataScience]

Thanks a lot.

[@harshagrawal8599]

best explanation in complete youtube

[@UnfoldDataScience]

Thanks Harsh.

[@Technicalchurn]

Great 🔥🔥

[@UnfoldDataScience]

Thank you.

[@vishwalamallikarjun5790]

Hello Aman, Great info .Thanks . I wanted to know if we have to encode the test data (Fit_transform is applied on train data and transform to be applied on test data) how to add these steps in pipeline , i have to use it to deploy the model

[@magdynasr6639]

thanks a lot for this informative video!

[@UnfoldDataScience]

Welcome

[@Sidcr07]

One of the best content...!

[@UnfoldDataScience]

Thanks Siddhartha. Please share to your friends if possible:)

[@robertsprasath8901]

Great Explanation!

[@UnfoldDataScience]

Thanks Roberts.

[@Phantom26092010]

Thank you my dude, you save my life. i dont even why i have to pay tuition to my prof when he fucking shit at his job

[@UnfoldDataScience]

Glad it was helpful for you.

[@Sumit_Harsha]

Excellent 👍

[@UnfoldDataScience]

Thank you Harsha :)

[@dees900]

Great video. What about imbalanced dataset? what transformer do you use?

[@BatBallBites]

Nice Work Man

[@UnfoldDataScience]

Thanks for watching Zaid :)

[@sadhnarai8757]

Very good Aman

[@UnfoldDataScience]

Thank you 😊

[@shivamdwivedi8911]

Please make vedios on how to choose right algorithms on real world problems .

[@UnfoldDataScience]

HI Shivam, sure, thanks for Feedback.

[@Ganesh-zj1qp]

Hi bro thank you for giving greate information please make video on streamlit

[@UnfoldDataScience]

Thanks Ganesh.

[@iioiggtrt9085]

Great! I suggest a hot topic on how to create a dataset CSV file from a group of audio no one illustrated this point I think you only who have can do that

[@UnfoldDataScience]

Great suggestion! Thank you :)

[@lakshmivagadargi2541]

sir, can you please make videos on kedros framework?

[@UnfoldDataScience]

Sure Lakshmi. Noted the suggestion.

[@shadow82000]

This is indeed easy to debug and process. But may I know when do we preprocess the data before splitting and vice versa?

[@UnfoldDataScience]

Thanks for watching. For some of the preprocessing task you need to do it on train and test both for example creating dummy variable. For some preprocessing, for example scaling, this step can be done only on train and that's fine.

[@shadow82000]

@@UnfoldDataScience Do we scale dummy variables or skip?

[@user-xn8wg6yw7g]

Thank you. It's a good video and you try to be helpful. I am still puzzled by how the output from one step in the pipeline gets inputted into the next step. The names of the outputs and inputs aren't explicitly written, so how do you specify or make sure that the right elements of the output will be used the right way and interpreted correctly as the input of the next step?

[@UnfoldDataScience]

Good question, please check what pipeline module does exactly

[@ajaykushwaha-je6mw]

HI Aman, I have a doubt. If we dont user pipeline then we do train test split and prepare same kind of data on X_train and X_test. Suppose if we have created a pipeline for missing value imputation --> onehot encoding -->scaling the feature then we need to apply pipeline on train and test data ?

[@UnfoldDataScience]

Good question Ajay, When you are doing training for first time, then its a different story. When u put model in production then different. While training we must take care of data leakage etc so that learning is not biased. While in production, we just use the trained model hence the new data for prediction will have its separate pipeline, nothing to do with train data here. Hope its clear to you.

[@sameertemkar]

Hi Aman, One question..When the X test, and Y test will go thru preprocessing steps (minmaxscaler and pca) as we are only using model.score(x_test,y_test) ..but what about the preprocessing for them?

[@UnfoldDataScience]

Very good question. Yes preprocessing should happen on test/validation/prediction data as well. Here may be I would not have included for demo.

[@souravbiswas6892]

Excellent 👌. Without doing the classifier=i, can I do . format(pipelinedict(i))] at cell no 41?

[@UnfoldDataScience]

Thanks Sourav, let me check it.

[@bushrajaveria]

hi. great tutorial. could you please guide on how to measure standard deviation using pipelines just like accuracy? Thanks

[@UnfoldDataScience]

Great suggestion!

[@ajaykushwaha-je6mw]

Requesting you to prepare a tutorial regarding How to use Jupiter Note Book.

[@UnfoldDataScience]

Already uploaded Ajay based on ur request :) kzbin.info/www/bejne/f57dhaeufbuiqKM

[@sandipansarkar9211]

finished coding practice

[@krishnendubhowmick481]

So, In this pipeline where to fit data cleansing code ...?

[@UnfoldDataScience]

While you call the pipeline, before applying scaling you can add a cleaner function.

[@Technicalchurn]

Sir please create video on kaggle competition question from scratch

[@UnfoldDataScience]

Ok Sure.

[@abhishekgaurav7786]

but sir list' object has no attribute 'fit' how your compiler is doing so

[@UnfoldDataScience]

May be variable or datatype issue.

[@mlvali1350]

hi i have one doubt i have 3 suppurate files in same environment for each file have Future Engineering, Future Selection, Model Training , Model Deploying . I want to create a pipeline in Model Deploying to call other files how can i do it

[@UnfoldDataScience]

If its in python, create a .py file for feature selection and other intermediate steps, call that .py in main python script.

[@mlvali1350]

@@UnfoldDataScience ya its in python. can i know how to call it in main.py file. can i have any sample file

[@UnfoldDataScience]

Save your other script as. Py file and import it as module in main.py

[@drakZes]

Where can I find this notebook?

[@UnfoldDataScience]

drive.google.com/drive/folders/1XdPbyAc9iWml0fPPNX91Yq3BRwkZAG2M

[@drakZes]

@@UnfoldDataScience You are the best! Literally the best!

[@naveenmami7438]

Pls share notebook

[@UnfoldDataScience]

Pls check in my google drive Naveen, Link in description.

[@rohanmalik2910]

But i don't see any valid advantage of using a pipeline over our traditional coding of each models seperately..🤔 (sorry, I'm a beginner in machine learning). I have already seperately made files for these models with data preprocessing file which contains all possible steps properly highlighted using titles and subtitles in ipynb files. So I just need to copy paste everytime I get a dataset from these pre-existing code templates. I think the method I use is a very convenient one..🤷‍♂️

[@UnfoldDataScience]

Good point! That is also a good approach.

[@amoza1689]

Whats your email. Need to discuss something

[@UnfoldDataScience]

Please find it in my channel homepage.