Live-Feature Engineering-All Standardization And Transformation Techniques- Day 6

[@marlinaismail804]

u r the best sir keep up the good work

[@shreyasb.s3819]

During covid situation ur helping lot. Thanks a lot for ur help. Your simple superb and awesome topics and explanation.

[@hejarshahabi114]

amazing Indian guy, you're doing great.

[@honeysharma3169]

Very nicely explained sir, courses which are worth of thousands rupees don't teach like this. I really appericiate your work. Please keep doing these live sessions ,they are amazing

[@sudheervennapu309]

Excellent session sir

[@gh504]

Very useful information .Thank you sir

[@joansaldanha5117]

Very nice session... 👍

[@ahmeterdonmez9195]

Kindly let me add my comment: "Standardization and transformation (like log transformation) serve different purposes in data preprocessing. One of them is Scaling, another one is Transformation and they are not same concepts, They can not be grouped under same title. If we want to group, this can be under Feature Engineering" Standardization just cares values. It's often used when algorithms are sensitive to feature scales. So we convert them in a simmillar form as name suggest -Standardization- . Transformation cares the type of distribution. Even people explain it with standardization actually both have different use cases. the common point just both work on features. So how we use, when we use them? When we as developer think that some features may require Transformation, (Let's say "Salary" feature is right-skewed) We should first apply Transformation for Salary feature then Standardize it.

[@kushalhu7189]

You are the best ...😇😇😇

[@edwinjohnson8488]

Thank you very much. These classes are really helpful to me.

[@souravde2283]

You r awesome Krish !! Thank you.

[@akashprabhakar6353]

Awesome video...Thankyou very much

[@vikasyetintala2736]

only one word excellent

[@rambaldotra2221]

Extremely Helpful Sir ✨Thanks A Lot ✨

[@vaibhavyaramwar]

Thank You So Much...Your Contents are really helpful

[@dipsikhadas9051]

@Krish thank you . Entire session was very much insightful

[@ammar46]

Linear regression or any other algorithms doesn't assume the feature's distribution to be normal. We convert it to normal just to avoid over fitting because of outliers.

[@dushyanthkumar8533]

Thank you. It's amazing session.

[@ankitac4994]

Mast session tha

[@ashiqhussainkumar1391]

It's already done sir in a 20 minute video

[@priyanshusharma2516]

Amazing stuff Sir , keep it up .

[@sandipansarkar9211]

finished watching

[@manojrangera]

For right skewed use log transform.. And for left skewed use square transform

[@srishtikumari6664]

Worth watching this session!

[@priyasai234]

We can also use df['fare_log']=np.log(df['Fare']+1) whenever we have zero values

[@Abhisheksingh-sk2fn]

as same as logp1 -real-valued input data types, log1p always returns real output.

[@hariKrishna-bg2gm]

sir its amazing session

[@aryanmukherjee5659]

thanks a lot for the beautiful video....liked very much... could u please help me to understand what is Jhonson Transformation and when it is used and the python code to run the same...

[@tanmaychopade647]

Do we have to standardise the data after converting data to Gaussian distribution by any transformation technique

[@AbcAbc-kx3xm]

I have only one confussion that is in Exponential Transformation df["Fare_exp"]=np.exp(df["Fare"]) plot_data(df,"Fare_exp") I wanna apply this code instead of Krish's but there is a complete difference between them, what is the problem?

[@shyamsundarramadoss3567]

Hi. I see lots of pre-processing and processing steps involved in modelling. Is there any generic steps in order to do provided if needed. I meant can you pls provide a sequence in which steps like the following has to be done to get the well performant model from the lot?? missing values treatment polynomial features addition scaling normalization correlation / multicollinearity check pca/lda/da/fa dimension reduction modelling cross validation hyper-param tuning (grid/random search) model calibration report generation is the above order correct in sequence and is there any of the above steps which can be switched if needed and what all steps have strictly need to be in the specified order? Can you pls elaborate on this? Above was thought of from a regression problem standpoint, even though maybe some of them might apply to classification as well.

[@vaishaliyadav9860]

after gaussian transformation did we require to do scalling

[@parthadx7ster]

Hi Krish Do you gavd a video on encoding with ecxmes on writing the code. Will highly appreciate.

[@gurdeepsinghbhatia2875]

Why Dislike i dont understand

[@joansaldanha5117]

They r ungrateful...

[@varungowda6521]

Sir in my data there is some columns data are right skewed and some column data are normally distributed should i apply both gaussian transformation for both the columns or only for right skewed column

[@pradumnar870]

Why are we transforming the encoded variable

[@manojrangera]

Fare is right skewed.

[@naveenrajan3765]

Is Normalization and Scaling same?

[@priyam66]

normalization is a type of scaling..:)

[@ashishbhagchandani6817]

can we use different feature engineering methods to the same dataset for different columns?

[@Abhisheksingh-sk2fn]

Q-Q plot also impute outliers?

[@sivachaitanya6330]

can you please tell what is range of standardscalar ?

[@santhoshkumarmatlapudi2851]

-1 to 1

[@suryagangadhar1735]

Quantile is nothing but quarter or 1/4

[@sudanmac4918]

could you please help me on this When to apply normalization and standardization before or after splitting the train & test data? still i didn't get correct answer from anyone. i hope you can give the answer for my question. And one request please do video on that. because many ppl applying the scaling method before splitting the data into train and test. it's ,y humble request to solve and give answer for my question.

[@imantadatascience4827]

before

[@mrigankshekhar384]

It is right skewed sir when you will try to smooth the histogram so as to get probability density function we will find tail towards right side in case of fare column

[@suryagangadhar1735]

Is standard scalar ranges b/w from - 1 to 1

[@krishnaik06]

No it scales down the values based on standard deviation i.e between +3 and -3

[@suryagangadhar1735]

OK, thanks