137 - What is one hot encoding in machine learning?

[@minipc123]

Thank you for the informative tutorial. I am learning deep learning using tensorflow as backend. May I ask whether it is really necessary to use one-hot encoding in training multiclass model? Since I understood in tensorflow there are sparsecategoricalcrossentropy (for non one-hot label) and categoricalcrossentropy (for one-hot label) for loss calculation. Hence I have an impression that both type of data structures are acceptable? Or is there a large difference in training performance? Thanks again

[@andrewmamchyn5031]

Grate video as always. If you guys are using pandas library for your data preprocessing you can use their get_dummies() function that will convert whatever categorical columns you want into one hot encoded. import pandas as pd df = pd.read_csv(“file path”) one_hot_encoded = df.get_dummies(df[“categorical column”]) It’s just a rough sketch but you can read more about it in pandas documentation. Plus if you’re let’s say using TensorFlow you can split your data into training and testing and feed it directly into network, because pandas runs on top of numpy and TensorFlow know hot to deal with numpy.

[@diegostaubfelipe4310]

I converted my image that had 2 label into one-hot, ie (0,1,2), but when I removed the first one, because I thought it was unnecessary (redundant, it presented the background), the model did not converge... my model only converges if I send all 3, including the background... does that make sense?

[@pavankrishnateja3154]

So what is the use if we train our model with this rather than a number... Will it be beneficial

[@DigitalSreeni]

You cannot use a vector with just numbers if you want to use categorical cross entropy as loss function for multiclass classification in deep learning. Many loss functions for multiclass problems require you to convert it to categorical (one-hot). If you are working with traditional machine learning (RF or SVM) you do not need to convert.

[@shilpashree4860]

Very informative videos sir. Can you please explain how to generate binary masks for segmentation of microscopic images.

[@DigitalSreeni]

This is a common question that people ask me and it is really challenging as there is no easy to use tool. Please sign up for a free APEER.com account and we plan on releasing a tool soon (mid-July 2020). By the way, APEER is an online image analysis platform that is free for academia, non-profits and individuals.

[@ZhiGangMei]

Python for Microscopists by Sreeni Dear Sir, is the tool you mentioned available now? Is there any tutorial showing how to use it? Thanks.

[@pablovaras9435]

nice video!