208 - Multiclass semantic segmentation using U-Net

Code generated in the video can be downloaded from here:
github.com/bnsreenu/python_fo...
The dataset used in this video can be downloaded from the link below. This dataset can be used to train and test machine learning algorithms designed for multiclass semantic segmentation. Please read the Readme document for more information.
drive.google.com/file/d/1HWtB...
To annotate images and generate labels, you can use APEER (for free):
www.apeer.com

[@5junkmail]

Exactly what I was looking for, you are a very knowledgeable person with a great talent for explaining things!!! Please don't stop!

[@eli_m6556]

Needed this so much. Seems like every time I run into a problem with my research you put out a video answering my prayers. Thanks Sreeni.

[@kaihsiangju]

This channel deserves millions of subscribers. Thanks for the amazing contents.

[@raguramramamoorthy8569]

true very true --- he can sell this course for at least 100 dollars ...but he has done it for free ...

[@gadaanet]

Exactly!

[@jacobusstrydom7017]

Wow the best explain of these concepts I have seen in a long time. Thanks for this

[@vaveileinn8402]

The only word : Great! please keep continue Sir. thank you so much.

[@kannanv9304]

Ajarn - Can fully understand the efforts and time you are putting in to create these contents.......The real value of gold is not known to the one who wears it......It is know to the miners who take out tons and tons of slush to extract 1 ounz of gold........Pranams......You have an amazing sense of sequels.......And I am sure, you are not going to stop the sequels on U-nets with this.......

[@awaisahmad5908]

One of the best channels for Research Students of Computer Vision discipline.

[@DigitalSreeni]

Thank you :)

[@jaykumarsanandiya9499]

Thanks for Multiclass segmentation. In Segmentation or even Image related Deep Learning your Videos are best.....

[@kavithashagadevan7698]

Thank you. Your tutorials are life savers for me

[@suyashdahale4355]

Thank you sreeni for the labelencoder path, all other places it was simply -1 , but my masks were in color and i just realised that differnce after wathing this tutorial..... super helpful insight.

[@bikkikumarsha]

Thank you, your tutorials are one of the best.

[@salarghaffarian4914]

Your U-net videos are very helpful for me. I would appreciate if you could produce videos on instance segmentation as well and particularly Mask RCNN model. Thanks a lot. 🙏🙏

[@EUMikkel]

Sreeni thank you so much for all the work you put into these videos. It has helped me so much get started with segmentation

[@DigitalSreeni]

You are so welcome!

[@samuelireke238]

Thank you very much for your videos. They have been of immense help for a histopathology cell counting project I am working on. I am trying to investigate the impact of auxiliary outputs on UNets for microscopic cell detection and counting but have been stuck with a bug for over a week now. Most documentation online hasn't helped. My auxiliary outputs use various blocks of the UNet model as inputs as such output different shapes from the original input size of (256,256,3). So the main challenge is how to declare this during training so it takes this into consideration. Error Message obtained: ValueError: Error when checking target: expected aux1 to have shape (32, 32, 1) but got an array with shape (256, 256, 1) Model Summary: Layer (type) Output Shape Param # Connected to ================================================================================================== input_6 (InputLayer) (None, 256, 256, 3) 0 __________________________________________________________________________________________________ ... __________________________________________________________________________________________________ aux1 (Conv2D) (None, 32, 32, 1) 33 activation_74[0][0] __________________________________________________________________________________________________ aux2 (Conv2D) (None, 64, 64, 1) 33 activation_76[0][0] __________________________________________________________________________________________________ aux3 (Conv2D) (None, 128, 128, 1) 33 activation_78[0][0] __________________________________________________________________________________________________ original (Conv2D) (None, 256, 256, 1) 33 activation_72[0][0]

[@adhiliqbal6020]

Great job srini, I'm learning alot

[@Ninguempensonesse]

Amazing content my and of many professor of Deep Learning, I think that a nice suggestion to your next videos, could be the addition of the version of the installed libraries and modules in each notebook. That´s it thanks.

[@angelopiasentin4197]

Thanks Sreeni. You always bring new ideas to the AI world.

[@DigitalSreeni]

My pleasure 😊

[@applejuice5785]

thanks I was just working on a multiclass segmentation with Unet

[@Divya-ok1ou]

Thank you for your videos. They are very much helpful.

[@bogdanchelu5578]

Thank you very much for your video, it helped a lot. Only thing I need to ask you about is the calculation of the IoU. I have a very unbalanced dataset, and I ran the model on it several times with several different loss functions, including some that are explicitly made to handle data unbalance, but every single time the IoU confusion matrix looks as if my model classified everything as background (i.e. the most common "class"). Since I'm sure the data is correctly labelled and I doubt there can be something wrong with the model especially after running it with different functions, I think there is something wrong with the IoU calculation. Do you have any idea? Thank you.

[@vimalshrivastava6586]

Best KZbin channel for deep learning researchers.

[@DigitalSreeni]

I'm glad you think so :)

[@anandsrivastava5951]

Great!! I was in need of it badly :) Great work. !!

[@hassanmahmood7284]

thanks for the contribution, appreciated.

[@davidhresko7351]

Finally video explained to details. Thanks

[@victordiasteixeira1694]

This class helped me sooo much! Thanks a lot s2

[@allishadow8134]

I like the way that you explain the concept... I will subscribe for future excellent content.... Thank you

[@frankrobert9199]

Great lectures, I follow up with your series.

[@DigitalSreeni]

Great to hear!

[@skynetpro549]

this channel is love !! supported me a lot

[@DigitalSreeni]

Happy to hear that!

[@gadaanet]

I've just found what I was looking for. Thank you!

[@DigitalSreeni]

Glad I could help!

[@mohammadarafathuzzaman5442]

Amazing content. Can you please name the tool you used for image analysis? The one with which you checked number of class, histogram, changing contrast and so on.

[@soumyadrip]

I needed this.

[@unamattina6023]

hello, thank you for this great tutorial. I want to download the exactly dataset but in the given link there are a few images. What should I do?

[@adeyinkaadejumobi5091]

Thank you so much, this video is really helpful

[@sivateja3975]

Thank you for your tutorials and lectures.

[@DigitalSreeni]

My pleasure.

[@subratabhattacharjee992]

Thank you for your tutorial. I would like to request an open-slide tutorial for generating patches from the whole-slide images. This is very important for the analysis of histopathology images.

[@mqfk3151985]

he did already, follow this link: kzbin.info/www/bejne/bXqvaH-BiLGVb6s

[@danielcohen5311]

Thanks for the video it was very helpful!

[@DrRubidium]

I am a simple man. I see your new video I press like!

[@siqiwang7259]

Thank you for the amazing tutorial!!

[@maryamsadeghi1199]

Your videos are great, thank you!

[@asmabenbrahem6]

If I use the iou loss and iou as metric do I have to do class_weighting ? I know that for semantic segmentation the accuracy and the crossentropy loss are not the right ones to use because of the unbalanced data but I use the iou loss and iou metric do I have to use class weighting ?

[@ashutoshbhushan7384]

Thanks for the free content through your channel!

[@DigitalSreeni]

Glad you like them!

[@soumi7356]

How are the different classes (labels) for the classes of images specified? Is there a specific directory structure for that?

[@guri000183]

Thank you soo much, i was looking for the exact same stuff and this single video helped me alot.

[@DigitalSreeni]

Glad it helped

[@guri000183]

Hi Sreeni, I have just a small doubt, is it really a multiclass problem? or is it a multilabel? because as per the definition given in video "140 - What in the world is regression, multi-label, multi-class and binary classification?" for me it's more likely a multilabel problem, or am I getting it wrong? Thanks in advance!

[@MarcoCortex]

thanks, Sreeni. Was the original training image carefully segmented in APEER by an expert? or is that job also done with Machine Learning? What is the weight of the EM images you are working with (100MB, 1GB, 10GB, 100GB)? I will follow your channel more closely :)! What kind of filter operations can we do in APEER platform for creating the feature maps to improve segmentation? I ask this final question thinking on QuPath (DoG, LoG, Structure and Hessian filters). Thanks in advance for your answer.

[@talha_anwar]

thanks, i am waiting for this and requested also

[@naimsassine]

Thank you so much, this video is amazing

[@ameyparanjape3292]

Thanks Sreeni, this is great!

[@pedroribeirossb]

Thank you for the great work! I have one question. Is the number of classes related to the number of colours/categories presented in the masks? If so, that means that in your case it's 4 but it could have been 5 or 20? Do we need to change the code in any way if the number of classes gets too much? Seems I'm having 224....Thank you in advance.

[@mrziddiladka]

yes depending on color number of classes depends.Yes it could be anything depending on labels 5 or 20. if number of classes is more the model should be robust no need to change the model attempt it and explore if you are having 224 give input shape 224*224*n_channels

[@suspense_shorts]

Thanks sir, for this wonderful tutorial. I wanted to know what is the software that you were using to view the masks?

[@tushihahahi]

Fantastic Explanation. Thank You.

[@DigitalSreeni]

You are welcome!

[@1UniverseGames]

Nice class sir. SIr, Can you please make some videos like how to read a scientific research paper and how we can get their results by performing our own code or reading that articles. It will really help many of us.

[@ashwinigavali3316]

Amazing explanation

[@nhatminhle1953]

I have a question. I have a task semantic segmentation with 2 classes: leg and foot in a first view order of leg and foot. So what is the number of channels of my output should be? 2 or 3 because I wonder if the background should be labeled

[@sujaybj7533]

Sir, you are the best!!!!! Thank you!!

[@niladrichakraborti5443]

Excellent explanation !!

[@Flyforward226]

Thank you very much! One question, I can only see 2 images under the folder 128_patches. did I miss anything here?

[@alexzambrano5972]

I have watched your videos several times to get my master's thesis right. I have a question, how can I pass the weight information from SegSem to a GAN?

[@MrRynRules]

Thank you for your content!

[@lucaskugler4083]

thanks a lot for your work!

[@tarasankarbanerjee]

Awesome!! Thank you so much...

[@PauloZiemer]

Thanks for this great content

[@ricardomartinez9895]

Hello! KZbin recommended me this video so I started with this one, but I can see that you have more than 208 ! I have one question, maybe there is a video where you explain this. If so, please recommend me that video. If keras works with jpg or png, is it possible to work with .tiff with reflectance units (0-1) ? Thank you so much.

[@diegostaubfelipe4310]

I had this problem with the class_weight -> ValueError: `class_weight` not supported for 3+ dimensional targets. Do you have any suggestions to solve it?

[@juliawa1846]

I have one small question. I am very new to this and trying to understand the method. Is it working also if not all of the classes are present in each test image/ mask? Let´s say I have 5 classes, but one is not present in some images.

[@ankitmars]

Hi Sreeni, Thanks for great video. How does one generate multiclass masks from already annotated images

[@xxxtj3679]

i would also like to know.

[@danialarab8013]

Hi Sreeni, Many thanks for the very useful materials. I tried your code and have the following question for you: When I tried to do the same as you did in the code, i.e., commenting the class_weight=class_weights, I cannot get a reduction in the loss at all! And when I tried to execute class_weight=class_weights, I am getting "ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()". Can you please give me some guidance? Appreciated.

[@DigitalSreeni]

Is this on a different data set or same one I showed? If it is the same data set then the same code should work, please make sure you haven’t skipped any steps. Also, try different kernel initializers, optimizer and loss function.

[@sudiptapaul2825]

class_weight=class_weights is NOT working either on the given dataset or any other type of dataset. Can you kindly give us any suggestions?

[@amirsohail2143]

thank you sir your lectures are very helpful. i have been stuck in class weight problems i have tried different methods but still got error. please help me out in this. how could possibly i do it. i have also tried focal loss but no benefit. i get 3D+ dimension error

[@vaveileinn8402]

Where can I find and download the dataset? Is this available for public/students

[@sebastianandrescajasordone8501]

Why hot-encoding is used here? What is the performance difference between this and having normal interger number's?

[@dr.aolsharon4733]

Thanks for the great content. However, I noticed that class_weight does not work for multiclass segmentation. It keeps throwing an error when I run the script you shared. Could there be a solution for this?

[@DigitalSreeni]

I did not test class_weight for multiclass. In fact, I recommend using focal loss for multiclass. You can also use a combination of focal loss and dice loss and for dice you can provide class weights. This is probably the easiest way to handle this. In general, focal loss did a great job for my datasets with multiple classes.

[@jiahongxie6884]

Thank you so much! How can I do multiclass instance segmentation in unet?

[@rohinigaikar4117]

Thank you so much. 👏👏👏

[@jizhang02]

I have a question, is focal loss fit for the multi-class segmentation? It needs to do some change or not? Thank you

[@DigitalSreeni]

Focal loss is specifically designed for multiclass problems where you have imbalance between classes or if you have a tough to classify class against easy classes.

[@ajinkyadeshpande4812]

Where can i get a video that explains datasets - I) Kidney (RCC) (II) Triple Negative Breast Cancer (TNBC) (III) MoNuSeg-2018 and many other nuclei segmentation datasets ?

[@cheriliafernandez1174]

please help me. when I plot the testing image, testing label, and prediction mask, it gives me different images (I plot it several times and it still gives me different images). any solution? thank you very much.

[@anushkaagarwal8250]

Hey, is there a way we can get access to the trained weights?

[@yjoliiyki706]

could you upload a mask RCNN for the instance image segmentation?

[@yassineone4196]

Hi sir, what if i have 17 classes and all of them in NIFTI format as well as the volumes ( three volumes with three different voltage/energy), what's the changes that i should make besides num_classes, thank you for the videos.

[@DQu-tm2qn]

Could you do a video about predicting continuous variable using Unet? Thanks!

[@fuegopuro5933]

Wow, do you plan to roll out tutorials on lits?

[@geogob]

Thank you for the video. A question, 4 classes including background?

[@DigitalSreeni]

In this example, there is nothing like background. If you have a background class then that can be assigned a value 0. The way I have written my code, the background would be the 5th class.

[@geogob]

@@DigitalSreeni 👍.

[@mustaphaelammari1128]

Thanks for the video. I'm having a problem with the code, I'm getting error ``` column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel(). y = column_or_1d(y, warn=True) ValueError: Shapes (16, 128, 128, 4) and (16, 128, 128, 1) are incompatible ``` How can i fix it please?

[@murli34]

hi, thank you for the knowledge you shared,how to calculate dice score for each class similer to IoU? i need for brats dataset(3D), thank you again

[@adam100PCI]

Thank you very much for your videos. if i change img=cv2.imread(img_patch,0) to img=cv2.imread(img_patch,3) .i.e, use rgb channels. what are the necessary changes in the code that i have to make.

[@pedromartinezbarron4720]

Using imageJ, how can I save my semantic labels in only one mask? Like in this vide where you get a single mask but represented with diferrent gray-scale levels

[@anishachakravorty1395]

Sir thank you for the video. Can you please help me with this error i am getting with compute class weight. It says compute class weight() takes 1 positional argument but 3 were given.

[@DigitalSreeni]

May be this video helps: kzbin.info/www/bejne/h5_XfXWsir-Fa8U

[@pheiroijamprishika6414]

Can you help get ground truth i.e mask image from a raw CXR image for segmentation using Unet..

[@spiritualghosh429]

I have a question. Let us say that I have 3 classes [1, 2, 3] and one unlabelled class (ambiguous pixels are kept in "0") [0]. Now when I used (3 + 1) these classes for training [0, 1, 2, 3], while reconstructing the feature map in the last layers of the decoder part, I mapped 32 layers to 4 classes. The problem I am facing is, I just wanted to exclude these predictions of unlabelled classes. However, while using the LOSS function, I had to consider the unlabelled classes. My question is how to deal with these unlabelled pixels during training, and backdrop and deconvolution in the last layers for predictions? I would be grateful if I can connect with you via call or any medium. Please respond. I am stuck with this aforementioned problem. Raktim Ghosh (Researcher on Planetary Science and Deep Learning).

[@boubakerasaadi363]

I think you can do that by assigning a 0 weight to the unlabeled data in the loss function. You can do this with sample_weight of the keras.compile.

[@yogipaleka2702]

Terimakasih banyak sir

[@DigitalSreeni]

Had to translate to find out what that means, apparently Thank you in Indonesian. Thank you too for watching the video, I hope you found it to be useful and educational.

[@plyap3872]

Hi Sreeni, May I know which keras version did you use for this tutorial. I use keras 2.3.1 and when I tried to execute MeanIoU, it came back with this error : RuntimeError: `MeanIoU` metric is currently supported only with TensorFlow backend and TF version >= 2.0.0.

[@DigitalSreeni]

As the error says, you need tensorflow > 2.0. You may have 1.x, please verify.

[@amuldhungel5060]

Suppose I have train set and test set of having 80k and 10k images respectively, should I have to label all of those images of both train and test set?

[@DigitalSreeni]

If your train set and test set have no labels then all you have is just a raw data set.

[@manuelpopp1687]

Do you need to_categorical when using SparseCategoricalCorssentropy? Are there differences?

[@DigitalSreeni]

No, you should not convert your labels to categorical when using sparse categorical cross-entropy. Sparse CCE can work with integer encoded labels. For mutually exclusive classes you can use either loss functions, SCCE or CCE as long as you make sure the labels are encoded the correct way.

[@df-cc4jo]

What application did you use to zoom into the mask images? The mask images start out dark. Thank you

[@DigitalSreeni]

You can use FiJi (imageJ)

[@mimolinodeviento]

Great as always!! Thank you so much for your hard work :)

[@nitheshr7000]

Why patching the image is preferred rather than resizing in segementation ?

[@khushpatelmd]

Thank you so much

[@matthewavaylon196]

When you exapnd the masks array to (1600, 128, 128, 1). What is the 1? I get it's to match the shape of the image set, but what does the 1 mean. We have 1600 images of 128x128. The 1 means?

[@DigitalSreeni]

The 1 represents the 4th dimension for this numpy array. There can be many dimensions depending on the nature of data. For images, you can interpret this array as 1600 images, each 128x128 and grayscale (or only single channel). For multichannel images, for example color images with RGB channels, the array can have a shape of (1600, 128,128,3).

[@alexandrustefan12345]

Can you please name the tool you used for image analysis?

[@plyap3872]

Hi Sreeni, Thank you very much for your videos on segmentation. I have watched most of them and have learned much. I am doing brain tumor segmentation and in brain tumor MRI scans, each scan came in 4 sets, so it is [240,240,155,4]. So in training, how should I prepare my data. Should I stay with the dimensions or should I squashed the 4th dimension into the 3rd like [240,240,620] ? The label shape is [240,240,155]. Your inputs will be very helpful

[@DigitalSreeni]

You seem to be referring to dataset similar to Brats2020. I will be releasing a couple of videos on this topic in August. The way I handled this dataset is by using 3D unet. I only used 3 channels instead of 4 as I found one of those to be redundant. I also broke the volumes down to 64x64x64 to make sure they fit my system memory. Also, I dropped all sub-volumes with less than 1% labeled regions.

[@plyap3872]

@@DigitalSreeni Thanks Sreeni, I was referring to the BRATS2021 dataset. Anyway I think the NifTI dataset format is the same. Any problem if I use the 4 channels instead of the 3. Looking forward to your coming videos!

[@ivanbellan1467]

Your videos are wonderful. I had a problem on the line 88. It said "class_weights = class_weight.compute_class_weight('balanced', np.unique(train_masks_reshaped_encoded), train_masks_reshaped_encoded) *** TypeError: compute_class_weight() takes 1 positional argument but 3 were given" Could you help me?