208 - Multiclass semantic segmentation using U-Net

Code generated in the video can be downloaded from here:
github.com/bns...
The dataset used in this video can be downloaded from the link below. This dataset can be used to train and test machine learning algorithms designed for multiclass semantic segmentation. Please read the Readme document for more information.
drive.google.c...
To annotate images and generate labels, you can use APEER (for free):
www.apeer.com

[@awaisahmad5908]

One of the best channels for Research Students of Computer Vision discipline.

[@DigitalSreeni]

Thank you :)

[@kaihsiangju]

This channel deserves millions of subscribers. Thanks for the amazing contents.

[@raguramramamoorthy8569]

true very true --- he can sell this course for at least 100 dollars ...but he has done it for free ...

[@gadaanet]

Exactly!

[@eli_m6556]

Needed this so much. Seems like every time I run into a problem with my research you put out a video answering my prayers. Thanks Sreeni.

[@hadim.4125]

Thank you so much! I was stock at my model and I couldn't figure out y for weeks! Thanks to u and ur straightforward videos, I fixed the problem!

[@5junkmail]

Exactly what I was looking for, you are a very knowledgeable person with a great talent for explaining things!!! Please don't stop!

[@kannanv9304]

Ajarn - Can fully understand the efforts and time you are putting in to create these contents.......The real value of gold is not known to the one who wears it......It is know to the miners who take out tons and tons of slush to extract 1 ounz of gold........Pranams......You have an amazing sense of sequels.......And I am sure, you are not going to stop the sequels on U-nets with this.......

[@vimalshrivastava6586]

Best KZbin channel for deep learning researchers.

[@DigitalSreeni]

I'm glad you think so :)

[@jacobusstrydom7017]

Wow the best explain of these concepts I have seen in a long time. Thanks for this

[@jk47nitk]

Thanks for Multiclass segmentation. In Segmentation or even Image related Deep Learning your Videos are best.....

[@vaveileinn8402]

The only word : Great! please keep continue Sir. thank you so much.

[@suyashdahale4355]

Thank you sreeni for the labelencoder path, all other places it was simply -1 , but my masks were in color and i just realised that differnce after wathing this tutorial..... super helpful insight.

[@EUMikkel]

Sreeni thank you so much for all the work you put into these videos. It has helped me so much get started with segmentation

[@DigitalSreeni]

You are so welcome!

[@DrRubidium]

I am a simple man. I see your new video I press like!

[@soumyadrip]

I needed this.

[@gadaanet]

I've just found what I was looking for. Thank you!

[@DigitalSreeni]

Glad I could help!

[@skynetpro549]

this channel is love !! supported me a lot

[@DigitalSreeni]

Happy to hear that!

[@ashutoshbhushan7384]

Thanks for the free content through your channel!

[@DigitalSreeni]

Glad you like them!

[@anandsrivastava5951]

Great!! I was in need of it badly :) Great work. !!

[@bikkikumarsha]

Thank you, your tutorials are one of the best.

[@davidhresko7351]

Finally video explained to details. Thanks

[@sivateja3975]

Thank you for your tutorials and lectures.

[@DigitalSreeni]

My pleasure.

[@kavithashagadevan7698]

Thank you. Your tutorials are life savers for me

[@frankrobert9199]

Great lectures, I follow up with your series.

[@DigitalSreeni]

Great to hear!

[@applejuice5785]

thanks I was just working on a multiclass segmentation with Unet

[@Stats0654]

Thank you soo much, i was looking for the exact same stuff and this single video helped me alot.

[@DigitalSreeni]

Glad it helped

[@Stats0654]

Hi Sreeni, I have just a small doubt, is it really a multiclass problem? or is it a multilabel? because as per the definition given in video "140 - What in the world is regression, multi-label, multi-class and binary classification?" for me it's more likely a multilabel problem, or am I getting it wrong? Thanks in advance!

[@adhiliqbal6020]

Great job srini, I'm learning alot

[@Divya-ok1ou]

Thank you for your videos. They are very much helpful.

[@angelopiasentin4197]

Thanks Sreeni. You always bring new ideas to the AI world.

[@DigitalSreeni]

My pleasure 😊

[@Ninguempensonesse]

Amazing content my and of many professor of Deep Learning, I think that a nice suggestion to your next videos, could be the addition of the version of the installed libraries and modules in each notebook. That´s it thanks.

[@salarghaffarian4914]

Your U-net videos are very helpful for me. I would appreciate if you could produce videos on instance segmentation as well and particularly Mask RCNN model. Thanks a lot. 🙏🙏

[@subratabhattacharjee992]

Thank you for your tutorial. I would like to request an open-slide tutorial for generating patches from the whole-slide images. This is very important for the analysis of histopathology images.

[@mqfk3151985]

he did already, follow this link: kzbin.info/www/bejne/bXqvaH-BiLGVb6s

[@niladrichakraborti5443]

Excellent explanation !!

[@victordiasteixeira1694]

This class helped me sooo much! Thanks a lot s2

[@sujaybj7533]

Sir, you are the best!!!!! Thank you!!

[@hassanmahmood7284]

thanks for the contribution, appreciated.

[@ashwinigavali3316]

Amazing explanation

[@danielcohen5311]

Thanks for the video it was very helpful!

[@adeyinkaadejumobi5091]

Thank you so much, this video is really helpful

[@tushihahahi]

Fantastic Explanation. Thank You.

[@DigitalSreeni]

You are welcome!

[@themaryamsadeghi]

Your videos are great, thank you!

[@ameyparanjape3292]

Thanks Sreeni, this is great!

[@siqiwang7259]

Thank you for the amazing tutorial!!

[@MrRynRules]

Thank you for your content!

[@talha_anwar]

thanks, i am waiting for this and requested also

[@samuelireke238]

Thank you very much for your videos. They have been of immense help for a histopathology cell counting project I am working on. I am trying to investigate the impact of auxiliary outputs on UNets for microscopic cell detection and counting but have been stuck with a bug for over a week now. Most documentation online hasn't helped. My auxiliary outputs use various blocks of the UNet model as inputs as such output different shapes from the original input size of (256,256,3). So the main challenge is how to declare this during training so it takes this into consideration. Error Message obtained: ValueError: Error when checking target: expected aux1 to have shape (32, 32, 1) but got an array with shape (256, 256, 1) Model Summary: Layer (type) Output Shape Param # Connected to ================================================================================================== input_6 (InputLayer) (None, 256, 256, 3) 0 __________________________________________________________________________________________________ ... __________________________________________________________________________________________________ aux1 (Conv2D) (None, 32, 32, 1) 33 activation_74[0][0] __________________________________________________________________________________________________ aux2 (Conv2D) (None, 64, 64, 1) 33 activation_76[0][0] __________________________________________________________________________________________________ aux3 (Conv2D) (None, 128, 128, 1) 33 activation_78[0][0] __________________________________________________________________________________________________ original (Conv2D) (None, 256, 256, 1) 33 activation_72[0][0]

[@allishadow8134]

I like the way that you explain the concept... I will subscribe for future excellent content.... Thank you

[@danialarab8013]

Hi Sreeni, Many thanks for the very useful materials. I tried your code and have the following question for you: When I tried to do the same as you did in the code, i.e., commenting the class_weight=class_weights, I cannot get a reduction in the loss at all! And when I tried to execute class_weight=class_weights, I am getting "ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()". Can you please give me some guidance? Appreciated.

[@DigitalSreeni]

Is this on a different data set or same one I showed? If it is the same data set then the same code should work, please make sure you haven’t skipped any steps. Also, try different kernel initializers, optimizer and loss function.

[@sudiptapaul2825]

class_weight=class_weights is NOT working either on the given dataset or any other type of dataset. Can you kindly give us any suggestions?

[@naimsassine]

Thank you so much, this video is amazing

[@lucaskugler4083]

thanks a lot for your work!

[@tarasankarbanerjee]

Awesome!! Thank you so much...

[@PauloZiemer]

Thanks for this great content

[@mimolinodeviento]

Great as always!! Thank you so much for your hard work :)

[@EB3103]

awesome video!

[@DigitalSreeni]

Thanks!

[@anirbank6579]

When we download the data from the link, we do not see the images and masks sub-folders. We only see images_as_128x128_patches.tif and similarly masks_as_128x128_patches.tif. How do we extract the images and masks from these, can you please give the code. Might seem elementary, but it will be helpful.

[@rohinigaikar4117]

Thank you so much. 👏👏👏

[@unamattina6023]

hello, thank you for this great tutorial. I want to download the exactly dataset but in the given link there are a few images. What should I do?

[@bogdanchelu5578]

Thank you very much for your video, it helped a lot. Only thing I need to ask you about is the calculation of the IoU. I have a very unbalanced dataset, and I ran the model on it several times with several different loss functions, including some that are explicitly made to handle data unbalance, but every single time the IoU confusion matrix looks as if my model classified everything as background (i.e. the most common "class"). Since I'm sure the data is correctly labelled and I doubt there can be something wrong with the model especially after running it with different functions, I think there is something wrong with the IoU calculation. Do you have any idea? Thank you.

[@rashariyad1821]

I don't understand the labels of your classes. I have multi-labeled colored images, each class is either red, green, or yellow, .... If I looked into the image vlaues it is between (0-255) so how did you make it 1,2,3... and should I change mine too?

[@ankitmars]

Hi Sreeni, Thanks for great video. How does one generate multiclass masks from already annotated images

[@xxxtj3679]

i would also like to know.

[@yogipaleka2702]

Terimakasih banyak sir

[@DigitalSreeni]

Had to translate to find out what that means, apparently Thank you in Indonesian. Thank you too for watching the video, I hope you found it to be useful and educational.

[@diegostaubfelipe4310]

I had this problem with the class_weight -> ValueError: `class_weight` not supported for 3+ dimensional targets. Do you have any suggestions to solve it?

[@aimen__]

i wish you showed how to use focal loss

[@asmabenbrahem6]

If I use the iou loss and iou as metric do I have to do class_weighting ? I know that for semantic segmentation the accuracy and the crossentropy loss are not the right ones to use because of the unbalanced data but I use the iou loss and iou metric do I have to use class weighting ?

[@hadim.4125]

Can you please tell us how I can use dice coefficient as the loss function for multiclass segmentation too?

[@pedroribeirossb]

Thank you for the great work! I have one question. Is the number of classes related to the number of colours/categories presented in the masks? If so, that means that in your case it's 4 but it could have been 5 or 20? Do we need to change the code in any way if the number of classes gets too much? Seems I'm having 224....Thank you in advance.

[@mrziddiladka]

yes depending on color number of classes depends.Yes it could be anything depending on labels 5 or 20. if number of classes is more the model should be robust no need to change the model attempt it and explore if you are having 224 give input shape 224*224*n_channels

[@fatemehmohseni5414]

what if the dataset's images were not grayscale and had 3 channels? could you please help on how train that kind of dataset

[@Flyforward226]

Thank you very much! One question, I can only see 2 images under the folder 128_patches. did I miss anything here?

[@abhishekabhishek4315]

Hi Sreeni, I am getting an error in the label encoder i.e., not enough values to unpack (expected 3, got 1). Can you help me with this?

[@1UniverseGames]

Nice class sir. SIr, Can you please make some videos like how to read a scientific research paper and how we can get their results by performing our own code or reading that articles. It will really help many of us.

[@MarcoCortex]

thanks, Sreeni. Was the original training image carefully segmented in APEER by an expert? or is that job also done with Machine Learning? What is the weight of the EM images you are working with (100MB, 1GB, 10GB, 100GB)? I will follow your channel more closely :)! What kind of filter operations can we do in APEER platform for creating the feature maps to improve segmentation? I ask this final question thinking on QuPath (DoG, LoG, Structure and Hessian filters). Thanks in advance for your answer.

[@pedromartinezbarron4720]

Using imageJ, how can I save my semantic labels in only one mask? Like in this vide where you get a single mask but represented with diferrent gray-scale levels

[@ivanbellan1467]

Your videos are wonderful. I had a problem on the line 88. It said "class_weights = class_weight.compute_class_weight('balanced', np.unique(train_masks_reshaped_encoded), train_masks_reshaped_encoded) *** TypeError: compute_class_weight() takes 1 positional argument but 3 were given" Could you help me?

[@suspense_shorts]

Thanks sir, for this wonderful tutorial. I wanted to know what is the software that you were using to view the masks?

[@alexandrustefan12345]

Can you please name the tool you used for image analysis?

[@khushpatelmd]

Thank you so much

[@NomanArif-n3u]

Do make a video on training the nnUnet and UNet with transformers using 3D dataset

[@nitheshr7000]

Why patching the image is preferred rather than resizing in segementation ?

[@MD-lc3kf]

in my case I have 10 classes, some of the splitted images that contains the classes dosen't necessarily contain all the labels, so in one image n number of classes and in another image I have a diffrent number of classes, exp: image 1 contains class 1 2 3 4 , image 2 contains class 1 2 5 6 7 8 9 10, image 3 contains class 5 4 8 2 ect... will this work ?

[@jiahongxie6884]

Thank you so much! How can I do multiclass instance segmentation in unet?

[@mqfk3151985]

As usual, your videos make life very easy for researchers. I have a question regarding class weights, when I uncommented the class_weight part in the model fitting, it returned an error that class_weights has to be a dictionary, something like this (on my own dataset): Class weights are...: {0: 0.4280686779466047, 1: 1.54654951724371, 2: 0.40951813587110275, 3: 42.324187597545105, 4: 1.5749410555965808, 5: 2.2925788973162344, 6: 2.080430679675916} even upon changing the class_weights into a dictionary, I faced another issue: `class_weight` not supported for 3+ dimensional targets meaning that my y_test_cat is a 3-D matrix which is not supported for class_weights. References suggested to use "sample weights" instead of class_weights any suggestions on how to solve this issue? Again, Many thanks for your amazing videos.

[@finlyk]

Hi, I face the same problem, did you manage to solve it ? :)

[@mqfk3151985]

@@finlyk not yet, I was hoping to get some answer. I may end up trying to solve it myself

[@finlyk]

@@mqfk3151985 after some research, it seems that you cannot apply weight to 2D array. The model output is (height, width, number of class), and should be flatten as (height * width, number of class) for the weights to be applied. Will try that tomorrow and tell you if it helps

[@finlyk]

I didn't managed to fix it unfortunately.. would appreciate any help if you try to handle the issue :)

[@matancadeporco]

does anyone solve this problem? i'm stuck here

[@mohammadarafathuzzaman5442]

Amazing content. Can you please name the tool you used for image analysis? The one with which you checked number of class, histogram, changing contrast and so on.

[@anishachakravorty1395]

Sir thank you for the video. Can you please help me with this error i am getting with compute class weight. It says compute class weight() takes 1 positional argument but 3 were given.

[@DigitalSreeni]

May be this video helps: kzbin.info/www/bejne/h5_XfXWsir-Fa8U

[@mr.shouvikdey8482]

solution of class_weight for multiclass semantic segmentation is-> SMOTE (Synthetic Oversampling Technique (SMOTE))

[@juliawa1846]

I have one small question. I am very new to this and trying to understand the method. Is it working also if not all of the classes are present in each test image/ mask? Let´s say I have 5 classes, but one is not present in some images.

[@nhatminhle1953]

I have a question. I have a task semantic segmentation with 2 classes: leg and foot in a first view order of leg and foot. So what is the number of channels of my output should be? 2 or 3 because I wonder if the background should be labeled

[@ricardomartinez9895]

Hello! KZbin recommended me this video so I started with this one, but I can see that you have more than 208 ! I have one question, maybe there is a video where you explain this. If so, please recommend me that video. If keras works with jpg or png, is it possible to work with .tiff with reflectance units (0-1) ? Thank you so much.

[@ajinkyadeshpande4812]

Where can i get a video that explains datasets - I) Kidney (RCC) (II) Triple Negative Breast Cancer (TNBC) (III) MoNuSeg-2018 and many other nuclei segmentation datasets ?

[@shamlabeevia9436]

Thank you sir ..

[@adam100PCI]

Thank you very much for your videos. if i change img=cv2.imread(img_patch,0) to img=cv2.imread(img_patch,3) .i.e, use rgb channels. what are the necessary changes in the code that i have to make.

[@geogob]

Thank you for the video. A question, 4 classes including background?

[@DigitalSreeni]

In this example, there is nothing like background. If you have a background class then that can be assigned a value 0. The way I have written my code, the background would be the 5th class.

[@geogob]

@@DigitalSreeni 👍.

[@dr.aolsharon4733]

Thanks for the great content. However, I noticed that class_weight does not work for multiclass segmentation. It keeps throwing an error when I run the script you shared. Could there be a solution for this?

[@DigitalSreeni]

I did not test class_weight for multiclass. In fact, I recommend using focal loss for multiclass. You can also use a combination of focal loss and dice loss and for dice you can provide class weights. This is probably the easiest way to handle this. In general, focal loss did a great job for my datasets with multiple classes.

[@amirsohail2143]

thank you sir your lectures are very helpful. i have been stuck in class weight problems i have tried different methods but still got error. please help me out in this. how could possibly i do it. i have also tried focal loss but no benefit. i get 3D+ dimension error

[@mr.shouvikdey8482]

class_weight in .fit is not working it says "`class_weight` not supported for 3+ dimensional targets".

[@parulianrenaldi7846]

can u explain how to make that mask dataset?

[@yassineone4196]

Hi sir, what if i have 17 classes and all of them in NIFTI format as well as the volumes ( three volumes with three different voltage/energy), what's the changes that i should make besides num_classes, thank you for the videos.

[@spiritualghosh429]

I have a question. Let us say that I have 3 classes [1, 2, 3] and one unlabelled class (ambiguous pixels are kept in "0") [0]. Now when I used (3 + 1) these classes for training [0, 1, 2, 3], while reconstructing the feature map in the last layers of the decoder part, I mapped 32 layers to 4 classes. The problem I am facing is, I just wanted to exclude these predictions of unlabelled classes. However, while using the LOSS function, I had to consider the unlabelled classes. My question is how to deal with these unlabelled pixels during training, and backdrop and deconvolution in the last layers for predictions? I would be grateful if I can connect with you via call or any medium. Please respond. I am stuck with this aforementioned problem. Raktim Ghosh (Researcher on Planetary Science and Deep Learning).

[@boubakerasaadi]

I think you can do that by assigning a 0 weight to the unlabeled data in the loss function. You can do this with sample_weight of the keras.compile.

[@rim-tt1wo]

Thank you for the video, but I have a probleme, every time I try to fit the model, the kernel crashes, does anyone experienced the same issue?

[@connielee4359]

Did you occur the error of " 'class_weight' not support for 3+ dimensional targets " when using class_weight?

[@DigitalSreeni]

Yes. For multiclass I recommend dice loss where you supply class weights or use focal loss that works well without providing class weights.

[@connielee4359]

@@DigitalSreeni Thanks for your reply.

[@matancadeporco]

@@connielee4359 have u solve this? i'm stuck on this

[@connielee4359]

@@matancadeporco I didn't solve this problem;however, I use a loss function named "weighted categorical_crossentropy" instead. Hope you find this information helpful.

[@plyap3872]

Hi Sreeni, May I know which keras version did you use for this tutorial. I use keras 2.3.1 and when I tried to execute MeanIoU, it came back with this error : RuntimeError: `MeanIoU` metric is currently supported only with TensorFlow backend and TF version >= 2.0.0.

[@DigitalSreeni]

As the error says, you need tensorflow > 2.0. You may have 1.x, please verify.

[@sebastianandrescajasordone8501]

Why hot-encoding is used here? What is the performance difference between this and having normal interger number's?