222 - Working with large data that doesn't fit your system memory - Semantic Segmentation

[@DigitalSreeni]

I should apologize for using the term 'PyCharm' instead of 'PyTorch' multiple times in the video.

[@07jyothir]

Thank you so much for the details. The deviations are actually helpful. Great job sir.

[@DigitalSreeni]

In that case I'll keep deviating :)

[@navroopgill7594]

@@DigitalSreeni Couldnt agree more with Jyothir. Appreciate the deviations with fine details.

[@aankitdas6566]

I CANNOT thank you enough for these videos. Love you!

[@Mohammad.python]

I think you are the best professor ❤❤❤❤❤

[@vianniealeazer8]

Great tutorial. Thanks a lot. A question what about RGB mask with more than 2 classes Do we have to do one hot encoding.

[@biplugins9312]

I always learn something new from your videos. Today I finished up with video 210. It worked like a charm on the sandstone, even with the latest tensorflow. I have to update the directory structure for the latest keras, but I now know how to do that. I decided I wanted to see what else I could learn from the vidcam data, which was available to me. The camera data is 360*480 which means I could make my patch size 120*120 instead of 128*128. Then I got hit on the head with an error: ValueError: Dimension 1 in both shapes must be equal, but are 16 and 15. Shapes are [?,16,16] and [?,15,15]. for '{{node model_3/decoder_stage1_concat/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32](model_3/decoder_stage1_upsampling/resize/ResizeNearestNeighbor, model_3/stage3_unit1_relu1/Relu, model_3/decoder_stage1_concat/concat/axis)' with input shapes: [?,16,16,256], [?,15,15,128], [] and with computed input tensors: input[2] = . I guessed that maybe the patch size was killing me. With the contraction of the matrix size I expected it to go 120->60->30->15, the same 15 it seems to be complaining about above. To check that I decided to change the patch size to a nice binary 128*128 with a step size of 116. That would fill in nicely the 360 direction and leave me 4 unused pixels in the 480 direction. Fair enough. I tried it and everything worked including a reasonable IoU of 80% on the training and 60% on the validation. (At least I think those numbers are reasonable....) In this video I got the impression that it wasn't in the criteria to have nice binary numbers for the images, so I thought I would ask for your opinion on my 210 results. It obviously didn't like my patch size of 120. There is another question I have. In 210 someone went to all the trouble to define 12 classes, but even the author used a subset of 3 or 4. My question is why? If you go to all the trouble to define 12 classes, why not use them all? It is a matter of space, or training time or what? Another question with regard to today's video. You hinted that one important reason for using patchify was to save memory. Since my local computer has no GPU, my only choice is collab. I think I have tons of memory there, so why should I care about space? I'm clearly missing something, but what? You were dead on when you said that this video is a long one so you guess if people are still watching, it must be interesting. Yes, yes, yes....

[@munaali4415]

Great Videos Thank you very much

[@linachato5817]

thank you so much for the valuable information! i have question: in the 21:00 you mentioned for binary segmentations if greater than 0 it will fill with 1. if I have 4 classes (0, 1, 2, 3) how can I do this please?

[@iulencabeza6454]

Hi. First of all, thanks for all your videos. They are really helpful for researchers :) Second, do you have a video but applying fit_generator (DataGenerator) to multiclass segmentation? As you mentioned it seems a little tricky and I only have found for binary problems. Thanks

[@DigitalSreeni]

Yes, I believe in many videos. For example, here ... kzbin.info/www/bejne/ZoiZfn6Qpriem5o

[@iulencabeza6454]

@@DigitalSreeni Thank you. Finally I did my own code based on the video (233) :). Practically I have seen all your recent videos. Awesome work. Really. Tomorrow I will see if it's valuable to load a pre-trained CNN as encoder :)

[@alex-beamslightchanal8743]

Great video as all your other content. Thank you!

[@navroopgill7594]

Thank you Sreeni for another deep dive video on semantic segmentation. Perfect timing for a bottleneck in my project. I need to perform semantic segmentation and classification on a mammogram image. For eg -segment the tumour and also classify it as benign and malignant. Have you already done a video around this ? Pls direct me to the same.

[@DigitalSreeni]

No, I haven't done a video on this topic but it sounds like a segmentation task followed by measurements and using measurements to assess benign/malignant. You can do it, sounds like a fun project.

[@navroopgill7594]

@@DigitalSreeni thanks for the revert. I am still looking for a good reference for semantic segmentation followed by classification (benign/malignant etc) but no luck so far. Will appreciate any guidance. Coming back to this video, i am training the model in the exact same way with 1208 training and 151 val imgs/masks with a batch size 8, but my "training dice metric "and "val dice metric" stay constant at 0.0078 throughout the training session. Wondering what am I missing. One small observation regarding my image generator training dataset- it mentions " found 1208 images and 2 classes" . I have selected "Class Mode:None " as prescribed in the video. Any advice?

[@prathameshjoshi1408]

As always awesome !! Thank you

[@DigitalSreeni]

Thanks

[@Ahmetkumas]

Great video thanks, what do you think the best method for detecting the moving particles approaches on microscope images?

[@retrodingo]

thank you for the great video, it helps me a lot. I have a question, how do you interpret the dice_metric, val_loss and val_dice_metric ? is that a percentage? because the number doesn't look like a percentage, here's how it looks in my notebook loss: 0.1432 - dice_metric: 0.0040 - val_loss: 0.3885 - val_dice_metric: 0.0026

[@sanjeetpatil1249]

Thank you for the videos! Can you kindly suggest a way for a mask generator in which 4 classes(0,1,2,3) are present instead of binary classes?

[@navroopgill7594]

Hi I am not able to train my Unet segmentation model at all with the image generator option. I am using google colab pro. The generated augmented 'images' look fine but for some reason the output of the 'mask generator' shows my masks as all black images. The training stops after 3 epochs without any convergence. I am working with approx 1200 grayscale mammogram images and masks both resized to 512x512x1. All input data shapes and dimensions look fine before feeding into the U-Net. Will appreciate any guidance on this.

[@aliqsaeed2878]

A very big THANK from my deepest heart for all the effort you put in your videos, this is an online treasure . Please, i am trying to implement GAN for image segmentation but i don't know how to figure up the G and D networks in such a way that the input_dim of G is an image and the output_dim of D is an image as well. Do i have to skip the noise vector of G? How D will discriminate real/fake if its output is not a single value (0 or 1) ? I couldn't find a video explaining segmentation with GAN, could you please help out with this issue? Thanks in advance.

[@DigitalSreeni]

I never tried to use GAN for image segmentation. Frankly, I don't see the point or any advantages of GAN over the well-established U-net approach for segmentation.

[@aliqsaeed2878]

@@DigitalSreeni thanks for your reply (^_*)

[@edmald1978]

@@DigitalSreeni if we do not have The labels. It is possible tô use gan tô extract features and then use any cluster algorithm to find the classes of each pixel?

[@nouajirahma9279]

Thank you for this tutorial it's very useful , Could you also please make a video about MorphNet ?. Much appreciated!

[@eransasson20]

This example is a binary segmentation (2 classes only). Has anyone successfully run ImageGenerator for segmentation with more than 2 classes? I am getting a tensor shape mismatch probably because my model is built with 8 classes, but the train and val folders contain only 1 class...How to overcome this?

[@zakariaesaidi843]

Thankyou for your cources, i have a problem my images and masks are not aligned

[@kerenevangeline3816]

This video is a life saver!! Thank you soo much! When I try to convert the augmented images to categorical for multiclass semantic segmentation my memory dies. Can you please provide a solution for that? I'm stuck there for quite sometime.

[@shashankkaryakarte8463]

Thanks!

[@shashankkaryakarte8463]

I am student right now at IITK, your videos are literally saving my life, I am going to join job in next month, at that time I will pay you more to thank you more..... request you to keep saving lives.... thanks....

[@DigitalSreeni]

Any amount for payment helps, do not worry. Bulk of it goes to supporting charities anyway. Congratulations and good luck with your new job.

[@sajjadmahmoudi2854]

Hi, I notice that the number of steps in the code should be added by one. Otherwise, you miss the last batch

[@kdm8568]

dataset link shows only 165 images,how did you get 1600 images. Could you send the video link that you extracted from this dataset?

[@DigitalSreeni]

165 large images (1024x768) that I cropped to 256x256 patches.

[@salarghaffarian4914]

Thanks for this great content. Could you also please run Mask-RCNN and make video on that. Much appreciated!

[@rezatabrizi4390]

Thank you so mach sir

[@DigitalSreeni]

You are welcome.

[@sivava1500]

I have a problem where the individual WSI images are too large to load, even single image. Can u suggest on this pls

[@ahmedtouzani6780]

Hello sir thanks alot for your work you really help me in my final project . Can you please make us a video about HOG+LBP in image classification . I hope that you answer me

[@angelceballos8714]

Hello sir, great video! Is there any way to contact you? I have been following you videos specially those of semantic segmentation and wanted to know if you could help me build an architecture.

[@DigitalSreeni]

Please post your questions here. I do not have time for personalized help as I barely find time to record videos nowadays, sorry.

[@angelceballos8714]

@@DigitalSreeni I wanted to know if there is a way to modify a Unet to get as input two images from the same place but different year, and create a segment mask that spots the difference.

[@ornobrahman5675]

Please make a tutorial regarding a big dataset of multi-label image classification with a CSV file.

[@talha_anwar]

If needed , I can share , it's in pytorch lightening with tpu

[@ornobrahman5675]

@@talha_anwar Thank you very much. Yes, please do share.

[@CharlanDellon]

It's possible to apply the image generator in other data types unlike image for to make a any data generator ? How do It?

[@DigitalSreeni]

Augmentation makes sense for images as you are just transforming existing images. For structured data, you need to synthesize new data if you want to augment. Please look into SMOTE data augmentation of structured data.

[@gabrigamer00skyrim]

Could you do a new tutorial on tf.data.dataset? ImageDataGenerator seems to be deprecated.

[@rameshwarsingh5859]

⭐⭐⭐⭐⭐⭐

[@matancadeporco]

cannot import normalize, i dont know why... "ImportError: cannot import name 'normalize' from 'keras.utils' (/usr/local/lib/python3.7/dist-packages/keras/utils/__init__.py)"

[@DigitalSreeni]

You can try MinMaxScaler scikit-learn. I ran into the same issue on my other system, didn't find time to investigate. Maybe normalize got removed. In any case, MinMaxScaler is probably a better option anyway.

[@rezatabrizi4390]

hello, sir how can I apply Lovasz loss to semantic segmentation?

[@DigitalSreeni]

You can apply any custom loss as long as you can define a function for the loss. I haven't experimented with Lovasz loss but I am sure you can find existing code on Github.

[@Diego0wnz]

how do u get all this knowledge? it is actually insane!

[@DigitalSreeni]

Not really insane. I started with 0 knowledge about python in early 2019. I just spent about 30 mins a day in teaching myself the basics of coding, with a focus on image analysis. Whenever I ran into issues, I just Google searched. Working on real life problems is the best way to learn and you can find many problems on Kaggle. Many more today than back in 2019.

[@talha_anwar]

Please cover monai

[@dishantdeshmukh5330]

Built custom models tutorial for training/fitting data

[@teddy911]

sir, correct me if I'm wrong. So, what you did is reopen the IDE and release the RAM to prevent the program to crash right? I recently got the same issue when I'm loading a large ISIC dataset, the kernel always crashes and I don't know how to solve the issue.

[@DigitalSreeni]

Whether you work on WIndows or on Colab, memory is always an issue. These systems do not easily release the memory so the best way would be to restart the kernel or the entire IDE.

[@teddy911]

​@@DigitalSreeni Sir, If I were to add 'tf.keras.callbacks.EarlyStopping(monitor='???', patience=3)' in my callbacks List, which parameter should I monitor? If I use BinaryFocalLoss as Loss function like you mentioned in the video.