232 - Semantic Segmentation of BraTS2020 - Part 1 - Getting the data ready

[@ssshukla26]

I have watched the bits and pieces of this series. I am going to watch the entire series again. What a superb content and that is for free. Thank God people like you exist. Thank you for this series.

[@DigitalSreeni]

Welcome aboard!

[@sayantanbhowmik4196]

thanks for explaining the basics and not considering as prerequisites . Many many thanks SRINI SIR

[@filip4370]

@DigitalSreeni I would like to thank you a lot. You are probably covering every necessary topic in deep learning. I am doing project with multiclass semantic segmentation of brain tissue and I'm struggling with fg-bg class imbalance. Your videos are very helpful with that issue.

[@aryansakhala3930]

This is cool prof, I hope no one sees your video and all your information is preserved with me.... kinda jealous XD :P

[@DigitalSreeni]

Thanks! 😃

[@mehdialibegli8233]

Hi and thank you for your good presentation. You may also teach about the severity, grade of tumor, and overall survival rate at BraTS 2020 following this discussion.

[@saltanatkhalyk3397]

Awesome, thank you for making such clear and useful videos.

[@DigitalSreeni]

You're very welcome!

[@mramzanshahidkhan3917]

Thank you so much for these very informative videos. These are very helpful to understand the segmentation of biomedical images.

[@DigitalSreeni]

Glad you like them!

[@MuzammalShafique-xn4do]

20:52 Hey professor. Where are we saving T1?

[@NomanArif-n3u]

Do make a video on training the nnUnet and UNet with transformers using 3D dataset

[@ThisIsAli1983]

Thanks for the detailed explanations.

[@marydayanaa9130]

Thank you sir for sharing wonderful information. Please post a video using diabetic retinopathy fundus image dataset.

[@tigabuyaya9039]

I like all your videos. I hope this task will end up with a 3d brain tumor segmentation.

[@AbhishekSinghSambyal]

Please upload other set of videos more frequently.

[@DigitalSreeni]

I schedule all my topical videos to be released on Wednesdays. If I publish a video each day, I will not be able to come up with useful content every week which results in long gaps in content delivery.

[@AbhishekSinghSambyal]

@@DigitalSreeniIn a series like this, please try to make it atleast twice a week. I feel once a week is not apt as you will only have 4 videos in a month. In case you are making larger series (maybe 7-8 videos), it will take almost 2 months, which in my view is a huge time span. As a current viewer i feel it disentangles us from the series content. It is just a suggestion. Like your videos. Thanks.

[@RaniaBensaid-y8c]

Thank you :) , this video really helped me a lot.

[@waddahalhajar]

the validation data folder does not have segmentations ! , just 4 images

[@r.walid2323]

Hi sir, thankbs for the explnatation, can yopu please attach the requirements.txt file as well as the used python version

[@SusmitaDas-gn5de]

kindly perform lung nodule segmentation

[@random-yu5hv]

Excellent explanation! very appreciate

[@utkarshi7588]

Sir can you make a video on malignant lymphoma classification

[@__-pd7tc]

Thank you sir for making this great video with good explanation. but why I didn't get the same result as yours? the mean IOU that I got is 0.24 and has a worth segmentation result . would you help me with that please?

[@imeshadesilve8626]

Great video. dear sir can you do a video for a building detection using spacenet

[@saqibqamar9270]

@DigitalSreeni Thanks for letting us know very well and easy implementation. I have question about your cropping to take mid part of each slice from 3D image. It might be issue to lack of information while we crop the image to mid. Would it be single not file for single 3D image?

[@msbrdmr]

The validation part of the dataset does not have masks. How can I proceed if I want to use that instead of splitting the train data ?

[@govtjobs7063]

sir where do you get combined tff file of all mri...kindly response

[@ramagarg2793]

Superb sir, Thank you for this lecture. I want to make a request that can you please upload a lecture on data augmentation in BRATS Database.

[@dimitheodoro]

great great great!! ImageDataGenerator with .flow method ,as i see can read .npy files.. In case someone doesn't want to create his custom generator function

[@jupieterexim9300]

Getting stuck wit this error when I run the part after stacking - "index 0 is out of bounds for axis 2 with size 0" . Anyone knows how to fix this?

[@KPSdiary]

please do some lectures on ADNI fmri dataset pre-processing with afni.

[@DigitalSreeni]

I have no idea what "ADNI fmri dataset pre-processing with afni" means. In general, my videos are intended to be tutorials where you learn many aspects of data handling and processing. You should be able to apply the principles to your own applications. I hope there is nothing tricky with your FMRI data so may be you can apply my video tutorials towards that data set.

[@kanchanrani5674]

Sir, I have trained 2d unet model on BRATs dataset. First time I executed the code on 100 epochs, it predicts only one label (0-normall class) out of total tumoral classes. while executing the same code second time again on 100 epochs it predicts for all the tumoral classes(0,1,2,3,4). what could be the possible causes of this erratic behaviour in results. Kindly help me in resolving this issue. Thank you very much sir for these video lectures!!

[@madhavviji7021]

Sir I have one doubt that in which application u do all this?

[@karthikbalamanokaran1986]

Hello Sir, Thanks for your video. I'm doing the same exercise in my system. Unable to save the 344 segmented volumes in my local files as .npy only C:\Users\karth\brain_tumor\Training\BTDS1\BraTS2020_TrainingData\input_data_3channels\images\image_368.npy this file saved for the for loop execution. kindly help me with the same.

[@akindelegrace4545]

Very explicit video sir, please where did you get the test image you use for segmentation? or how did you save your splitfolder? Also, when I changed my loss function and metrics, am having some errors"ValueError: logits and labels must have the same shape ((None, 512, 512, 1) vs (None, 512, 512, 1, 3))"

[@tejendraraom3074]

Hi sir, i just checked the dataset link it is showing only 4 gb. is there any changes in current dataset

[@connielee4359]

Thanks for your useful video. I would like to know is there any reference paper on this video subject. Thanks!!

[@DigitalSreeni]

The video content is not presenting original research, it is a method of handling and segmenting multidimensional data sets. It is also not presenting a novel method of segmentation, we are leveraging existing tools to segment our data set. Therefore, no reference paper on the process of segmentation. You can look up U-Net original paper and also information on Brats2020 data set.

[@connielee4359]

@@DigitalSreeni Got it!! Thanks a lot for your reply.

[@pathikghugare]

I had a doubt , Instead of taking a whole volume of shape (240,240,155) why don't we just find a index of the image out of 155 images in a single volume which contains most of the information like a clear image and use that same index to on other volumes too ? So that we will have final volume of shape (240,240,3) which will contain each of the clear images from flair, t1ce, t2.

[@jawher9]

I submitted a paper with a similar approach to the one you've described. I got roasted due to the lack of generalization that incurred.

[@DigitalSreeni]

I don't understand the approach. Going from 240,240,155 to 240,240,3 means you are sacrificing information.

[@patilvinod555]

Sir, Please make video series on Infants MRI Brain tissue segmentation

[@DigitalSreeni]

Not sure of any data sets in this application area.

[@yoyomcg]

Do one on GANs

[@mehul_kamboj08]

greta information sir.... godddd

[@0xcalmaf976]

Dear Sreeni, when should we wait for the next stage video?

[@DigitalSreeni]

I release my regular series videos on Wednesdays.

[@0xcalmaf976]

@@DigitalSreeni Thanks for the reply, I could see that your tf version is 2.4.1, and I assume your is python 3.8 or above. May I exactly know your setup?

[@abdulrahimshihabuddin1119]

Sir, I have a doubt. It's not related to this video. I got most of my practical knowledge from your videos. Have you worked with pretrained neural networks for grayscale images? Most of the transfer learning models are trained on Imagenet which consist of RGB images.

[@mehdialibegli8233]

Hello and thank you. Why should mask values of 4 to 3 be reasign?

[@DigitalSreeni]

I clearly explained that in the video. The data set has missing 3 label so I reassigned all 4s to 3. You do not have to do that but when you try to compute metrics such as IoU, they assume your values go from 0, 1, 2, 3, 4, .... I do not like missing classes so I reassigned 4 to 3.

[@homerianreyes8688]

Great tutorial, it is very clear, i've learned alot! Just want to ask related to CNN, is it possible that I can deploy my CNN model in real-time? Thank you for the response.

[@AbhishekSinghSambyal]

Yes you can.

[@homerianreyes8688]

@@AbhishekSinghSambyal oh thank you, is there any videos u can recommend about it? Im trying to deploy my model, and I want to detect multiple classes in one images.

[@AbhishekSinghSambyal]

@@homerianreyes8688 you can search for any multi class segmentation network (unet, segnet). Input image dim: batch size X channels X height X width , target image dim: batch size X height X width and the target image pixels can have any value I.e. class numbers(1,2,3,4 - for 4 class problem). I am considering Pytorch framework for dimensions I mentioned before.

[@homerianreyes8688]

@@AbhishekSinghSambyal Thanks a lot!

[@DigitalSreeni]

Yes you can! But, it is not easy to deploy a stable real-time model that works all the time. In other words, there is a way you can make it work as a prototype and there is a way you can put it in production quality environment. You may find helpful content online for the prototype approach. For production scale quality, it usually takes a team with diverse skillset. If you have experience in full stack development, you can definitely deploy it with some effort.

[@鄭博文-d5i]

Thank you very useful

[@pathikghugare]

Any idea how to create a GIF from any one of the volumes ? I tried from PIL import Image from matplotlib import cm def create_gif(volume): frames = [] for img in volume: new_frame = Image.fromarray(np.uint8(cm.gist_earth(img)*255)) frames.append(new_frame) frames[0].save('sample.gif', format='GIF', append_images=frames[1:], save_all=True, duration=300, loop=0) But this doesn't seems to be working as I am only getting a dark image

[@ghulamhussain8424]

OSError: problem writing element 633344 to file ,how to solve this

[@DigitalSreeni]

Seems to be an issue with writing large numpy arrays to file. May be this helps: github.com/numpy/numpy/issues/5336

[@ddarhe]

Thank you so much for these videos! However, I have a question (for anyone that thinks they might help); I am using the segmentation_models library and with an efficientnet backbone and unet top i get over 99% on my training/validation split but out of sample segmentation masks don't look nearly as good. That is kind of telling me I am overfitting on the validation set but does anyone else have any other ideas? Best,

[@kanimozhisoundararajan9850]

Which python IDE have you used for your work?

[@DigitalSreeni]

I use Spyder.

[@marcjourne707]

Thank you very much for this great video. I am a beginner with this type of work and am not quite understanding the cropping to 128x128x128 using "combined_x=combined_x[56:184, 56:184, 13:141]". How did you determine "56:184, 56:184, 13:141" ? Sorry if this is an obvious answer, but I am somewhat confused here. I would appreciate your response. Thank you!

[@DigitalSreeni]

No specific reason for exactly using 56 and 184. I needed an area around the central region so picked a range that gives me central region.

[@fuegopuro5933]

Wow! Super useful!

[@DigitalSreeni]

Glad to hear that!

[@venkatesanr9455]

Hi Sreeni sir, Thanks for your inputs and informative content. I hav one doubt from the video whether uint8 can be saved (.npy) and trained by doing float64 into uint8 using astype(np.uint8) instead of still maintaining float64 in .npy. Kindly reply.

[@DigitalSreeni]

I am sorry, I don't follow your question. If the question is whether you can save uint8 instead of float64 then the answer is Yes you can save any format while saving as .npy. I stored them as float64 so I don't have to deal with preprocessing steps anymore after loading the data.

[@ShahryarSalmani]

Dear sir, first of all nice video no doubt, but you are mistaken somehow in cropping the image in around 15 minutes, you have 155 slice for every modality (t1,t2...). Cropping the width and height of image is completely ok and reasonable, but cropping slices means nothing. you cannot crop them. you are just selecting 128 slices. @DigitalSreeni

[@DigitalSreeni]

Not sure what you are trying to imply here, may be I am not understanding your concern. You seem to say that slicing arrays in x and y direction refers to cropping and slicing the same array in the 3rd dimension (z) is not cropping but 'selecting'. Not sure why you'd change the terminology for z and not for x/y. Selecting 128 slices is same as cropping the volume.

[@ShahryarSalmani]

@@DigitalSreeni yes i know the terminology is the same. but why should you choose 128 slices? in the third dimension you are choosing 128 slices. but why? cropping the x, y i understand. you are trying to remove useless data. but cropping volumes of image, why? sir i have one more question, how do you interpret the to_categorical of masks? what does it mean when you add a fourth dimension? can you give me a more detailed explanation of why? thank you in advance sir.

[@ShahryarSalmani]

@@DigitalSreeni and it is medical image terminology (third dimension are slices are images because in CT or MRI we have slices of full images packing sequentially.)

[@hasnainalishah8619]

Anyone who is stuck with or Unable to save the 344 segmented volumes in my local files as .npy only. try to run the whole code for getting data ready.. or else it will save only the image 369 and mask 369 in input_data_3channels.

[@b.kannan2668]

Amazing sir

[@rathinhalder5034]

@DigitalSreeni sir, I have a .nii file . After converting into NumPy array it shows that the shape of the image is 192,192,19,1,80. What does it meant by and how can I convert it into 192,192,19

[@DigitalSreeni]

.nii files can store multiple dimensions and usually the first three are for x, y, and z. The fourth one is for time and the last one is for multi channels - although 80 sounds like way too many. I am not sure what you mean by converting your image into 192, 192, 19. If you just need to pick those dimensions, then just drop the last two.

[@rathinhalder5034]

@@DigitalSreeni Thank you sir.

[@sabirkhanakash]

Sir, I have a question, there are 5 files (flair, seg, t1, t1ce, t2) in each of the training data directories, where seg file is the segmented or ground truth image. But for validation data directories, there are 4 files in each of the directory. How can I get the truth image / equivalent seg file for validation data directory? Thank you.

[@DigitalSreeni]

Validation data does not have ground truth as it gets used by kaggle to validate the accuracy of your model.

[@zaidilyas5192]

Hi Sir! Nice Explanation! Thank you for making such videos. I have a general question, is there any relationship between the number of images required for training U-Net for semantic segmentation and number of classes in the case under consideration (Like in this case there are 04 classes)? In short how much data is sufficient? Regards!

[@DigitalSreeni]

The number of training images is more dependent on the complexity of features rather than the number of classes. Think of it as the number of pixels needed for a given class. If the class is easy to segment (e.g., bright pixels), you do not need many images. If it is difficult (e.g., hard to separate from the background), you need many images. In summary, it is difficult to guess beforehand.