194 - Semantic segmentation using XGBoost and VGG16 imagenet as feature extractor

[@newcooldiscoveries5711]

I've taken many tutorials before. Some teachers should not be teaching. You are a natural teacher. The information is well presented, in a pleasing manner, and your knowledge is evident. Thank You.

[@DigitalSreeni]

You're very welcome!

[@giulsdrll]

Thank you for the semantic segmentation videos, it is really difficult to find high quality videos like yours. Keep it up with these excellent tutorials 💪

[@DigitalSreeni]

I plan on doing more semantic segmentation videos, especially using Unet and may be applying the trained model to large images. I hope the viewers find them useful.

[@giulsdrll]

Can't wait to see more about it :)

[@drforest]

Thanks! This is a general thanks for your videos. They're great!

[@DigitalSreeni]

Thank you very much Ken - really appreciate the generosity.

[@bellooluwaseyi4193]

Well done sir, always spot on in your explanation.

[@DigitalSreeni]

Glad to hear it

[@dardar9913]

Appreciate the detailed explanations Mr Sreeni, please keep up the good work!!

[@AbdullahJirjees]

Hi In the line dataset['Label'] = Y I am getting this error ValueError: Length of values (0) does not match length of index (9179136) any idea ?

[@marwanabdou1644]

did u find a solution sir cuz am facing the same problem

[@ravibhushanthakur8913]

Thank you sir for the video. the type of 'prediction' in my case is Array of int64 instead of Array of uint8. why?

[@AbdullahJirjees]

I redo everything all over again, and I realized when I used Spyder, when I read the SIZE_X =1024 and SIZE_Y = 996 it didn't display in Variable Explorer in Spyder!! does that mean there is issue somewhere? while when I move to the line train_images = [ ] it display in Variable Explorer

[@ayushmanchakraborty2296]

sir in the above mentioned folder im not getting the test mask. Its cropped 768X768. Could you pls help ?

[@ShawnH0914]

Amazing tutorial, I learned from you A LOT, thank you for sharing.

[@DigitalSreeni]

Glad to hear that!

[@XX-vu5jo]

Please make a video dedicated to creating a dataset for this type of task and models. Like a really random dataset that we can turn into a valuable one. Please notice this sir. Thank you! I love your videos!

[@DigitalSreeni]

Sure will do. Thanks for the suggestion.

[@XX-vu5jo]

@@DigitalSreeni will be waiting for it! Please! ❤️

[@jayayadav-ih1yz]

Hi Sreeni, your videos are really helpful. I have my own dataset. But i am not able to create multilabel masks from a single image using any free software. Label studio gives binary masks. Kindly help. Thanks in advance.

[@tanmaydeshpande2409]

Hello Sreenivas sir! I am working on a project named *Semantic Segmentation for Autonomous Vehicles in different weather conditions* For this, we are using *A2D2 Semantic Segmentation* dataset. This dataset contains images with their annotations ready. Our aim is to create a model which is robust in different weather conditions. For the adding the different weather effects, have used different Image Processing Techniques for image augmentation for 4 weather effects : rainy, foggy, cloudy & snowy. Now for Semantic Segmentation, we are using *ENCODER-DECODER* model where we are using VGG16 pretrained model on Imagenet + FCN (Fully Convolutional Networks). I am trying the standard process of adding convolutional layers, deconvolutional layers, unpooling, etc for the FCN part but i am not that confident. *Questions:-* 1) How should i approach FCN part? I am doing trial-error for this purpose. Any suggestions. 2) I have created 950 images of each weather condition. The annotated images of all the 4 weather effects will be same. Can it overfit the model where the truth value will be same for all the 4 weather effects? 3) I was thinking of adding another feature extraction NN which will provide information about the weather effects. The o/p of this feature extraction network would be added to the FCN part to increase robustness of the model. Any suggestions or tips will be helpful. Thank you & Stay Safe!

[@Flyforward226]

Hi Sreeni, is that possible to have VGG model and XGBoost training in the same pipeline? Currently, VGG model is not getting trained.

[@DigitalSreeni]

The VGG is just acting as a feature extractor. We are using pre-trained weights to extract features which are fed to the XGBoost. So, no training of VGG, only XGBoost.

[@kunningtang4149]

Hi Sreeni, thanks for the great video. I have a question, are there any ways that we can do to use more convolutional layers in the pre-trained networks, not only the first block? like Upsampling layer?

[@marcsilviu5665]

I have tried to annotate my images using different tools. The problem I have is that when I drag the image in ImageJ i don't have the values like 1,2,3 ... etc and I have values such as 255,180. How can I solve this problem?

[@DigitalSreeni]

I don’t see an issue here. You just read your masks as usual and use label encoder to change them to 0, 1, 2, etc.

[@marcsilviu5665]

@@DigitalSreeni Thank you!

[@giulsdrll]

I've followed the tutorial but the predictions are always black images. What could be the problem? My dataset includes 35 images (and 35 masks). The masks are binary masks: black is the background (pixel=0) and white is the cell I'm interested to predict (pixel=255). There is only one type of cell. So I've a label equal to 0 and a label equal to 255. Even if I do pixel normalization, it doesn't work. Any idea? (For completeness, the dataset is widely unbalanced because the images are mainly black with some white "circle")

[@DigitalSreeni]

Two possible reasons... 1. The images are segmented correctly but you are not saving them the right way (e.g. convert to 8 bit before saving). Also, the image may be segmented correct,y but the pixels values are 0 and 1, so you don’t see it when you view on standard windows photo viewer. Try plotting in python first to confirm. 2. The dataset is so unbalanced it gets overwhelmed by one class making it hard for the other class to be recognized. Obviously, you need to balance the labels.

[@giulsdrll]

Thank you so much for your response. Have a great day!

[@XX-vu5jo]

How can we build a dataset to train these type of models?? Please reply.

[@DigitalSreeni]

I use www.apeer.com to annotate my images. It is free. I will do a video.

[@venkatesanr9455]

Hi Sreeni sir, Highly helpful and informative as usual. Whether other boosting approaches like catboost and LightGBM are useful? for semantic segmentation, waiting for hyperparameter tuning on Xgboost

[@shilpashree4860]

Can u please explain, how to plot ROC curves for segmentation problem.

[@DigitalSreeni]

Please watch my video 145 - Confusion matrix, ROC and AUC in machine learning

[@prep756]

HI everyone, When I am download given dataset then there is only one image instead of 462 images. Anyone know why this is happening with me. please help.

[@DigitalSreeni]

How do you know there is only one image? Which program do you use to open the image?

[@prep756]

I have download the dataset from above link but there is only one image and corresponding label in (partial_labels_for_traditional_ML folder ).

[@prep756]

@digitalSeerni sir please help

[@DigitalSreeni]

How did you open the image? How do you know it is only one image? Did you just use Windows photo viewer? Or did you read the image in python and only see one slice?

[@DigitalSreeni]

I just checked, it has 9 images for each of the images and corresponding masks. I think you are opening the images using some generic photo viewing application that cannot handle multi-image tiff files. Try loading them into python and looking at the array dimensions. Or use some scientific image analysis program like imageJ.

[@fardinsaboori8770]

Hello Sreeni, Thanks a lot for this great tutorial, would you please make a video on how to balance the week IOU class? or at least give us some guidance on how to do it? Cheers Fardin

[@DigitalSreeni]

I've already made a video but with structured data, not image data. However, it should be useful. kzbin.info/www/bejne/jILYe6l9id91ndk Please keep an eye on the upcoming videos the next few weeks. Also I recommend subscribing to apeer_micro channel where I recently covered this topic.

[@fardinsaboori8770]

@@DigitalSreeni Hi SReeni, I have been working on your suggested solution to increase the accuracy of Semantic segmentation by resampling, I did it on my training images and masks and predicted the model on a test MRI image. just to make sure I did it right, I didn't resample the test image because it will mess up the test image and prediction output. right?

[@evyatarcoco]

A great video 🙏🏻 I have a little question : You are using pre-trained vgg16 on imagenet rgb images, But your data is in gray level. Am i wrong?

[@DigitalSreeni]

My data consists of grey level images, correct. But I am reading them as 3 channel (RGB) so I can use VGG16 pre-trained weights.

[@evyatarcoco]

@@DigitalSreeni Thanks 🙏🏻

[@ZhiGangMei]

Hi, could you make videos about instance segmentation? Thanks.

[@DigitalSreeni]

I do not like instance segmentation using mask RCNN as I could not manage decent segmentation on any of the images I tried. It requires tremendous amount of data that I do not have. But I do like to perform semantic segmentation and then use watershed to separate objects to convert it to instance segmentation. I will record those videos soon.

[@ZhiGangMei]

@@DigitalSreeni You are absolutely right about the effort needed for instance segmentation using mask RCNN. Cannot wait to watch your new videos using the approach you mentioned for instance segmentation. Thanks.

[@AbdullahJirjees]

Thanks for this video, I am running this experiment using CoLab, and the section of extracting the new features which is: #Now, let us apply feature extractor to our training data features = new_model.predict(X_train) I am getting this error 1623 callbacks.on_predict_begin() 1624 batch_outputs = None -> 1625 for _, iterator in data_handler.enumerate_epochs(): # Single epoch. 1626 with data_handler.catch_stop_iteration(): 1627 for step in data_handler.steps():

[@ferielfettal7703]

I have the same error, is that you have found the solution

[@MonikaKumari-yr3dt]

Can you please do a video on autoencoders for a multiclass classification problem for tabular input(csv)?

[@DigitalSreeni]

By definition autoencoders reconstruct original input data, so the output would be your original input. You can look at reconstruction error to design an anomaly detector. If you want a classifier you can train the autoencoder and chop off the decoder part and only use the encoder for feature generation. Then you can add your classifier of choice to make the whole thing a classification solution. I will try to record a video on this process.

[@MonikaKumari-yr3dt]

@@DigitalSreeni autoencoder.fit(x_video_scaled, x_video_scaled, batch_size =16, epochs = 100, shuffle = True, validation_split = 0.20) What should be changed in this piece of code for multiclass problem? There are 8 classes: video, audio, chat, p2p...

[@DigitalSreeni]

You are fitting an autoencoder with input and output set to same, x-video-scaled. You are not predicting / classifying anything here, you are just training the network to reconstruct your original input (x-video-scaled). This is not set up to do classification. For classification problems you need to define X and Y where X will be your inputs and Y will be prediction classes. If you want to,use your autoencoder structure please go ahead and train the autoencoder and after training only take the encoder part of the network and combine it with a classifier where you define X and Y.

[@MonikaKumari-yr3dt]

@@DigitalSreeni Can you please share the piece of code where we fit the autoencoder for classification. Thanks.

[@MonikaKumari-yr3dt]

I am following this :machinelearningmastery.com/autoencoder-for-classification/

[@vulnerablegrowth3774]

I see, so xgboost trained on nn weights and actually perform comparably to Unet. Did you use a Unet from scratch or a Unet with a backbone?

[@AbdulQayyum-kd3gf]

Great video.

[@nippondatta2050]

sir, kindly make a tutorial on creating masks from images for segmentation.