Segment Anything Model (SAM): Build Custom Image Segmentation Model Using YOLOv8 and SAM

[@vivekpaswan7946]

Getting AttributeError: 'Boxes' object has no attribute 'tolist'.

[@Htyagi1998]

The bounding boxes you will get from yolo will jot be in the format which SAM accepts, so you first need to covert the bounding boxes into the boxes which SAM accepts and then you can invoke SAM model

[@vanditaagarwal8802]

how to get your code? The code which you have showed in this video

[@CodeWithAarohi]

This code is for members

[@PrakashSuthar-d1q]

great explanation, very useful output, but is there any possibility to change the segmented wheel of car with another wheel and should be fit only the presented mask area not to overlap with car??

[@nitishrathore14]

Ultralytics isn't commerically viable to use, is there any other open source distribution that is free to use commercially and is fast?

[@CodeWithAarohi]

yolov7

[@nitishrathore14]

@@CodeWithAarohi YoloV7 isn't accurate with videos. So I am looking for automatic video instance segmentation which should be fast and as accurate as possible, while Segment Anything Model is extremely accurate, but because the automatic generator is giving about 100 masks for a single image it takes a lot of time to render those masks for the images itself. Do you think PixelLib is a good solution or any other open source distribution's you'd suggest?

[@TomTran0206]

Hi, greet video! If possible, could we have access to the github so we could implement it in our project.

[@CodeWithAarohi]

Mail me at [e-mail]

[@NMs-w2h]

Thank you so much for your tutorial！I want to know how can I create my own data to train SAM?

[@CodeWithAarohi]

SAM is already trained on huge dataset. You can have your object detection mode and train it on custom dataset to detect custom objects and send those custom objects to SAM for putting a mask on them.

[@NMs-w2h]

Thank you for your reply！ May I ask for specific instructions on how to create a custom dataset?@@CodeWithAarohi

[@LaviSrivastava-u5y]

please provide the notebook file of this video on custom dataset with SAM

[@alexsudakov8554]

Github link is not available. Please, send me a link to the notebook from vigeo

[@soravsingla8782]

Hi Aarohi, your content is excellent and your channel is one of the best Artificial Intelligence channel but still not getting that much of likes which your channel deserves. Hope you succeed #AI #ArtificialIntelligence #DataScience #EducationalContent

[@CodeWithAarohi]

Thank you so much for your kind words and support! It means a lot to me. 😊🙏

[@ayanbahukhandi1869]

Hello, Im working on a similar project but instead of truck i want to segment human faces out of an image, i've got to the point where i pass the face to the model, but the model is segmenting different parts of face, like nose, eyes etc etc. its not segmenting the face as a whole. How can i fix it? so that it segments out the whole face.

[@mahejabeenhossain9727]

That was really helpful! I would love to know how you save the segmentation so it can be used for training later. I am planning on using this to automate my image annotation process.

[@CodeWithAarohi]

Check this video: kzbin.info/www/bejne/gZC6iqCpjs11e9k

[@steriodsgaming]

i want to train this SAM model to make it work for medical data-set

[@BabasahebMane-w2z]

Hiii Arohi thanks for the video it helped a lot. I want to know how can we get label on masked and bounding boxed object.

[@CodeWithAarohi]

Use a detection model to plot bounding box and label. Then draw mask on the detected object using SAM

[@nadeembaig5943]

Can we isolate and extract only the truck by using the mask coordinates, to perform color palette generation only on the truck and not the background?

[@CodeWithAarohi]

Yes

[@aditiprasad5549]

Hi Arohi, Thanks for the tutorial. Is it possible to get shape of object from the segmentation mask if it is then how can we get it?

[@CodeWithAarohi]

Determine a threshold value that separates the object from the background. This threshold value will depend on how the segmentation mask is encoded. For example, convert the mask into a binary mask (0 and 1), you can set a threshold of 0.5 to convert the mask into a binary image where the object is represented by 1 and the background by 0.

[@aditiprasad5549]

@@CodeWithAarohi Thanks for the reply but let's say we get a binary contour after doing this then how can we classify the shape of object as rectangle or triangle or circle etc.

[@fatemadalal4370]

Thank you for the tutorial. Found it helpful. Also found a couple of other useful tutorials on your channel will be checking that out as well. I would like to ask if you have experience in reinforcement learning as well and have implemented that specifically for deep learning related tasks such as model training etc. If yes and if you could make video on that it will be helpful. Thank you!!

[@fatemadalal4370]

Also I couldn't find the link for training custom object detection model.

[@T-mac-c2y]

Great video! Thanks! As I want to use SAM and YOLOv8 both to do a classification task, and the raw images are medical images. So lack of enough images, the pretraining model doesn't perform well. Any suggestions on that?

[@CodeWithAarohi]

Data Augmentation can increase the training samples

[@nkirankumar8294]

Is there any way if we just get the mask layer, instead of overlay it with the images

[@shreetejghodekar9715]

Hey sorry, no offense but i want to know when you have yolo segmentation model then why to use yolo and then SAM?

[@CodeWithAarohi]

@@shreetejghodekar9715 Because for yolo segmentation, you need to first prepare dataset to train the model on custom classes for custom segmentation task. And in this video we are not preparing dataset for image segmentation task even for custom classes which saves lot of time. Annotation of image segmentation dataset is more complex than annotation of object detection dataset. So, we can prepare custom object detection dataset and then use that output as input to image segmentation model (SAM) to get masks on custom classes.

[@shreetejghodekar9715]

@@CodeWithAarohi Thanks for information, as we actually are looking specifically into custom dataset that doesnt detect other things as it affects lot of predictions. So we are training custom dataset. Can you suggest how do we get 3d mask instead of 2d for a cuboid object like parcel as it is not possible using yolo segmentation or SAM i think.

[@shamukshi]

hi...do you do freelancing ? my ACADEMIC topic is "solar panel detection from UAV image using SAM"

[@CodeWithAarohi]

Please email me at [e-mail]

[@qqD-h2r]

Can I separately train the generated masks from SAM to obtain the class labels for the masks?

[@CodeWithAarohi]

My view point is : The Segment Anything Model (SAM) predicts object masks given prompts that indicate the desired object. Prompts can be provided via the predict method to efficiently predict masks from those prompts. The model can take as input both point and box prompts.Also, SAM model predicts object masks only and does not generate labels. Suppose you have a image which have person and bus class. SAM will put masks on both the objects (it will even put the mask on subparts of the objects like separate mask for tyres, windows etc.). We can get a mask for specific object using point or box prompts. Suppose we chose box prompt to display mask. When we pass our image to SAM, it will give us boxes for all the objects. But we only want a box on Person. To put the mask on the box that corresponds to a person, you need to first identify the mask that represents the person. You can do this by checking the label associated with each mask and selecting the mask with the label "person". SAM is not trained on corresponding labels. Here you need a detection algorithm which will give you the box co-ordinates of the object you are interested in and will help you with labels.

[@qqD-h2r]

@@CodeWithAarohi Thank you for your response. I have been thinking about how to efficiently deploy the SAM model in practical applications. It would be really cool if SAM could automatically display the object labels and masks.

[@staudammrecords5774]

HI, thanks for the tutorial - very insightful! is the code from your example available somewhere?

[@CodeWithAarohi]

This code is available for channel members (Contribution Level-2)

[@alocatnaf]

thanks for this nice tutorial! where can we find this jupyter notebook? thanks alot in advance!

[@CodeWithAarohi]

Code is only available for channel members.

[@tomtouma]

Great video. Is there any way to use yolo or opencv to identify the object and pass it to SAM for segmentation as a video stream? Any tips on that?

[@CodeWithAarohi]

You can provide video stream to Object detector (yolo) as input using OpenCV. Object detector will read each frame 1 by 1 and will perform detection. There you can use SAM after the detection on each frame.

[@tomtouma]

​@@CodeWithAarohi Cool. Can I ask you some more questions? I am thinking of leaving my camera in a fixed/static position. And when someone walks into the frame, I want it to be able to give me geometry information about them. Height and width of the head, and shirt and pants. I am sure there are CV libraries out there already but I want to use SAM to segment the head, shirt, pants and then use python somehow to get the geometric information of these segmented parts. Do you think you could help me decide what the pipeline for this should be and what tools I should be using? Thank you!

[@tomtouma]

Just thought I'd bump this.

[@sarahnawoya498]

thank you madam for the nice video. how can I extract that mask to use it further in the code? for example to binarize it

[@etimbukukpe5992]

Hello, I am working on a custom dataset (skin cancer image). I have annotated the images using roboflow and trained it using Yolov8, I want to use Segment Anything Model to put mask. I tried this on some skin cancer images, but SAM predicted them as Pizza

[@CodeWithAarohi]

SAM model doesn't provide class labels. It only put mask. So if you are getting output as pizza that is the issue with your detection model. Check if you have correct classes in your detection model. Check this page for more details on how SAM is trained: ai.meta.com/datasets/segment-anything/

[@adnane4008]

How decrease the inference test time for SAM

[@deepikasharmar1778]

👏👏👏👏👏 Mam waiting for ur video on tensorflow in jetson nano !!!

[@CodeWithAarohi]

Will make it soon

[@vishushah2232]

Hello Aarohi, It was an amazing implementation. I wanted to know if that after applying masks to the particular object, if I want to change the texture/colour of that particular object in the image, how would that be possible?

[@CodeWithAarohi]

It is possible but I need to try it first in order to answer this query.

[@vishushah2232]

@@CodeWithAarohi Can you please try it? I am stuck at this point in a project and want to learn further processing, it will be really helpful !

[@PankajKumar-j6v2l]

Thank you so much for your tutorial. I would like to request that you make a video on Road lane line detection and road lane detection using any deep-learning model (Segmentation anything model (SAM) + Any deep-learning model like ResNet 50 etc..) Thank you

[@CodeWithAarohi]

I will try

[@rajkumarsrini4821]

Instead of single class can I segment multi object in image using custom object model

[@CodeWithAarohi]

Yes

[@sachinkapoor-c9k]

aarohi where is the link for yolo v8 custom object detector. could you please provide here

[@CodeWithAarohi]

kzbin.info/www/bejne/nJnddKqAoKmChKc

[@sachinkapoor-c9k]

@@CodeWithAarohi thanks. i must say your teaching skills is very crisp and good

[@mohammedelmehdimehaya2875]

you amazing i'm one of your followers from long time

[@CodeWithAarohi]

Thank you so much for your kind words! It's always great to hear from loyal followers like you.

[@rishabhsharma2776]

ma'am, the github link is not working

[@CodeWithAarohi]

github.com/facebookresearch/segment-anything

[@jovanastojanovic1710]

Hii, I used this video to help me with doing my college project. What should I write as a reference?

[@CodeWithAarohi]

Hi! I'm glad the video helped with your project! 😊 For referencing, you can use the following - Segment Anything Model (SAM): Build Custom Image Segmentation Model Using YOLOv8 and SAM (kzbin.info/www/bejne/jnOcq5pvbt1loKc)

[@angelospapadopoulos7679]

amazing work as always! We would like to see an implementation of SAM with text prompt (till now we have point and box as input prompt)

[@CodeWithAarohi]

Will try to do that soon 🙂

[@pavanchandravundurthy6740]

Text prompt is not released by SAM yet, as I understand. As of now, only box and point.

[@kvaibhav23]

Thank you, this is too good. COuld you please share the code.

[@CodeWithAarohi]

Segment anything code (each and every line) from github.com/facebookresearch/segment-anything/blob/main/notebooks/predictor_example.ipynb Paste this SAM code after detection code. Detection code github.com/ultralytics/ultralytics

[@mohd.abdulghani6172]

hai, can you make tutorial about how to deploy yolov5 instance segmentation on android? thank you

[@CodeWithAarohi]

I will try!

[@bryanttello7142]

hello, do you have github?

[@CodeWithAarohi]

Not for this one

[@DevmaniRajvanshi]

Hello Madam, Thankyou for the tutorial it was very informative. I have implemented this code on CPU. However when I implement it with GPU (Nvidia GTX1650) I am getting this error: attn = (q * self.scale) @ k.transpose(-2, -1) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 1024.00 MiB (GPU 0; 4.00 GiB total capacity; 2.78 GiB already allocated; 0 bytes free; 2.85 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF Kindly suggest

[@CodeWithAarohi]

Reduce batch size

[@soravsingla8782]

Thank you so much for sharing this valuable knowledge

[@CodeWithAarohi]

Glad it was helpful!

[@soravsingla8782]

Keep posting ma’am

[@CodeWithAarohi]

Sure

[@shinwarikhan4677]

hello mam .. thank you for such an amazing contents.. we learn much more from your videos... mam can you make video on yolo-NAS and what is yolo-NAS and how it work and how can we detect object using it ..we will be thankful

[@CodeWithAarohi]

Just posted a video on YOLO-NAS

[@shinwarikhan4677]

@@CodeWithAarohi thank you so much

[@soravsingla6574]

Thank you

[@CodeWithAarohi]

Welcome!

[@vignesh_waran]

Amazing tutorial

[@CodeWithAarohi]

Thanks

[@carthagely122]

Very thankful

[@CodeWithAarohi]

Most welcome

[@carthagely122]

Very thabkful

[@CodeWithAarohi]

Most welcome

[@RiseUpCommunity]

❤️❤️

[@soravsingla8782]

Best KZbin channel / Platform to learn Artificial Intelligence , Data Science , Data Analysis , Machine Learning. #BestChannel #KZbinChannel #ArtificialIntelligence #CodeWithAarohi #DataScience #Engineering #MachineLearning #DataAnalysis #BestLearning #LearnDataScience #DataScienceCourse #AytificialIntelligenceCourse #Codewithaarohi #CodeWithAarohi Code with Aarohi

[@jonathanr4242]

Thank you

[@CodeWithAarohi]

You're welcome