307 - Segment your images in python without training using Segment Anything Model (SAM)

[@Lalit-yw2tb]

This model is brilliant. When I read the news, I was blown away that they made it open-source and allowing commercial use as well. We are living in wild times. As usual love your content Sreeni.

[@jurischaber6935]

Hey DigitalSreeni, I must thank you for your videos helping me a lot with my master thesis, I even got the best degree. Thanks....

[@datapro007]

Wow! I watched your video again and ran through the example you provided. SAM is a great annotation tool and a real labor saver. Fabulous presentation Sreeni.

[@datapro007]

What a great and timely video. Thank you! I wasn't aware of SAM prior to seeing your presentation.

[@biswanathsaha2208]

The model is so good, in our lab we need segmentation for multi-channel images, it definitely streamlines lot of things

[@mohammedkrameche5378]

Do you think is can segment a set of elemental maps from EDS mapping?

[@yoverale]

Thanks! Impressive results and perfectly short explanation. Fast and clear

[@OlgaChambers-x7z]

thanks a lot, it works great for my microscopic data, just very slow segmentation for live monitoring

[@maxgadd]

your contributions are amazing. wow. thank you so much

[@sara_kassani]

ViT stands for Vision Transformers. I used UNETR and SwinUNTER (vision transformers) and both of them had excellent performance specifically if your dataset size is very small.

[@faicalammisaid3705]

i have 1600 pair image and mask dataset and i am having hard time with unet and others i might got to ViT and save time

[@rajdeeppawar8287]

Won't you have to train them? Or are they Few-shot too

[@texasfossilguy]

How do you apply labels to them? Making a mask is one thing, but how do you label classes?

[@ozanapaydin15]

Thank you for preparing this really really helpful tutorial and also teaching this model in a very simple way!

[@edmald1978]

To extract labels I did this function maybe can help: def create_labeled_mask(anns, height, width, max_num_classess): if len(anns) == 0: return sorted_anns = sorted(anns, key=(lambda x: x['area']), reverse=True) sorted_anns = sorted_anns[:max_num_classess] label = 1 labeled_mask = np.zeros((height, width)) for ann in sorted_anns: m = ann['segmentation'] labeled_mask[m > 0] = label label += 1 labeled_mask = labeled_mask.astype(np.uint8) return labeled_mask

[@sudoshivam]

SAM is segmenting multiple instances of same object in different colours. For ex. I tried SAM on an image containing many flowers, the model segments same type of flowers separately in different colours. So if I apply this function on my image it will assign different labels to all flowers even though they belong to same class.

[@Brickkzz]

A video for fine tuning Segement Anything Model would be great

[@niteshtakarker]

Thank you for the informative video tutorial! I'm currently working on agriculture land cover monitoring, and I'm wondering if this model can be used for that purpose. Can you confirm if it's suitable for agriculture land cover monitoring?

[@yangyang6008]

Thank you for sharing! By the way, I like your Spyder IDE.

[@hikvruzhunter2747]

Hi, could you make a tutorial on how to connect SAM outputs with convolutional networks to determine what object it is or extract information? Regards. Great video!

[@ajaypatro1554]

Hy, sir I am from Geoinformatics background, and I want to get started with Py to process image processing on remote sensed data, where should I start what Gui would you recommend for Py Spyder or vs code is fine i want to visualize the data too while working on it.

[@streamingdev1163]

I would love to know how they do the hover and onclick segment / mask

[@sithibanu9019]

This is a useful information for my research. Sir can you make a video for prior setting like installing sam, transformer, checkpoints.

[@ftmftm7627]

Finally omg 😃 Sir please use pytorch

[@raphaelmachado9437]

please more videos about this model 🙏

[@aprashnani]

Does this make the other models we discussed on this channel, like UNet, ResNet etc., obsolete? Is it still worth it to learn about them?

[@Syedneloyahmed]

Hello, Thanks for the great video. But I have a question, Can I export the masked output as shp or tiff format?

[@domillima]

Do you any video where you show how to incorporate a (manual) segmentation from 3D slicer into a CNN using CT scans ?

[@rushirajparmar9602]

Heyy, thanks Sreeni for your awesome content as always! BTW is there any way I can access a specific portion of mask from the segmentation mask dictionary?? Let's say if I just want to access the mask value where the person is in the image? This could also differ from image to image so I don't know if it works

[@dr.aravindacvnmamit3770]

Excellent one superb

[@jurischaber6935]

Danke!

[@DigitalSreeni]

Thank you very much.

[@kelixoderamirez]

permission to learn sir. thank you

[@DigitalSreeni]

Always welcome

[@carthagely122]

Thank you very much

[@senthilkumart.k8605]

Thank u for this excellent video sir. Currently this is doing instance segmentation. Is it possible to alter this code to perform semantic segmentation ?

[@shantilalzanwar8687]

Nice one thanks

[@gustavojuantorena]

Thank you for the video

[@vihalkaviyarau9044]

Hello. I am in need of binary segmentation. Can you do a video on that and explain it clearly like this. Thank you

[@elmerjaen]

Great tutorial! Thanks for sharing. I would like to create a new cut out image based on the segmentation made by the model. The web demo of SAM already does this and I would like to do the same but using python. Any ideas in how should I attack this problem? Thanks in advance!😄

[@anirudhgangadhar6158]

Do you mean you want to create a binary image after segmentation ? You can just use the bounding box info. of the mask if that's the case.

[@shivangi9]

this model generate masked image if single image., what if I want to generate 2000 images mask from this model , is there any step?

[@nyariimani7281]

SWEEEEET!!! Thank you for this!

[@AnoldMupa]

Hi. I am trying to perform Slicing Aided Hyper Inference with the Segment Anything Model to detect smaller objects on an image. Is it possible?

[@AntonyJerald-e3e]

Can you please show how to segment an object using the text prompt

[@tektronix475]

Hey there ppl! . Is it possible to test the model in CPU? How?. I got an: CUDA is available: False warning trying to do that.Thx!

[@tektronix475]

m sorry, it took ages to segment with cpu, but finally made it.

[@OmoregieBright-p2j]

I use a MAC -> Anaconda -> Jupyter notebook. it says CUDA is available: False. How can i fix this?.

[@GiangTran-u6s]

Can we apply this method to 3D images?

[@bharaths5603]

How does it work on images without dye staining?

[@gracealkhawand5413]

CUDA works only on windows and linux apparently, is there a workaround to make it work for Mac, please?

[@ashav6590]

Sir, Getting error with sam chcekpoint please suggest how it can be resolved

[@faicalammisaid3705]

how to controle segmented classes like only houses or road and houses only

[@muhammadhuzaifa-11]

How do I use this model either locally or via api?

[@prithvirajpani7190]

Hi Sreeni. Thanks for this lively tutorial of SAM examples. May I know the GPU memory you have? I have run the house.jpg (which is 200KB) and neurons.jpg (which is examples on my CPU (with 16GB RAM) and they worked fine. But when I tried to work with an image that is 25MB size, my system reported as OOM. So, I was wondering if we need a high memory GPU or high RAM CPU to run this for even simple images?

[@DigitalSreeni]

16GB GPU

[@jlgiorio]

@@DigitalSreeni Are you using GPU or CPU? My GPU (4GB) gave OOM for me too. Thank you in advance!

[@los4776]

Thanks!

[@varunkota1289]

I don't have a GPU but I have a colab pro plus subscription, can I run this on Colab, can somebody tell me how I can do that please?

[@buketkaraoglu683]

How can ı labelimg data using sam's mask output? Is this possible?

[@porchelvananbarasu1697]

How to save the mask as vector or the masked image ?

[@edmald1978]

How to get the labels of the segmentation?

[@itayhilel2168]

can you do a video using the text prompt with the code?

[@tilkesh]

Thanks

[@ChandreshTrivedi]

Thank you. Other than the fact that it is open source, what advantage does this toolkit offer over Image-Pro and it’s programming environment for microscopy?

[@sofiavaldez500]

I would imagine researchers will want to use SAM to pre-process images for input into other models. But, it seems that SAM takes 20-30 seconds to segment images which is not feasible for thousands of training images. What are your thoughts on how SAM can be used in cases like these? Do you think there will be a way to speed up the segmentation per image?

[@chironlionel6770]

to annotate more quickly..

[@raounekzeghdoud1023]

@@chironlionel6770 Hi Lionel how can i use sam to annotate an image? thank you

[@chironlionel6770]

@@raounekzeghdoud1023 Hi Raounek, well in my case I have a GPU with 8MB of memory which is too small for using the main SAM model.. the intermediate one (which works on my GPU) does not for moment seem to be as good as the big model unhappily.. for a model I need around 20 pics so the time Sofia was speaking about is not for me a limitation.. also passing by SAM is always faster.. one disadvantage is when the objects are not well separated.. I need quite often to segment organisms which are stuck one to another and in this case I would be obliged to correct the masks since the model segment "too large".. To sum up I think it can be very useful if you are in the good conditions.. I tried also the One Shot Learning with perSAM but did not work as expected.. It could be a solution for making an annotating tool well customized ! but for the moment I don't manage this as good as I want but still quite promising !

[@raounekzeghdoud1023]

@@chironlionel6770 Thank you for your response and for clarifying. In my case, I have already extracted contours around each object from the segmentation. However, I am unsure about how to convert these contours into masks that can be used for annotations. Can you help me on this

[@raounekzeghdoud1023]

PS: i used another algorithem and not SAM

[@shantilalzanwar8687]

I am getting CUDA FALSE, sorry but simple question, how to install it ?

[@johnnysmith6876]

Can you do binary segmentation using SAM?

[@Feanor102]

If you find anything could you please share it ?

[@johnnysmith6876]

@@Feanor102 Will do!

[@benoitchouinard4240]

Hello, what kind of GPU are you using? I want to know if it's possible to do this in real time at at least 1Hz for video.

[@Eng_world]

Can we used this one on segmentation of geotiff file format data?

[@mohajanemeriame1051]

Hi , did you get reply, if yes please do share with me. Thank you

[@sarathkumar-gq8be]

how to measure the area of segmented image??

[@DigitalSreeni]

Just sum up all the segmented pixels.

[@sarathkumar-gq8be]

@@DigitalSreeni I do one project, for that I'm using kaggle alzheimer dataset using i do classification after classify I do segmentation, So here my point comes , I want measure the area of alzheimers in real world of kaggle dataset images , that how would I do sir??

[@Feanor102]

I'm trying to create tumor segmentator with SAM. When i give mri images to it, it is segments anything in the images absolutely magnificently, but the problem is that it segments all the things and it does not gives any meaningful label. I can't distinguish the tumor masks among all the maskes. I don't know how to do it. I'm open to any suggestion. I apretiace.

[@mohammedkrameche5378]

@DigitalSreeni an example for segmentation with training plz ?:D

[@theforester_]

can we use tiff images?

[@berndpeglow701]

*promosm* 😂

[@edmald1978]

Thanks!

[@bindurao3463]

Thanks

[@sreehlalam]

Thanks!

[@DigitalSreeni]

Thank you :)