SAM 2 | Segment Anything Model 2

[@mylifeasaarav]

I appreciate you creating these videos on modern technology.

[@CodeWithAarohi]

Thank you so much!

[@siddharthsen3394]

Very clear and good explaination. Thank you Ma'am.

[@World-um5vo]

Madam if we want to fine tune the model for videos based on mask decoder how to do it.

[@shekharkumar1902]

As usual great video. Image segmentation. Two questions.. 1st how to find the appropriate point cordinate of the object of interest? 2nd Can you add demonstrations of image and video segmentation other than the existing git repo? Just to see actual model response?

[@CodeWithAarohi]

You can use object detection model to find the coordinates and then pass those coordinates to SAM in order to put mask on.

[@devavratpro7061]

@@CodeWithAarohi But What if I want reverse operation? How to perform object detection using SAM? Any suggestion would be help in better understanding.

[@Guest-r3t]

Wonderful explanation mam

[@CodeWithAarohi]

Thanks a lot

[@kmalhotra3096]

Very nice explanation!❤

[@CodeWithAarohi]

Thank you! 😃

[@kalkagyani1726]

I just wanna know how can we use this model to our project, as an AI enthusiast, one of biggest question is that, is their any need to create our own model now, or we have to change few lines in others jupyter notebook code claim it is as ours? BTW great explanation

[@CodeWithAarohi]

SAM 2 is open-source and designed to enable zero-shot segmentation of objects in images. The decision to create your own AI model versus adapting existing ones depends on several factors: 1- If you have a unique problem or a highly specialized dataset that existing models are not well-suited for, creating your own model may be necessary. 2- If you need a model that is highly optimized for your specific use case, including performance, accuracy, or computational efficiency. 3- If owning the intellectual property of the model is important for your business or research. For many common problems, such as image classification, object detection, and text generation, existing models are mostly sufficient and can be fine-tuned or adapted to your specific needs. Training a new model from scratch can be resource-intensive in terms of both time and computational power, so fine-tuning existing models can be a more efficient approach.

[@CnnrVizuals]

so how can you actually export the masked out subject as a separate video without the extra stuff? Where are the elements of the graph and other assets

[@CodeWithAarohi]

Noted!

[@anujshah7350]

How can we label the identified masked, let's say list of labels? what could be thought process sort of? coming from context like template matching

[@JayYoung-r5s]

This is very helpful to me. However, regarding the details of the memory module, I still need to study them further. I'm just a beginner 😭

[@CodeWithAarohi]

Glad it was helpful!

[@nicolasarayacaro94]

Thanks Aarohi, greetings from chile

[@CodeWithAarohi]

Welcome 🙂

[@lorenzoleongutierrez7927]

Great explanation, thanks a lot 😊

[@CodeWithAarohi]

Welcome 😊

[@GianmarcoGoycocheaCasas]

Thanks Madam, great explanation 👍

[@CodeWithAarohi]

You are welcome 😊

[@RatherBeCancelledThanHandled]

Thanks . Great explanation 👍

[@CodeWithAarohi]

Glad it was helpful!

[@SaumyaSrivastava-y1d]

From where to get python codes for this?

[@nitishrathore14]

Hi can you create a video using tensorRT on SAM2 to reduce inference time?

[@CodeWithAarohi]

Noted!

[@shantilalzanwar8687]

Gr8 video as usual. Pls share code. Also why it is 6x faster ?

[@CodeWithAarohi]

SAM 2 utilizes a more efficient model architecture compared to SAM, which reduces the computational complexity and speeds up the segmentation process.

[@shantilalzanwar8687]

@CodeWithAarohi need to run it and chk. Tt

[@Username56291]

can you do it with videos like the blog meta demo please?

[@CodeWithAarohi]

Sure

[@deemon101]

ok, but if we compare SAM 2 to Yolo, then what are the differences, also I guess different use-cases? I mean, with yolo, I train the model to detect a certain item on a picture or a video, and after that I can feed it random new video/picture and my items will get detected. But with SAM 2, is there a similar use case?

[@work.aniketkumar]

SAM is segmentation mode designed to segment anything in an image where as yolo is object detection model designed to identify object in video. Correct me If I am wrong.

[@deemon101]

@@work.aniketkumar you're correct. My current usecase is to use SAM2 for dataset creation. So I created a tool that uses SAM2 to create segment label files for images. Basically I created a roboflow copy, but it all lives in my workstation.

[@devavratpro7061]

@@work.aniketkumar remember, Mask-RCNN has both segmentation and detection. So can we think of a pipeline using SAM that is similar to Mask RCNN?

[@shantilalzanwar8687]

Aarohi, pip install -e . is not working, you said in description details provided but no details found.

[@CodeWithAarohi]

I have added now. Please check description section.

[@shantilalzanwar8687]

@@CodeWithAarohi thanks

[@shantilalzanwar8687]

Your videos are great help for new people like us. WOndering why you choose anaconda over Google colab for SAM 2 ? ALso in SAM1 they provided colab notebooks from Meta, did they provided those this time ?

[@avinashrai7588]

Mam Can we run it on Jetson agx orin

[@CodeWithAarohi]

Yes

[@mouizjarral6486]

Highly Appreciated Aarohi. When you have time kindly make a videos on Semi and Self Supervised Learning. A Humble request. :)

[@CodeWithAarohi]

Noted. Sure I will make videos on Semi and Self Supervised Learning.

[@mouizjarral6486]

@@CodeWithAarohi Thank you so much. Stay Blessed ✨

[@shantilalzanwar8687]

did you faced this error ? --------------------------------------------------------------------------- OSError Traceback (most recent call last) Cell In[1], line 5 3 os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1" 4 import numpy as np ----> 5 import torch 6 import matplotlib.pyplot as plt 7 from PIL import Image File ~\AppData\Local\anaconda3\envs\sam2\Lib\site-packages\torch\__init__.py:148 146 err = ctypes.WinError(ctypes.get_last_error()) 147 err.strerror += f' Error loading "{dll}" or one of its dependencies.' --> 148 raise err 150 kernel32.SetErrorMode(prev_error_mode) 153 def _preload_cuda_deps(lib_folder, lib_name): OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\ss7a0328\AppData\Local\anaconda3\envs\sam2\Lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.