Unveiling Meta's Impressive CV Model: Sam 2

[@randotkatsenko5157]

Ive developed a fully automatic video editor fort short form content. It would read the frames of the video, images, transcript and apply various effects based on simple text instructions. THIS Meta's new model is THE missing piece to enable editing more complex scenarios in full KZbin-length videos. Impressive indeed and thank you for sharing this prompty after the release, Sam!

[@samwitteveenai]

Very cool use and makes total sense. These models are getting so good we are bound mostly by our imagination of how to use them.

[@picklenickil]

@@randotkatsenko5157 you have peaked my intrigue.. fellow nerd..

[@navroopsingh8902]

I had the same idea few months back. But had mercy on my laptop.

[@navroopsingh8902]

Can you share the repo link if its an open source project?

[@amandamate9117]

yeah but can you use this SAM2 locally(what hardware you need) and how would you implement in your workflow?

[@nemonomen3340]

Cool to see Meta is releasing a new and improved version of Sam Altman.

[@picklenickil]

Sam talking about Sam! Multiverse of Sam😂

[@samwitteveenai]

lol yes a well named model! 😂

[@starblaiz1986]

I heard you like Sam, so we got Sam to tell you about Sam so that you can put a Sam in your Sam 😂

[@saipraneethdevunuri3156]

You've beat me to it

[@flyingapple7119]

It's very Meta

[@DanieleCorradetti-hn9nm]

You should start covering vision models as you do LLMs🎉❤

[@samwitteveenai]

Thanks for the feedback, was wondering if there was interest.

[@mshonle]

I think the concept of transfer learning is really successful for vision models and hence vision models are a great way to explain this larger concept.

[@dheerajvarma9527]

Please cover industrial usecases using CV models as well

[@samwitteveenai]

Curious are there any techniques you want in particular?

[@DanieleCorradetti-hn9nm]

​@@samwitteveenaicrack detections e segmentation? Is a kind of standard

[@user-ew8ld1cy4d]

Never been a fan of Zuck. But that's all changing now. What hes doing by *truly* open sourcing everything is game changing for humanity. Thanks Sam! Keep it going Zuckerburg!!!

[@thenoblerot]

Love the channel. I appreciate content that's not just all LLMs, all the time

[@samwitteveenai]

Thanks for the feedback, I will probably broaden a the coverage going forward.

[@FredPauling]

More exciting than Llama 405b

[@unclecode]

Wow! What’s left for OpenAI? How can they still be valued at $70 billion? Again Meta released another large model that allows people to generate synthetic data. I think the moat wasn't the LLM was becoming the one democratise it for everyone and Meta did that!

[@tecnom7133]

Thanks for sharing, waiting for a Project using this Model !

[@toadlguy]

This model, released in this way, will lead to so many interesting new applications. It would seem that it's use in sports and fitness analysis could be impressive. And even something like traffic analysis, which is currently being done by expensive systems can be done with a consumer camera and open source software. Kudo's to Meta, Mark, and, of course, the OG Sam for letting us all know about it 😁.

[@paulmiller591]

Oh no I will certainly need to add this to my ever-expanding list of AI investigations. Yes this is undoubtedly worthy cheers Sam.

[@henkhbit5748]

Impressive indeed. 👍 for Meta for open sourcing the model. I suppose Marc like your name and did a rocket launch😄

[@ricosrealm]

Meta is killing it right now with fully open models.

[@NakedSageAstrology]

Good video. Thank you for sharing.

[@ahmadzaimhilmi]

Those who aren't following AI news are missing out on all the advancements in this field.

[@MrDanINSANE]

Is there a friendly user interface project such as Gradio interface to test SAM2 with video locally? (demo is there of course) but Locally will be nice to test with different hardware since SAM2 supposed to be faster.

[@NicolasEmbleton]

This is sick 😮. Thanks for sharing. Missed that.

[@sambarjunk]

Great walkthrough

[@NakedSageAstrology]

Her name is Samantha. Tomorrow is July 31st.

[@sondoan3070]

🎯 Key points for quick navigation: 🆕 Meta released the SAM 2 model, enhancing computer vision capabilities with real-time video processing. 🎥 SAM 2 supports video analysis, capable of processing up to 44 frames per second for real-time segmentation and tracking. 🛠️ The model allows segmentation with prompts and has been simplified for easier use, now integrating temporal memory. 🚀 SAM 2 is six times faster than its predecessor and offers improved accuracy and efficiency for data annotation. 📈 Meta has released SAM 2 with open-source code and weights under the Apache 2 license, promoting broader accessibility. 📊 A dataset of 51,000 videos and over 600,000 masklets accompanies SAM 2, aiding in the development of custom models. 🎨 The model can be used for various effects and applications, including real-time video effects and creative annotations. 💻 Example notebooks provided demonstrate how to use SAM 2 for accurate segmentation and tracking in both images and videos. Made with HARPA AI

[@rohaidyrodriguez7179]

Great for AR

[@florentromanet5439]

Great analysis thanks 👍🙏

[@starblaiz1986]

Meta beating up OpenAI: Everyone: "Pleaaaaaase stoooop! He's already unalive!" 😂

[@albertsitoe7340]

Should run on the Apple Vision Pro with its M2?

[@samwitteveenai]

probably but will need to be converted

[@florentromanet5439]

Meta releases it's 2024 second quarter results tomorrow. Sounds like they spent a "lot" of money 💰 into 405b and SAM2 training... Open-source licensing all this is maybe their way to balance this 😅

[@samwitteveenai]

Yeah it would be interesting if the broke out the costs of these models.

[@Psychopatz]

I hope the gimp team can utilize this

[@samwitteveenai]

yeah you could imagine it could do some nice things in that app

[@SonGoku-pc7jl]

10gb vram? is posible divide 16 ram with 4 vram nvidia 1070gtx? or cuantatizate uint8 or int4 i don't remember exactly word for presicion :P and you can make example of data from sam2 and make something with florence :)

[@nullsmack]

is this something that can be run locally on custom video?

[@ZIYUNI-h5k]

thx for sharing🎉

[@MultiverseMayhemtoyou]

wow AI Apache helicopter ! ?

[@imevano21]

Does anyone else have trouble installing sam2. I have cuda 12.4 and set my env variable but I'm getting: raise OSError('CUDA_HOME environment variable is not set. ' OSError: CUDA_HOME environment variable is not set. Please set it to your CUDA install root. Would really appreciate any help

[@rahulprajapati941]

You'd need to add CUDA and cuDNN paths to the environment variables on your Windows

[@nufh]

Can we run this in normal computer or need a high end one?

[@IdPreferNot1]

Access to demo denied? Can you please provide uopdated link to code, thx

[@frazuppi4897]

I mean the model is generating segmentation mask so it is generative ai

[@fontenbleau]

Police are very interested in this, considering masses of video material they have from public places, laws are very bad about this esp in USA with cellphones wiretapping by Stinger & other systems.

[@samwitteveenai]

This is certainly true!

[@toadlguy]

The police already have technology similar to SAM 2, now this will be available to all of us. 😉

[@WillJohnston-wg9ew]

how do you think this would do for real-time sentiment analysis on a face?

[@pladselsker8340]

There are many really good face tracking and preprocessing algorithms out there. SAM2 would only be able to do the "tracking" part. You would still need to do further processing to infer emotions. You could maybe replace the first and last few layers of the architecture with a custom one, freeze the middle parameters, and train on custom data. This last approach would probably give better results.

[@marioignacio3440]

Is it ready to use now? Im in Asia right now..

[@samwitteveenai]

yes you should be able to use it anywhere, it is an open weights model

[@erobusblack4856]

i tried but it wasn't compatible with my beastly phone 😂

[@erobusblack4856]

my android software experience is perfectly customized

[@tonyppe]

Sam a.i. 😁

[@minhazrahman7085]

Sam, help me on an approach. I want to use the RAG for different type of task. Rather than making knowledgebase, I want it to able to differentiate documents as different reports, and compare-contrsat, search across docs type of stuff. What type of pipeline should I follow? All the Rag models are mostly build to make knowledgebase, not compare, a lot of documents with precise output.

[@samwitteveenai]

you can use meta data to keep their identities separate etc as reports. What else do you want to do with them? You can use query rewriting to get and compare info between the reports etc.

[@KevinKreger]

Ask it to rank or rate instead of compare, if possible. You'll get better results.

[@gokulakrishnanm]

Meta slaughtering OpenAI😂

[@Csepowertrip123]

Jab at open AI look what they named it lol

[@MrNobodyX3]

I don't like the fact that you're basically marketing this and not reviewing it you're not pointing out the glaring issues that you see in the video like the ball not being tracked or the eyeball at the Byrd flickering in and out