OneFormer - SOTA Instance Segmentation with Detectron2 | Tutorial | Google Colab

OneFormer is fresh segmentation model that earned 5x state-of-the-art badges from Papers with Code. Model managed to beat MaskFormer and Mask2Former and is now ranked number one in Instance, Semantic and Panoptic Segmentation. Take a look at our Google Colab tutorial and learn how to use it in your Computer Vision project.
Chapters:
0:00 Introduction
0:45 Semantic vs Instance vs Panoptic
1:38 Inference Demo
2:50 Setting up python environment
4:02 ADE20K Dataset
5:36 Cityscapes Dataset
6:23 COCO Dataset
7:15 Roboflow Notebooks
8:04 Measure size of the real life object using segmentation
16:55 Outro
Roboflow: roboflow.com
Roboflow Universe: universe.roboflow.com
Roboflow Notebooks: github.com/roboflow-ai/notebooks
OneFormer GitHub repository: github.com/SHI-Labs/OneFormer
OneFormer arXiv paper: arxiv.org/abs/2211.06220
"How to Train Detectron2 on Custom Object Detection Data" Blog Post: blog.roboflow.com/how-to-trai...
"Measure Distance in Photos and Videos Using Computer Vision" Blog Post: blog.roboflow.com/computer-vi...
Stay up to date with the projects I'm working on at github.com/roboflow-ai and github.com/SkalskiP! ⭐

[@TheBradleydwyer]

✋ Log my vote for the custom training tutorial

[@Roboflow]

+1 💪

[@abdshomad]

+1 👍

[@youtuberschannel12]

@@Roboflow Yes we need a video guide on how to train on custom dataset! And sample notebook as well.

[@computer_vision]

up vote

[@mikeazatov1537]

+1

[@prathameshsarang1593]

Already got this working on Replicate! Excited to use it in production!

[@Roboflow]

Hi 👋! Did you managed to train it on custom dataset?

[@sevemz3359]

Hi thanks for the tutorial, how did you disable the class name/ label at time 10:41? thanks I am trying to disable the class name from showing.

[@hassambaig4434]

@Roboflow, what and amazing video. Next video on "how to train on custom dateset".

[@Roboflow]

It is on our TODO list… however that list is pretty long. Keep your fingers crossed 🤞

[@iamsirprize]

video on how to train on custom dataset with provided notebook just like other models please? Thanks!

[@youtuberschannel12]

Next video on how to train on custom dataset with provided notebook just like segformer please?. Thanks!

[@Roboflow]

There is a really good chance that we will do it! The authors of OneFormer have declared that they are very willing to help us. Stay tuned and give us just o bit of time to get the ball rolling.

[@youtuberschannel12]

@@Roboflow Great! Looking forward to it 😀

[@Roboflow]

@@youtuberschannel12 stay tuned ;)

[@youtuberschannel12]

@@Roboflow Sorry for nagging. Please show how to do data augmentation for OneFormer as it wasn't done for the video tutorial for SegFormer. I'm still struggling to do it till this day.

[@Roboflow]

@@youtuberschannel12 could you submit a feature request here: github.com/roboflow-ai/notebooks ? It is our notebooks repository and we will use to menage requests like that. Thank you 🙏

[@vimalathithand917]

Could you do a video on custom training this model using incremental and transfer learning please! It would be of great help

[@anishkuvelkar1876]

hey , does the colab of oneformer run ? its not working for me.

[@sarahgarcia3063]

Hi ! Thanks for the tutoriel ! I am a begginer and I would like to know how could I save only the annotation mask without the background scene. (I am currently running the demo fine from github with a pretrained model and I would like to output only mask without the initial scene picture in the background ) Thanks you in advance ! :)

[@anishkuvelkar1876]

hey, does it work for u , could u provide me the link to your notebook if it does.

[@computer_vision]

please provide info about custom training

[@Roboflow]

We are working on it! 💪 Please have a little more patience.

[@coder5199]

@@Roboflow Hi is the video coming? I can't wait to use it!

[@snehitvaddi]

Can you please share the colab link shown in the video? (Code which includes object size estimation part)

[@Roboflow]

Oh... That part was written on the fly :/ And I'm really really sorry to say that but I don't have it any more

[@snehitvaddi]

@@Roboflow Okay, NP

[@anishkuvelkar1876]

hey, have u tried running it without the errors, could u provide me the notebook link?

[@snehitvaddi]

@@anishkuvelkar1876i guess I did but I too lost track of it.

[@anishkuvelkar1876]

@@snehitvaddican u check your drive if it happens to exist, i really need this tomorrow, if not thanks!

[@mikeazatov1537]

What's the latest on the custom training tutorial ? :)

[@Roboflow]

Good question… there was so many new excite omg models lately and I simply had no time to do that. I hope we will be able to revisit that idea.

[@TusharMSK]

How to perform inference on video ?

[@Roboflow]

Could you set up a new thread here: github.com/roboflow/notebooks/discussions/categories/q-a I’ll try to help you out.

[@anishkuvelkar1876]

Hey , the colab notebook of oneformer is not working.

[@anishkuvelkar1876]

can u provide me the link to your notebook for this code?