Instance Segmentation in 12 minutes with YOLOv8 and Python

YOLOv8 is the latest installment of the highly influential YOLO (You Only Look Once) architecture. YOLOv8 was developed by Ultralytics, a team known for its work on YOLOv3 and YOLOv5.
Following the trend set by  YOLOv6 and YOLOv7, we have at our disposal object detection, but also instance segmentation, and image classification. The model itself is created in PyTorch and runs on both the CPU and GPU. As with YOLOv5, we also have a number of various exports such as TF.js or CoreML.
In this video, I'll take you through a step-by-step tutorial on Google Colab, and show you how to train your own YOLOv8 instance segmentation model.
Chapters:
0:00 Introduction
1:03 Setting up the Python environment
3:08 Inference with YOLOv8 model pre-trained on COCO dataset
5:42 Download custom dataset from Roboflow Universe
7:26 Train YOLOv8 model on custom dataset
9:16 YOLOv8 model evaluation
11:05 Custom YOLOv8 model inference
11:43 Conclusion
Resources:
🌏 Roboflow: roboflow.com
🌌 Roboflow Universe: universe.roboflow.com
📝 How to Train YOLOv8 Instance Segmentation on a Custom Dataset Blogpost: blog.roboflow.com/how-to-trai...
📓How to Train YOLOv8 Instance Segmentation on a Custom Dataset Notebook: colab.research.google.com/git...
⭐ YOLOv8 repository: github.com/ultralytics/ultral...
📄 YOLOv8 docs: v8docs.ultralytics.com
📓 Learn more about YOLOv8 and other Computer Vision models with Roboflow Notebooks: github.com/roboflow/notebooks
🆕 What's New in YOLOv8 Architecture: blog.roboflow.com/whats-new-i...
💯 RF100 Dataset blog.roboflow.com/roboflow-100
Stay up to date with the projects I'm working on at github.com/roboflow and github.com/SkalskiP! ⭐

[@Pingu_astrocat21]

Thank you for uplaoding this :). I have started researching on this and this is such a great intro!

[@ikramessafi9560]

Thank you, but i would like if there is a method that can help to count objects in images ?

[@7th.convict]

How do you integrate this into android project like for example a food object detection + instance segmentation from an image particularly working with an imageview component?

[@PrakashKeshari-ig6pk]

@SkalskiP i am stuck in a step ,so i need your help FileNotFoundError: Dataset '/content/ice-lolly-2/data.yaml' not found ⚠, missing paths ['/content/ice-lolly-2/ice-lolly-2/valid/images'] code and address are same but unable to get output -- %cd {HOME} !yolo task=segment mode=train model=yolov8m-seg.pt data={dataset.location}/data.yaml epochs=25 imgsz=640 address in my drive ---- # /content/ice-lolly-2/data.yaml---address is same please guide me

[@dserenini]

Open data.yaml and change from: test: ../test/images train: {datasetName}/train/images val: {datasetName}/valid/images to: test: ../test/images train: /content/{datasetName}/train/images val: /content/{datasetName}/valid/images

[@user-sg6tw4hx3o]

@@dserenini Thank you so much buddy love you.

[@abbaali-concern7457]

@@dserenini thank you so much, why didn't he specify that in the video?

[@FilippoGronchi]

Very cool land interesting video Piotr...awesome!

[@SkalskiP]

Thanks a lot! 🙏

[@pushpendrakushwaha604]

Hey! Thats really great, I have a question if I want to extract the segemented masks from the predictions is there any way?

[@tomaszbazelczuk4987]

really good staff🙂

[@SkalskiP]

Thanks a lot! Dzięki ;)

[@mehrshad9]

Thank you for the content! I really enjoyed it. Could you please show how to get a mask array of an specific segment (object).

[@SkalskiP]

Hi 👋🏻 It’s Peter from the video. I plan to revisit segmentation, as we plan to support it in Supervision. But I’m not 100% sure when that will happen. But I’ll keep your request in mind.

[@vikashkumar-cr7ee]

Dear Peter, Could you please share the link of the Coral Segmentation 4 dataset used in this tutorial?

[@user-si5ly5vb5j]

Hey, I wanted to know whether I can use the yolov8 model and list of objects which it can detect stored in a file, say objects.txt and use it for custom instance segmentation instead of fine-tuning the model on a custom dataset?

[@zaidahmed4069]

Any idea about this error of yolo v8? model.export(format='saved_model', imgsz=640) TensorFlow SavedModel: export failure ❌ 35.2s: SavedModel file does not exist at: train\weights\last_saved_model\{saved_model.pbtxt|saved_model.pb} same thing is happening with tflite..

[@ELOUNGOUAmine]

thanks for the video. How to display in the inference only the mask and confidence without bounding box?

[@amir-ui3vh]

i love you man you are greate

[@ganeshjoshi4426]

Good Work By you always. It would be more helpful if you can add also python code as this is only yolov8 cli commands.

[@SkalskiP]

Hi it is Peter from the video. So showing how to process YOLOv8 Instance Segmentation model output in Python?

[@ganeshjoshi4426]

@@SkalskiP yes.

[@SkalskiP]

@@ganeshjoshi4426 cool idea 💡 I’m not sure we will do it on Roboflow channel but something like that could for sure happen on my private channel or on Twitch

[@baoang6966]

Very helpful video. Thank you for your works. I am just wondering after testing and you will have a set of images contain bounding box and instance segmentation. Can we using those images to turn it into binary mask. From that we can compare those masks from the model versus our ground truth?

[@TlTAN]

is there anything i can do if the training stops in order to not to start again?

[@titusfx]

Tell us what is your name now?😅 Thanks for sharing!!! One thing that I don't like about Roboflow is has too many steps to try out something new. I know in a big project you need a good organisation, but sometimes you just need to try out stuff and then if it works as expected being organised.

[@SkalskiP]

hahaha, my name is a corporate secret :) That's a really cool feedback, one of my responsibilities is to make that process as smooth as possible. Any examples that you could share?

[@cappittall]

@@SkalskiP Hi Peter . Thanks for the tutorials :)

[@SkalskiP]

@@cappittall My pleasure!

[@td8131]

Thank you for the content. I love it

[@arpitapujapanda8415]

Can you make a video on person re identification.

[@aminmemar416]

i get this error. Sizes of tensors must match except in dimension 1. Expected size 134 but got size 0 for tensor number 1 in the list. i tried image sizes like 1072,1088,1024 but not fixed can i get some help? i resized images with roboflow

[@Roboflow]

I’m happy to help. Could you create an issue here github.com/roboflow/notebooks/issues Describe your problem, provide link to your dataset and your version of Google Colab?

[@aminmemar416]

@@Roboflow i've created the new issue

[@Roboflow]

@@aminmemar416 thanks a lot. Could you provide link to your dataset?

[@user-lt5yt8uz4z]

is it really instance segmentation since all instances of the same class are assigned the SAME COLOUR mask...same colour mask refers to semantic and not instance.

[@srisathvikapandu427]

hi did you get the answer , I also want to work on the instance segmentation model

[@DeepakKumar-uh2bx]

Hi, I want to get the segmented mask, but not the bounding box and the labeling that comes with it. How can I do this with Ultralytics Yolov8 segmentation model? Thank you.

[@naseef6420]

Are you able toe extract mask points??

[@baoang6966]

Hi, Did you do it? I have a same concerns as you.

[@alebachewmekuriaw2233]

That is great. Thank You.

[@gbo10001]

what is the use case of instance segmentation? when does it being used rather than object detection?

[@Roboflow]

Great question. You would pick segmentation over regular detection when precision is key. Most typical use case would be processing medical images or passing detections to robotic arm 🦾. Every millimeter counts.

[@user-lb8rs9uw1s]

Thank you for the content!! You saved me ;)

[@FatemehZaremehrjardi]

is it possible to use yolov8 seg model only for specific classes? (ex: only segment people in the image) Thank you for the video!

[@Roboflow]

Hi yes it is possible :) Just use classes argument. Take a look here: github.com/ultralytics/ultralytics/blob/main/ultralytics/yolo/cfg/default.yaml#L65

[@rajmeetsingh1625]

Thanks for sharing, I need ids number also. How i will get that.

[@kvnptl4400]

Hey, it is not instance segmentation, it's semantic segmentation. All the same class objects should have unique IDs. Please update.

[@TitoWPurnomo]

hi. after following the instance segmentation tutorial with yolov8, I found that the train mode output contained one confusion matrix. My question is, does the CM belong to the box or the mask? Thank You

[@adnanahmad4168]

Can you please explain on how to know the accuracy of my model i trained using yolov8? Also explain the precision recall curve, f1 score and confusion matrix. I need for my project immediately within 2 days and i can't find any help regarding that.

[@abangfikri1865]

Thank u for the video, but can this yolov8 segmentation be exported into tf lite? And if so how can it be tested or deployed on android?

[@Roboflow]

Did you try to call model.export(format="tflite") on the model that you trained? That should be the way to export TF Lite, at least according to the documentation.

[@body1024]

Amazing as always but can we upload this instance segmentation model to roboflow like with the object detection ?

[@SkalskiP]

Hi it is Peter from the video! Not yet... But we are working on it! Stay tuned. I plan to produce much more content around deployment functionalities.

[@dimitrijemitic497]

Great video. Is it possible to determine for two overlapping objects which one is behind the other one ?

[@MadeYourVideo]

Hello, do I need the object detection weights for the rectangles and the segmentation weights for the segment area inside the rectangles or the segmentation custom dataset is superior to the object detection data set and the rectangle area appears also? So do I have to label my custom dataset in future only on segmentation? Is the object detection dataset obsolete?

[@Roboflow]

Hi 👋🏻 If you do segmentation you get object detection for free.

[@MadeYourVideo]

​@@Roboflow ohhh :) yes that's sad because already finished my detection labeling.... and now i have to do the segmentation :) oh noooooo! ok.....

[@Roboflow]

@@MadeYourVideo oooh :/ I’m sorry to hear that.

[@MadeYourVideo]

@@Roboflow maybe the recognition algorithms work better than the segmentation algorithms ? is there a combination possibility ?

[@semperzero]

how can i determine which pixel belongs to which class? i tried looking in the labels logs but i'm not sure how to decode that into a map of pixel coordinates to identified class

[@Roboflow]

Could you create a thread here: github.com/roboflow/notebooks/discussions/categories/q-a I’ll try to help you out.

[@semperzero]

@@Roboflow i did. you should also include the response in the docs, as this would be a very useful information

[@Roboflow]

@@semperzero you men YOLOv8 docs? I’m only here to show you how to use the model. :) It’s not our. I guess you’d need to reach out to Ultralytics team :)

[@semperzero]

@@Roboflow Okay. Thought you guys were working together or something. Waiting for your response on github and thanks a lot.

[@lorenzoleongutierrez7927]

Great !

[@thanhphatpham9965]

Thank you for the content, but I have a question. Is it possible to combine model Yolo-Nas and the part of segmentation in Yolov8-seg to model YoloNas-seg ? Thank you.

[@Roboflow]

Unfortunately nope. But you can combo YOLO NAS and Segment Anything Model ;)

[@igorg4129]

Please clarify me something There is a way to train with the roboflow web interface (where there are some paid and free plans) which means that I actually train not using my local GPU On the other hand there is a way to train via roboflow notebook as presented in the video. What I do not understand is when I train via notebook, is it the same as in the web interface? Particularly: does it also train via roboflow GPU (and not mine) and I am being charged for this training? (I mean if my plan includes 10 trainigs, so this trainig will be made from this very pool?)

[@Roboflow]

When you train in the notebook on Google Colab, you are using Googles GPUs. Colab allows you to use NVIDIA T4 for free with some limitations.

[@igorg4129]

Please, do I understand right the following: 1) I have purchased paid roboflow account, (for example, the one that includes 10 trainings per month) Do I understand right that such trainigs I can use ONLY via roboflow web interface? Right? 2)If I train via notebook I can use both - my local GPU or GOOGLE colab GPU? Right? 3)If I train via web interface I see it is somehow limited in detailing results, for example, I trined yolo v8 instance segmentation, and could not find any way to get AP per class or confusion matrix. Do I understand right that such information will be only available while training via notebook?

[@mf4s]

Nice video, thanks for the content. Would it be possible to get the coordinates of the segmented instance and use them to calculate an area? (assuming I could estimate the length/height of the real object)

[@SkalskiP]

Hi 👋Yes. I'm actually thinking about making video about it. Would you like to see that?

[@mf4s]

@@SkalskiPhi! Yes, please :)

[@Anutosh13]

@@SkalskiP yes man plz whats stopping you

[@zy.r.4323]

Hi! I tried the notebook used in this tutorial on a custom dataset but whenever I start training getting run time error: RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 834 but got size 0 for tensor number 1 in the list. I did as you showed in the video.

[@Roboflow]

Could you create issue here: github.com/roboflow/notebooks/issues make sure to provide as much details as possible. I’ll try to help you.

[@filippoossuzio1346]

Have you find a solution? because I have a similar error

[@zy.r.4323]

@@filippoossuzio1346 u should train instance segmentation dataset, not object detection dataset

[@filippoossuzio1346]

@@zy.r.4323 It is a instance segmentation dataset.. I labelled using roboflow and from the start I set instance segmentation as a task

[@sowmiya1283]

@filippoossuzio1346 I have only object detection dataset in roboflow what can I do now should I start from scratch (annotating for instance segmentation in roboflow) or we can convert this dataset

[@mudigondaanirudh6456]

Is there any way to try and test our trained model on a video file?

[@Roboflow]

You should be able to pass that video file as a source in the CLI command.

[@PiyushTiwari-xp6qs]

I was trying the same and the problem now is - my results aren't getting saved in runs/segment/ Can you help me out with this ?

[@SkalskiP]

Hi it's Peter from the video. 👋Try adding `save=True` to the command ;)

[@PiyushTiwari-xp6qs]

@@SkalskiP Fantastic, worked like a charm. Thanks man

[@PiyushTiwari-xp6qs]

Also while using Yolov8 instance segmentation code - after the below line of code %cd {HOME} !yolo task=segment mode=train model=yolov8x-seg.pt data={dataset.location}/data.yaml epochs=100 imgsz=640 Getting error - the datasets and runs/segment/ are formed but are not detected FileNotFoundError

[@PiyushTiwari-xp6qs]

Also another question - How can we find and convert the bounding boxes co-ordinates from tensors to numpy Because I want to find the area covered by the masks or the bounding boxes

[@PiyushTiwari-xp6qs]

@SkalskiP

[@darshankumar2537]

Can we segment only person using pre trained model

[@Roboflow]

Yes, we can. You can filter detections by class using classes argument: github.com/ultralytics/ultralytics/blob/main/ultralytics/yolo/cfg/default.yaml#L65

[@darkjudic]

Hi, do you have a model supporting both: segmentation and pose?

[@Roboflow]

YOLOv7 and YOLOv8 both support pose estimation and instance segmentation.

[@darkjudic]

@@Roboflow I mean we have to use 2 models like yolov8l-pose and yolov8l-seg one after another, or we can somehow use/train a single model with single config file/data set? Thanks

[@Roboflow]

@@darkjudic ah so you would like to get pose and segmentation result from single model at once? Sorry... But no model comes to my mind.

[@sayedhasan5997]

how to perform classification on custom dataset in yolov8?

[@Roboflow]

We have example notebook for you to start: colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/train-yolov8-classification-on-custom-dataset.ipynb

[@sayedhasan5997]

@@Roboflow i see this notebook, but whenever i am trying to execute classification 'training' part, i found error in data={dataset.location}..i dont understand 'dataset.location' means what!! Bt in detection and segmentation 'training' part i use data=data.yaml and this is working fine.. Plz solve this issue, i will be grateful to u

[@Roboflow]

@@sayedhasan5997 you get that error with our notebook without any custom changes?

[@sayedhasan5997]

@@Roboflow i get that error in my notebook,,i follow your notebook code bt perform it in my own data.. In the 'custom training' option i dont understand "data=dataset.location" what does it mean in classification purpose!! In detection and segmentation 'custom training' i use data='my own yaml file' bt if i use this for classification 'custom training' then i found that error

[@Roboflow]

@@sayedhasan5997 I’ll try to take a look. But it would be awesome if you could create new issue here: github.com/roboflow/notebooks/issues

[@predator3088]

Hi, your api key was visible at 7.27 =/

[@Roboflow]

Great catch, thanks for noticing. As a note for others, when the API key is visible in a video or example that is shared with other people, click into your Workspace Settings -> "Roboflow API" -> "Revoke API Key" and finally -> "Generate New API Key"

[@predator3088]

@@Roboflow Hope not so many people noticed / used that)

[@SkalskiP]

Hi it's Peter from the video :) Thank you very much! I already regenerated the key, and blured the fragment where it was visible. That's what happens when you cat the video at 3-4 AM :)

[@krimafarjallah7553]

❤

[@muhammadabubakr4250]

This is semantic segmentation not instance segmentation