YOLACT++ Instance Segmentation (Google Colab Tutorial)

Learn the basics of YOLACT++ and try it out in a free Google Colab notebook.
Tutorial: www.immersivelimit.com/tutori...
Research paper: arxiv.org/abs/1912.06218
(Big thanks to the authors: Daniel Bolya, Chong Zhou, Fanyi Xiao, Yong Jae Lee!)
Daniel Bolya's channel: / @danielbolya91
+ Follow These For Updates +
Immersive Limit Facebook: / immersivelimit
Immersive Limit Twitter: / immersivelimit
Subscribe to the KZbin channel too!
+ Connect with Us! +
Adam's Twitter: / aktwelve
Adam's LinkedIn: / arkelly12
Kayla's Twitter: / kayladkelly
Kayla's LinkedIn: / kayladk
+ More Ways to Connect +
www.immersivelimit.com/connect

[@simantikachoudhury9148]

Please make a tutorial video on training a custom dataset using yolact.. and also how to annotate the custom dataset.. it will be very helpful. Thanks in advance.

[@diythatworks]

GREAT WORK, Please make more of these. Thanks!

[@filipequincas1485]

Excelent tutorial! I am a PhD student on computer vision and just found your channel with this video. I am really impressed. Subscribed

[@ImmersiveLimit]

Thanks!

[@drociel8020]

This was very helpful. Thank you.

[@snippletrap]

Thanks a lot for this tutorial. I would definitely watch the successor video (ie using your own dataset) and I am definitely going to check out your other videos and services.

[@ImmersiveLimit]

Thank YOU!

[@lorenzoleongutierrez7927]

Great tutorial ! Looking forwards for training on new datasets ... perhaps in weeds ? 😉👍

[@leenatrivedi3]

It is very useful video.Thank you

[@woolfel]

YOLACT is an improvement to YOLO. It adds masks instead of just bounding boxes.At the top of the paper it quotes Joseph Redmon author of YOLO. If I understand the paper correctly the main improvement is mask proposal branch.

[@ArunMPEdison]

Thank you for sharing, very useful.

[@ImmersiveLimit]

Glad it was helpful!

[@rajkumarmukherjee5705]

Please make a video on how to train on costume images dataset by using yolact. How to create the dataset, train it and check the output by tacking images from webcam.

[@Fireway12]

Not working... # Downgrade torch to accommodate DCNv2 !pip install torchvision==0.5.0 !pip install torch==1.4.0 >> ERROR: Could not find a version that satisfies the requirement torchvision==0.5.0 (from versions: 0.1.6, 0.1.7, 0.1.8, 0.1.9, 0.2.0, 0.2.1, 0.2.2, 0.2.2.post2, 0.2.2.post3, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2) ERROR: No matching distribution found for torchvision==0.5.0 ERROR: Could not find a version that satisfies the requirement torch==1.4.0 (from versions: 1.11.0, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 2.0.0, 2.0.1) ERROR: No matching distribution found for torch==1.4.0

[@ajaykushwah6793]

can yolact be used with raspberry pi 4b with 4gb of ram?

[@iyshwaryakannan6677]

Sir, can the input image be any arbitrary size?

[@yogeshwarshendye4857]

Lot of things have changed recently!! Can't use that notebook! Could you please update?

[@lanschl]

great tutorial, would be really nice if you could make a tutorial how to create your on dataset. Do you know of any easy to use image sequence labeler for instance segmentation ?

[@ImmersiveLimit]

You may be interested in my most popular tutorial of all time where I teach how to make COCO datasets. 😀 www.immersivelimit.com/tutorials/create-coco-annotations-from-scratch

[@XX-vu5jo]

I tend not to just investigate on the model detecting all bananas as a single object. The dataset/weights should also be observed for that matter as it could have been taught to detect banana/s as a single instance.

[@kulturlumantar1634]

Thank you for sharing. Is there any way to get only masks of detected images?

[@iyshwaryakannan6677]

try using only fcn

[@wafabaccouch2693]

thank you for sharing, clear and well explained, please could you tell me how i can put my own video in place of yours i imported it on google drive but it did not accept the link of the video

[@ImmersiveLimit]

I had to experiment until I figured out the right link to use. It could be a permissions issue too.

[@ArunMPEdison]

Hi, Thanks again for this colab and this may be one popular one got many to google colab and its premium version. I had two questions being a beginner. 1. What aspect controls the sharp perfect edges of a segmented ROI? Is it the quality of annotation with more points and more images or along with that, the training model used to build the pre-trained weights, like epochs >300 to get more angles/vectors? 2. How a new custom model can be trained, which can include the existing coco object + the custom object? e.g. cascading/appending R-CNN training or similar should need to be employed? I was trying to leverage PIxelLib for the same.

[@ImmersiveLimit]

I actually don’t know the answer to either question, haha. I would suggest reading the research paper because that’s what I would have to do to answer.

[@ArunMPEdison]

@@ImmersiveLimit Thank you for the response.

[@payamb9601]

can this be used to segment parts of image , like segmenting street in aerial images?

[@ImmersiveLimit]

This is designed for instance segmentation. What you want is called semantic segmentation. I haven’t looked for a good semantic segmentation algorithm, but I’m sure they exist.

[@arigato39000]

thank you

[@serhatiscan]

The tutorial and the Colab notebook was really helpful. I have a question, though. While running inference, I occasionally observe the attribute error (" 'list' object has no attribute 'argsort' ") just as you have shown in the video. Is this an ongoing bug or something is wrong? There is no active issue related to this on the Github page of Yolact.

[@anirbannath7595]

Have you found a solution for the same?

[@ImmersiveLimit]

No idea! It seems to have worked, so while it's probably a bug with YOLACT, it may be harmless. 🤷‍♂️

[@anirbannath7595]

@@ImmersiveLimit Hi. I run the notebook a lot of times and I encountered this error every single time. The inference stops as soon as I get this error. I am unable to get an output.

[@acatch22]

@@anirbannath7595 line 155 of eval.py idx = t[1].argsort(-----) t[1] for some reason is a python list, while it is expected to be a numpy array so you can fix this issue by replacing t[1].argsort(---) with np.asarray(t[1]).argsort(---)

[@gayrattangriberganov8802]

Good job!!! Have you ever tried it for custom dataset? or Do you know any sources like github or sth like that?

[@ImmersiveLimit]

Check out our website, there are a few tutorials that should help

[@tatianeolivatto9156]

Please, make a tutorial video on training a custom dataset in Google Colab. I saw in another publication of yours that "Unfortunately you won’t be able to train on Google Colab. Colab times out and resets if you leave it training too long.". However, I found a way to "hack" this, by hold (CTRL + SHIFT + i) in Google Colab, at the same time to open up the inspector view on your browser. Paste the following code into your Console window and hit Enter: function ClickConnect(){ console.log("Working"); document.querySelector("colab-toolbar-button#connect").click() } setInterval(ClickConnect,60000) It did work when I was training a YoloV3 detection (I was able to run training for 3 hours).

[@ImmersiveLimit]

Wow, interesting trick! I'll have to try that when I get a chance.

[@prateekgupta3932]

How do you decide the value of max_iter with respect to the number of images in the custom dataset?

[@ImmersiveLimit]

Honestly I just experiment with a few numbers and see what works best. You can choose a high number and if it seems to have stopped making progress then you can stop early and cap it there next time you train.

[@ACloudGuruji]

Can you pls make a tutorial for custom dataset with yolact++

[@ImmersiveLimit]

We’ve got a lot of ideas for what’s next. We’ll add it to the list!

[@dursundurmaz915]

Thank you very much! All process worked and explanation is very clear. I have a question: How can i reach Object bounding box coordinates? Thank you again have a nice day!

[@ImmersiveLimit]

According to the script (github.com/dbolya/yolact/blob/master/eval.py) you can use the parameter "--output_coco_json" to output the results to a json file. Haven't tested it, but that should work.

[@dursundurmaz915]

@@ImmersiveLimit I will try and inform you, thx a lot

[@dursundurmaz915]

@@ImmersiveLimit Where can i find this json file? I couldn't find it

[@dursundurmaz915]

@@ImmersiveLimit You can use this parameter just for coco dataset, not for your own dataset.

[@ImmersiveLimit]

Could you convert your dataset to COCO format or convert the result to match your format?

[@guillermoesquivel6134]

Thank you for this tutorial. I have used it before, but now I try to run it again and I've got an error when executing DCNv2: 2 errors detected in the compilation of "/tmp/tmpxft_000000e2_00000000-6_dcn_v2_cuda.cpp1.ii". error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1 How can I fix it? Also, how can I modify the segmentation style, I mean, change colors and features of labels? Thank you in advanced for your reply.

[@ShubhamPatil-uz9vb]

i m facing the same issue. plz help with the solution is it related to incompatibility of versions cuda,gcc versions?

[@ShubhamPatil-uz9vb]

it would be a great help if u help fixing this @immersive limit . help from anyone who could fix it would be appreciated

[@ImmersiveLimit]

I’ll try to take a look at this soon.

[@adithiajovandy8572]

@@ImmersiveLimit same please help about that sir

[@maxgordienko9979]

@@ShubhamPatil-uz9vb got same error, changing yolact-master\external\DCNv2 to this github.com/CharlesShang/DCNv2 and it builds

[@iyshwaryakannan6677]

Hi sir, great tutorial, I tried running the same way as you did, but I'm getting this error: _pickle.UnpicklingError: invalid load key, '

[@anonymousorg3560]

thanks a lot for this tutoria,I have same problème in DCNv2 can you help me??

[@ImmersiveLimit]

Sorry, I haven’t been working on that

[@guillermoesquivel6134]

Same problem. Could you solve it?

[@cristiandavidparramunoz6101]

Great work!, do you know if I can use Yolact for detect and counting?

[@ImmersiveLimit]

Counting may be difficult if you have overlapping objects in an image

[@cristiandavidparramunoz6101]

@@ImmersiveLimit Oh, I see. In my thesis, I need to implement object detection in real time and counting, but I don't know which one to use, YOLO or YOLACT? Could you give me an advice ? I'm beginner :(

[@ImmersiveLimit]

Honestly not sure, haven’t had to solve that problem before

[@boualemnouari8446]

Thanks a lot for this tutoria,I have same problème in DCNv2 can you help me?? this is message errer "/usr/local/lib/python3.6/dist-packages/torch/utils/cpp_extension.py:304: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend. warnings.warn(msg.format('we could not find ninja.'))"

[@anonymousorg3560]

same problem

[@guillermoesquivel6134]

Same problem. Could you fix it?

[@criscanto7040]

anyone else getting corrupted output video when using google drive videos?

[@sebastiancarmonagomez4833]

yes, me

[@criscanto7040]

@@sebastiancarmonagomez4833 by now there are better implementations

[@sebastiancarmonagomez4833]

@@criscanto7040 where can i find these implementations?

[@snippletrap]

Huh, I always heard it as "CO-lab", like co-worker and lab..oratory. "cuh-LAB", emphasis on 2d syllable, as said in collaborate, sounds a little funny to me

[@ImmersiveLimit]

LOL... poh-tay-toe, poh-tah-toe? 🤷🏼‍♂️

[@ngsingwong3083]

"Run Inference on Images" File "/usr/local/lib/python3.8/dist-packages/torch/serialization.py", line 692, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '

[@deepakpandey3346]

you found any solution? for that