Roboflow 100 Benchmarking Tutorial with Google Colab and Docker

[@deanwebb9929]

Very informative. I would be curious to see the COCO pretrained weight included in the benchmark to add a signal on how easily the architecture is to fine tune on the benchmark. For example, if the mAP barely budges or God forbid it decreases.

[@Roboflow]

Thank you! I'm not sure if I understand you correctly, but if so than we actually included COCO baseline benchmark. Take a look at 11:34 at the very top, there are COCO baseline values.

[@DeanWebbDeveloper]

@@Roboflow I’m curious to see the baseline zero-shot across the Benchmark 100 instead of only COCO that it was trained on. To get an idea of its zero-shot mAP in addition to the fine tuned results. Some architectures are harder to transfer learn than others.

[@DeanWebbDeveloper]

I understand this can’t happen across the entire benchmark but wherever there’s overlap between classes in COCO and classes in the Benchmark 100

[@Roboflow]

@@DeanWebbDeveloper If you are curious how GLIT performed on RF100, that data is not included in video unfortunately, but you can find that information in paper - arxiv.org/abs/2211.13523 A Appendix. As for the overlap there is minimal overlap. For example this dataset universe.roboflow.com/roboflow-100/construction-safety-gsnvb contain person class. We can come up with precise information if you are really curious about it :)

[@DeanWebbDeveloper]

@@Roboflow Thanks for the info. I can probably run some tests to investigate further based on this. Thanks