DETR - End to end object detection with transformers (ECCV2020)

This is the talk associated with the ECCV 2020 oral paper "End to end object detection using transformer" by Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, Sergey Zagoruyko.
Github: github.com/fac...
Paper: arxiv.org/abs/...
Blog: / end-to-end-object-dete...

[@kvnptl4400]

A very nice presentation with clear visualizations and easy-to-understand explanations! Great Work!!🌟🌟🌟🌟🌟 Smooth animations 👌

[@syedabdul8509]

Excellent Explanation. But I want to know the most important thing in this video, How did you create those cool animations like @1:58-@2:20 and @8:00-@8:05

[@praveen9083]

I'm expecting this answer too!

[@nicollenunes4459]

@@praveen9083 me 2!

[@QuintinMassey]

Outstanding work. I’m also very interested in the, arguably more difficult, small object detection problem.

[@MarioHari]

Nice work! A small correction to what you said: "Semantic segmentation labels each pixel in the whole image. It is not restricted to only pixels in the background".

[@nicolascarion3111]

You're right, my statement is imprecise. I meant that semantic annotations of foreground classes are not used in the panoptic task.

[@MarioHari]

@@nicolascarion3111 merci infiniment :)

[@ujjalkrdutta7854]

@@nicolascarion3111 Can we then say that: "Panoptic Segmentation= Instance Segmentation+Semantic Segmentation minus annotations of foreground classes" ?

[@Ramakrishnan-bq9is]

Thanks for sharing! Could you please explain what you mean by full differentiable and how other methods might not be fully differentiable?

[@goldenshale]

This is an end to end neural network defined by functions which all have derivatives. In the R-CNN family of algorithms you have one procedure that produces a bunch of region proposals, then you crop out these regions and feed them to a classifier, and then you run another algorithm to prune out overlapping and low confidence predictions. Since there are multiple steps that have logical rather than mathematical implementations, you can't take derivatives all the way through to back propagate information through the whole system.

[@rohinim7707]

Amazing! What was the main motivation behind using a sequence model for an object detection?

[@redjammie8342]

It is not a sequence model. It was successfully used for sequences, but it's not a sequence model by definition.

[@chandrahasp6697]

Really good work!

[@ujjalkrdutta7854]

Elegant explanation. liked it

[@Nino234mff]

Thank you for the great work and the presentation!

[@kaceangelo132]

i realize it is quite off topic but do anyone know of a good website to watch new movies online ?

[@bakercain265]

@Kace Angelo try Flixzone. Just google for it =)

[@ZobeirRaisi]

What this mean?: "since the transformer is a permutation equivalent some extra care is required to retain the 2d structure of the image."

[@nicolascarion3111]

The transformer isn't aware of the 2D structure of the image, because 1) we flatten it and 2) permuting the inputs of a transformer simply permutes its outputs (permutation equivariance). That's why we add 2D positional encodings. This is similar to what is done in NLP, to retain the order of the sentence.

[@ZobeirRaisi]

@@nicolascarion3111 Thanks for your explanation. I have another question: Right now DETR because of rectangle bboxes of COCO-dataset produces rectangle-bboxes outputs, if we had polygon bboxes (8 points), which parts of the architecture must be modified to output a polygon shape bboxes?

[@nicolascarion3111]

@@ZobeirRaisi Well you need to modify the regression head as well as the loss and matching function (GiOU may not make sense anymore, so you'll likely have to stick to L1). For this kind of questions, it's best to open an issue on our github. Thanks!