How DINO learns to see the world - Paper Explained

[@akshaymundra1052]

Loved your series on self-supervised learning. Are you also planning to cover DINOv2? I am particularly curios about the emergence property of the model -- how it is able to regress semantically consistent features for different parts of the objects (and not simple FG-BG separation as in DINOv1)!

[@benmainbird]

Great video! Keep it up👍

[@borismeinardus]

Genuinely happy to hear you liked it, thanks! ☺️

[@江楓漁火-e5u]

Hi, I'm a bit confused about the centering method you described in this video(3:25). In your video, you're adding the center to the online network's output, which is different from what I've seen in other implementations of DINO (kzbin.info/www/bejne/nmTMm2Z8aMiDf80si=BUj7iQMXKaEs0Nr1&t=1296). Most implementations subtract the center from the output. Could you please clarify if there's an error in the video or if this is a different approach to centering?

[@yossefdiab7452]

great explaination

[@borismeinardus]

thank you ☺️

[@nasosgerontopoulos5267]

Very good content. Congrats 👍. Reading papers can be tough for many people, and such videos make it a lot easier to keep up with these state of the art advancements. As a fellow researcher, do you think investing time in self-supervised learning research is worth it right now? Considering that me and my team do not have access to such computational power as META and Google, I am not sure if we can keep up.

[@borismeinardus]

Hey, thanks! 😊 I think it is worth it! SSL is a broad field and SSL in the case of Multi-Modal Learning is very relevant. Yes, you will likely not be able to build the largest foundation models and go for scale, but you can definitely work on more nuanced research. E.g. Imagebind is a great example of a simple idea that does not require all the data and compute in the world. Btw. I also have a video on that paper :) kzbin.info/www/bejne/h4KtZHyIZcabg80si=VYxxIQPiyAXnlsw9

[@pankajmaheshwari6033]

In the training of dino I got same loss every time 10.09030 however I changed the Teacher Temperature hyperparameter below 0.06 which written in the paper.?can anyone suggest something beacuse I seen on the internet evryone have same problem with same exact loss value 10.09030 …please wrtite down any golbal solution!!!!

[@carsongutierrez7072]

Transformers~ ML bro~

[@borismeinardus]

👾

[@menkiguo7805]

it dose has the projection head though