[Own work] VALSE 💃: Benchmark for Vision and Language Models Centered on Linguistic Phenomena

[@booksandshowers3997]

Congrats🎉

[@AICoffeeBreak]

Thanks! 🐻

[@willsmithorg]

Nice idea and benchmark, well presented.

[@ivoschaper]

Great work! 🙌

[@AICoffeeBreak]

Thanks Ivo! 😊

[@DerPylz]

It's always nice when you show some of your own work! Very interesting, no reason at all to feel bad in comparison to the huge papers from the big companies!

[@soumyasarkar4100]

very interesting work, thanks for the video

[@hannesstark5024]

Nice!

[@gergerger53]

Very nice! I'll definitely stop by your poster at ACL. So that will be (...checks programme...) 23rd @ 2pm? Excited to meet Miss Coffee Bean 🤣

[@Diego0wnz]

Very interesting to test what moderm highly complex models actually learn, instead of just being amazed by it’s “better than previous models” This could lead new researchers to come up with different architecture/preprocessing solutions instead of just increasing model size and the amount of train data!

[@manoszaranis397]

Really impressive! Keep up the good work!

[@AICoffeeBreak]

Thanks! ☺️

[@HoriaCristescu]

Your findings correlate with the issues found in the "A very preliminary analysis of DALL-E 2" paper (Gary Marcus et al), but you went the extra mile and made a benchmark instead of just trying the model 10 times.

[@AICoffeeBreak]

And here we see how slow the publication cycle is in ML. DALL-E 2 wasn't even out there when we submitted the camera ready of the paper. 😅

[@DerPylz]

"slow publication cycle" - * sad biologist noises *

[@psebsellars]

Asian Champions League accepted this :o

[@AICoffeeBreak]

Lol, removed the hashtag. Wasn't aware of what ACL means on KZbin. 😅

[@giantbee9763]

We need more benchmarks :D !!! Though it is important to have standard benchmarks in one place, and ideally be able to be ran with one command and select the benchmark needed. Do anyone know of any project/repository which does this?

[@sebastiane7735]

I enjoy the vindictiveness of this paper. Shut up TPUs. Marry me?

[@jonathansum9084]

Listen, it is not really meaningful too see explanation without "how to implementing"! If it is your own work, I hope you will explain how to implement it. Thus, everyone in the world can use your work, with citing under your name, to build all kind of application. Rather than hearing someone says it is great.