Discovering hidden connections in art with deep, interpretable visual analogies

Image retrieval systems allow individuals to find images that are semantically similar to a query image. This serves as the backbone of reverse image search engines and many product recommendation engines. Restricting an image retrieval system to particular subsets of images can yield new insights into relationships in the visual world.
In this webinar, Microsoft Research Development Engineer Mark Hamilton presents a novel method for specializing image retrieval systems called conditional image retrieval. When applied over large art datasets in particular, conditional image retrieval provides visual analogies that bring to light hidden connections among different artists, cultures, and media. This aims to encourage a new level of engagement with creative artifacts and inspire people to imagine new works of art. Hamilton will demonstrate how conditional image retrieval systems can efficiently find shared semantics between works of vastly different media and cultural origin. He’ll also demonstrate how this approach can improve generative adversarial networks (GANs), algorithms that create novel images from scratch, by identifying where GANs fail to model the true data distribution. Finally, Hamilton will introduce a generalization of Additive Shapley Values (SHAP) for better understanding why neural networks view images as similar.
Together, you’ll learn how to:
■ Build a reverse image search engine
■ Generalize these engines to handle extra requirements on the retrieved images, such as a filter for a particular artistic culture or time period
■ Find shared inspiration and semantics in the visual arts
■ Interpret and explain a reverse image search engine
■ Use conditional image retrieval to discover blind spots in GANs
𝗥𝗲𝘀𝗼𝘂𝗿𝗰𝗲 𝗹𝗶𝘀𝘁:
■ Conditional Image Retrieval (Publication): www.microsoft.com/en-us/resea...
■ MIT blog: www.csail.mit.edu/news/algori...
■ Interactive Website: www.aka.ms/mosaic
■ GitHub Code: www.aka.ms/visual-analogy
■ Mark Hamilton at Microsoft Research: www.microsoft.com/en-us/resea...
■ Cognitive Services Research (Group Page): www.microsoft.com/en-us/resea...
*This on-demand webinar features a previously recorded Q&A session and open captioning
This webinar originally aired on July 30, 2020
Explore more Microsoft Research webinars: aka.ms/msrwebinars

[@AZTECMAN]

Thank you for sharing this very fine information. Distinguishing bias in our algorithms serves the betterment of society, in my opinion.

[@OldEarthWisdom]

Wow, I got a better understanding that AI can and probably will be used to control people without them even knowing how or when it is being done. It makes Yuval Noah Harari's warning that we have to know ourselves better than Google us even more relevant. I feel afraid for the future.

[@AZTECMAN]

Did you watch the video?

[@AZTECMAN]

Are you thinking this is the kind of technology which compares pictures of people for identification...? Perhaps there is some cross pollination, but this is a separate branch of research.