CIIR Talk Series - 9/16/2022: Mostafa Dehghani - Universal Models for Language and Beyond

Speaker: Mostafa Dehghani, Research Scientist at Google Brain
Website: mostafadehghan...
Faculty Host: Hamed Zamani, Assistant Professor at UMass Amherst
Talk Title: Universal Models for Language and Beyond
Abstract:
This talk makes the case for universal models in language as well as potential ways to go beyond text modality - an emergent and compelling research direction. Universal models are models that dispense with task specific niches altogether. I discuss the recent emerging trend of unification with respect to model, task, and input representation and how these research areas come together to push forward the vision of truly universal language models. Specifically, I discuss several efforts that we are working on such as Unifying Language Learning Paradigms (UL2 model), Unifying Retrieval and Search with NLP (Differentiable Search Indexes) and Multimodal Unifications (ViT, PolyViT, etc.), and finally how scaling up is a safe bet toward such a goal.
Bio:
Mostafa Dehghani is a research scientist at Google Brain. He has been working on scaling neural networks for language, vision, and robotics. Besides large scale models, he works on improving the allocation of compute in neural networks, in particular Transformers, via adaptive and conditional computation. Mostafa obtained his Ph.D. from the University of Amsterdam where he worked on training neural networks with imperfect supervision.

[@billykotsos4642]

Must watch