Applications of Generative AI with Jim Fan, Amjad Masad, and Michele Catasta

In this webinar, which includes Michele Catasta, Replit ML Advisor, Jim Fan from NVIDIA AI, and Amjad Masad, CEO and co-founder of Replit. The panel discusses the recent advancements in AI and the potential impact of multi-modality on the field.
Jim Fan is a research scientist at NVIDIA AI and has worked in AI for a decade, collaborating with several prominent AI researchers. He highlights the growth of AI from image recognition to large language models like GPT-4. Amjad shares his background in Dev tools and his excitement about applying machine learning and statistical approaches to code.
The panel discusses the recent NVIDIA GTC event, with Jim Fan describing NVIDIA's transition from a hardware provider to an enterprise-focused AI provider. He is mainly excited about the NVIDIA AI foundations, offering customization services that allow enterprises to create unique use cases with multimodal language models. This foundation model will also help incorporate images, videos, and 3D data into AI systems. Jim mentions NVIDIA's collaborations with Getty Images, Shutterstock, and Adobe for copyright-free generative art, as well as a large-scale AlphaFold API for drug discovery and biology research.
Michele highlights the importance of multi-modality and how it grants superpowers in communications with computers; not only humans will be able to communicate with AI using text but also using images, videos, and other sensory signals. Jim envisions a range of possibilities with multi-modal language models, including being able to interact with more natural human input, enhancing note-taking, and automating home decoration plans.
The panel sees an exciting future for AI due to multi-modal language models, which will continue to expand and offer endless possibilities. By incorporating various sensory signals in addition to text, AI systems will become more powerful and versatile, leading to groundbreaking innovations in a variety of industries.

[@jessamartin]

Really great summary of the current state of the art while also pointing backwards to historical papers and defining terms.

[@hiteshnaval3183]

Informative discussion Thank you for conducting this webinar

[@JoeRepka]

Thanks for an important view into where things are and might go. Of particular interest to me was the discussion around constitutional AI implemented as a separate model that 'teaches' or otherwise influences the pretrained megamodel. I tend to see the monolithic LLMs as a Swiss army knife that has just a single complex blade that tried to do everything, and that's the primary source of all their limitations and faults. I believe that all the tools that will come out between now and AGI will evolve as organic systems of models, with each smaller and designed as optimum for the tasks they contribute to the system. Adding the 'self attention' function was a micro-step in that direction (but look at the result!). The smaller 'teacher' models' or constraining models that modulate output from the LLM (pre-frontal cortex?) are larger steps, and should set the path to the organic module system (real brain model based on gross biological architecture). Robots will need a cerebellum & motor cortex, etc.

[@real23lions]

Audio is not clear. You should clean the audio using AI. I think Runway has that feature.