Multimodal RAG, UX For Multimodal Search, and AI NPCs in Immersive Games | Multimodal Weekly 23

​In the 23rd session of Multimodal Weekly, we have three exciting speakers with diverse backgrounds in open-source, search, and gaming.
​✅ Yi Ding, Head of Partnerships at LlamaIndex, will give an overview of GPT-4V and how it can be combined with Retrieval-Augmented Generation to enable a multimodal future.
- Connect with Yi: / dingy
- Read this blog post on performing Multimodal RAG with LlamaIndex: blog.llamainde...
​✅ Angela Hoover, Co-Founder and CEO of Andi Search, will share her learnings from taking a 'UX-first' approach to AI.
- Connect with Angela: / angela-hoover
- Check out Andi Search: andisearch.com/
​✅ Russell Sng, Developer Relations Manager at Inworld, will present the future of immersive games with AI non-player characters.
- Connect with Russell: / russellsng
- Build with Inworld Studio: studio.inworld...
Timestamps:
00:10 Introduction
03:58 Yi starts
04:36 What is GPT-4V?
07:05 Visual Retrieval-Augmented Generation
09:20 Generation-Augmented Retrieval-Augmented Generation (GARAG)
11:30 Retrieval-Augmented Generation-Augmented Retrieval-Augmented Generation (RAGARAG)
13:30 Yi answers audience questions
18:00 Angela starts
19:23 The co-founders of Andi Search
20:10 The journey of Andi since YC Startup School
21:55 A demo of Andi Search
25:20 Technical challenges building multimodal search
27:32 Angela answers audience questions
34:55 Russell starts
37:06 Why AI Non-Playable Characters?
42:00 The Inworld AI Platform for AI NPCs
45:28 Key Pillars of Inworld (Performance, Personality, and Awareness)
47:08 Inworld's Dev-First Approach
47:31 Inworld is building mods
48:45 Inworld's owl character (Wol) built with Niantic
49:46 Inworld is availabile on Unity and Unreal Engine
50:38 Inworld's use cases beyond gaming
54:29 Inworld is available on Steam
55:20 Russell answers audience questions
01:02:10 Conclusion
Join the Multimodal Minds community to receive an invite for future webinars: / discord