Generally Capable Agents in Open-Ended Worlds, Jim Fan, NVIDIA Lead of Embodied AI | NVIDIA GTC 2024

Hear from Jim Fan, Senior Research Manager & Lead of Embodied AI at NVIDIA Research, as he explores a future where everything that moves will eventually be autonomous. He outlines a blueprint for the Foundation Agent, a single model that generalizes across diverse tasks, embodiments, and realities.
ChatGPT unifies all kinds of natural language understanding tasks in a single interface: text in, text out.
What is the equivalent for an AI agent? What does it take to build a model that actively explores the world, ingests multimodal sensory stream, plans over long horizons, acquires new skills, and bootstraps its own capabilities in a self-improving loop?
Speaker: Jim Fan, Senior Research Manager & Lead of Embodied AI, NVIDIA
Explore more GTC 2024 sessions like this on-demand: nvda.ws/3U33qo7
See the latest from #NVIDIAResearch: www.nvidia.com/en-us/research/
Read and subscribe to the NVIDIA Technical Blog: nvda.ws/3XHae9F
#GTC24 #NVIDIA #GTC #AI #GenAI #Generative AI #AIAgents #RL #ReinforcementLearning #JimFan #NVIDIAResearch

[@InTranceofficial]

the volume is so low that when I got a notification I fell of my chair

[@tananta29]

Please fix the low volume on some of these videos.

[@positivearrow]

It's something AI could easily take care of...

[@tananta29]

@@positivearrow haha someone should create an autonomous agent to check for low audio videos on KZbin and fix them.

[@Copa20777]

Thanks Nvidia for bring up Dr Jim Fan to talk about the Foundation agent in a comprehensive manner. Hope Dr Fan implements a usable foundation agent that would work on modern day computer systems, that can streamline across multiple apps, even play games more complex than minecraft. The future is truly exciting

[@SamantShubham]

people will have no idea where this is going, imo greatly impactful foundation work for generalist agents 🙌

[@brycebrousseau7921]

Jim is naturally hilarious.

[@nosferatuboi]

Jim Fan is the GOAT.

[@kayrealist9793]

Looks like we will have some amazing games coming soon.

[@AdrianRodriguezAIR]

They want us to read but they deliver a video... wth is going on with the guy/lady uploading all this videos? The volume is so low.

[@pengouin]

Again @NVIDIA Developer, you updaloded a video with the volume wayyyyy too low as many other said before, you have this issue with many of your videos. I have to put the maximum and having my volume booster extention at 600% and even then , i can only hear him if the fan of my laptop not spin fast.

[@Rick-rl9qq]

Although robotics still have a long way to go, the acceleration speed of technology is only increasing. I wouldn't be surprised if they actually did solve robotics by next year already. Ray Kurzweil may have actually predicted correctly after all. The future is only going to become more and more interesting. I can't wait to have an AI gaming pal

[@erpduocg2589]

Good

[@hussienalsafi1149]

☺️☺️☺️☺️

[@iAkashPaul]

The audio levels on all of these GTC videos is abysmally low

[@ApexJnr]

So a voice controlled minecraft army like an rts.

[@r34ct4]

WHAAT?

[@ParalityXR]

Game bots are not too far, an April Fools Nvidia once played might turn out true after all XD

[@ParalityXR]

Game Bots are not too far, an April Fools Nvidia once played might turn out to be true after all XD