Backstage with BD-X Star Wars Droids and Disney Imagineering!

Disney Imagineering is sharing their BD-X droids at NVIDIA's GTC 2024. Here's a backstage look at how they get ready to perform and enchant! No edits here, just full joy.
Technically, 2 NVIDIA Jetson Orin NXs control the robot along with custom electronics. 11 brushless motors and 5 servos drive the legs and head. NVIDIA ISAAC Sim uses reinforcement learning to get the droids to walk. Generative AI in action! The magic part is that Disney animators have a special layer of control to bring the character to life.
It's pretty cool when a couple of them are running and appear to 'communicate'. It's a very convincing illusion, even when the puppeteers are standing beside you. All your attention is on the droids, which is why the magic trick works.
Should we build one on the channel?
00:06 Blue Droid start up
02:43 Green Droid start up
02:57 Blue & Green Interactions
Join this channel to get access to perks:
/ @jetsonhacks
As an Amazon Associate I earn from qualifying purchases.
Visit the JetsonHacks storefront on Amazon: www.amazon.com...
Visit the website at jetsonhacks.com
Sign up for the newsletter! newsletter.jet...
Github accounts: github.com/jet...
github.com/jet...
Twitter: / jetsonhacks
Some of these links here are affiliate links. As an Amazon Associate I earn from qualifying purchases at no extra cost to you.

[@JeffLarkin]

Oh, man, I was looking for these on the show floor and never found them. So cool!

[@JetsonHacks]

They're such a great illusion when you see them in real life. Thanks for watching!

[@Robothut]

Amazing how stable they are. Thank you for sharing this video with us.

[@JetsonHacks]

They put on a great show! You're welcome, and thanks for watching!

[@FHWalker007]

Yes please, lets build one!

[@JetsonHacks]

Thanks for the encouragement, and thanks for watching!

[@HybridRobotix]

@@JetsonHacksYour videos are always useful. I just wish I could afford one of the new Jetson Orin boards. I need serious processing for one of my robots.

[@HybridRobotix]

Yes, you should build one!!

[@JetsonHacks]

Thank you for the words of encouragement, and thanks for watching!

[@DeBord123]

Yes! Make one on the channel please!

[@JetsonHacks]

Thank you for the encouragement, and thanks for watching!

[@TheLunarSurface]

this is what dreams are fucking made of right here boys.

[@JetsonHacks]

They are certainly an impressive display of what engineering and creativity can produce! Thanks for watching!

[@77Avadon77]

Sooo coool.

[@JetsonHacks]

Thanks for watching!

[@huseman21]

Now cut the remote and add AI.

[@JetsonHacks]

There's a bunch of AI in there now. The locomotion models were trained in a simulator and deployed on the robot. Since this is a research project, the next logical steps would seem to be what you suggest. Autonomy and goal seeking. I didn't notice any obvious cameras or lidars, so bringing in a set of sensors seems to be part of the project to be explored. Thanks for watching!

[@withvinayak]

So cute lol

[@JetsonHacks]

They have cuteness turned to 11! Thanks for watching!

[@parttimehuman]

I was under the impression that they were autonomous AIs. =/

[@JetsonHacks]

They are not being used that way currently. There appears to be a RGBD camera on the body, and the ones in the Disneyland park appear to have extra places in the head where cameras could peak through. However, Disney Imagineering gave a detailed talk at GTC (which you can watch on demand at www.nvidia.com/gtc/ ) about how the robots operate. There's still lots of AI involved, they used reinforcement learning to build a model for the gait. They model the robot and simulate it in Isaac SIM. It takes about 48 hours to train the gait on a RTX4090. The model is then transferred to the robot. There are two Jetson Orin NXs in the robot, one in the body, one in the head. On top of the gait, Disney animators have trained in animations which you see here. Remembering that this is from a research group and this is a relatively new robot, it's interesting to think what they may be exploring next. Autonomy seems like an interesting path. Thanks for watching!

[@1Darknews11]

They are controled?

[@JetsonHacks]

Yes. They use machine learning models running on the Jetson to control their gait and animation kinematics. An operator directs the robot where to go and can trigger animations. There is additional hardware that could be used for autonomy in future development. Thanks for watching!

[@1Darknews11]

@@JetsonHacks Thanks!

[@Markste-in]

they r really cute

[@JetsonHacks]

People go nuts over them! Thanks for watching!

[@sohanrai3850]

Tutorial please

[@JetsonHacks]

That's an interesting idea. Thanks for watching!

[@user-uz1xr4jc5d]

ow~~ how much is it?

[@JetsonHacks]

I don't think they are for sale. They are research projects for Disney theme parks. Here's some video of them at Disneyland in Anaheim, California: kzbin.info/www/bejne/qYKVep6ufL57o7Msi=EO2oOtpv0-l_5kS6 Thanks for watching!

[@user-uz1xr4jc5d]

@@JetsonHacks i'm your fan.

[@JetsonHacks]

@@user-uz1xr4jc5d Thank you, I appreciate it!

[@stuntcar]

Did WD make this? I thought it was all cgi?

[@JetsonHacks]

It is real life. You can see where the paint has worn off on the head from people petting them. It certainly is Disney magic! Thanks for watching!

[@denirodarkqwerty]

lol the movie was cgi. these are either going to a park or made as promotional material for jetson

[@JetsonHacks]

They've been to Disneyland, and worked there for a day or two. This is part of Disney Imagineering research, which is always working on new approaches to bring unique character experiences to guests. I think it was more coincidence than tie in with NVIDIA marketing. Some of the work was done in Glendale, CA in conjunction with ETH Zurich. Thanks for watching!