Curiosity-Driven Learning of Joint Locomotion and Manipulation Tasks

Learning complex locomotion and manipulation tasks presents significant challenges, often requiring extensive engineering of, e.g., reward functions or curricula to provide meaningful feedback to the Reinforcement Learning (RL) algorithm. This paper proposes an intrinsically motivated RL approach to reduce task-specific engineering. The desired task is encoded in a single sparse reward, i.e., a reward of “+1” is given if the task is achieved. Intrinsic motivation enables learning by guiding exploration toward the sparse reward signal. Specifically, we adapt the idea of Random Network Distillation (RND) to the robotics domain to learn holistic motion control policies involving simultaneous locomotion and manipulation. We investigate opening doors as an exemplary task for robotic applications. A second task involving package manipulation from a table to a bin highlights the generalization capabilities of the presented approach. Finally, the resulting RL policies are executed in real-world experiments on a wheeled-legged robot in biped mode. We experienced no failure in our experiments, which con- sisted of opening push doors (over 15 times in a row) and manipulating packages (over 5 times in a row).
Paper: openreview.net...
Authors: Clemens Schwarke, Victor Klemm, Matthijs van der Boon, Marko Bjelonic, and Marco Hutter
Learn more: www.swiss-mile...
Disclaimer: Robot from ANYbotics; customized by ETH Zürich; strictly for research purposes.

[@TNERA]

I love the way it throws the boxes around, it obviously ready for package delivery service! :D

[@leggedrobotics]

That is what comes out when you train on human data. :D

[@stardude2006]

LOL !

[@jkyoft78]

So true...just like my regular delivery guy. Add in 'run back to the delivery truck' and the robot would be all set.

[@Bit-while_going]

He's like a jump-out boy. He doesn't drive, but just hefts packages when you tell 'im. Then, if he goes bad, he jumps out for the wrong reason and grabs the package and runs.

[@ProjectPhysX]

1:44 it passend the hot potato test!

[@leggedrobotics]

haha

[@Genadvideo]

He moves like he's alive

[@leggedrobotics]

Embodied AI at its best.

[@kudi82]

You guys made big steps!

[@leggedrobotics]

Thank you!

[@juliansc9018]

Bedarf geweckt, Bedarf gedeckt! Awesome job! Bigweld would be proud! :)

[@leggedrobotics]

Danke 🙏

[@RabidWoodchuckofDoom]

Bursting through that door like Kramer

[@leggedrobotics]

As if it’s about life and death 🫡

[@interestedinstuff1499]

It is pretty dramatic with packages. Love this machine. Fabulous channel.

[@leggedrobotics]

Glad you enjoy it!

[@kzalesak4]

It's super interesting - I am guessing that the task was specified as "put item in box" and as such, it found a very efficient way to do it. Had the task been "stack the box neatly", I assume it would find a more conservative way to put the box down

[@interestedinstuff1499]

@@kzalesak4 You are probably right. I think we'll get exactly what we ask for so it will be a good idea to make sure we ask for the right thing. Specificity will be king for a little while. Then the machines will get smarter.

[@3alabo]

NICE

[@leggedrobotics]

Thanks!

[@neithanm]

Been following you guys for years. The most underrated robot platform in the world. And they keep showing that crap from Tesla, arg...

[@leggedrobotics]

Thank you for this honor!

[@xspager]

Need some rewards for not scaring the living crap of people on the other side of the door.

[@leggedrobotics]

The industrial version will try not to destroy the door :)

[@herocraftturkiye9868]

Letsss gooooo

[@leggedrobotics]

lets roll!

[@logitech4873]

Wow this is fantastic!

[@leggedrobotics]

Thank you 🙏

[@rjung_ch]

👍💪✌

[@leggedrobotics]

thanks!

[@lancethrustworthy]

The guy lets the robot go through then pulls the robot back; go through, pull back. Eventually the robot figures it out and wounds the guy. I look forward to seeing how good these robots can get with a basketball.

[@leggedrobotics]

Basketball… challenge accepted

[@ziwenzhuang]

btw, are you still using isaacgym? or already transferred to IsaacSim?

[@eongan]

I love animal

[@leggedrobotics]

We love all robots

[@FunwithBlender]

really cool stuff guys would be cool if you touched on curiosity driven learning and the challenges you had with distractions or as I call it ADHD behaviour

[@divinusfilius3771]

The fact that it goes thru doors freaks me the hell out (esp if it had circular saws attached instead of front wheels - please make this upgrade). Robots aren't supposed to go thru doors! Haven't you played Generation Zero?!

[@leggedrobotics]

Haha will check it out now

[@huszaratraktor]

"Recently the robot learned how to stand up" So it begins...

[@leggedrobotics]

What begins?

[@yinyang2385]

​@@leggedroboticsI think he was reffering to a scene from The Planet of Apes, with standing up being the sign of it evolving

[@MicronOnline]

Holy shoot it's a GoBot

[@leggedrobotics]

GoBots are coming

[@divinusfilius3771]

Amazon will order 200,000. This bot knows how to handle his packages!!!

[@leggedrobotics]

Let’s hope they do!

[@PewterBird]

look I understand we're purely showcasing a technology here, but visually speaking, could this thing be any more terrifying? please don't answer that lol

[@leggedrobotics]

This is human perception:)

[@divinusfilius3771]

Attach saws for the front wheels and you've got yourself a terminator!!! Sweeettt!!! Gemini will kill us all.

[@leggedrobotics]

Robots need to be helpers! 🤘

[@eltodesukane]

Wheels make very bad hands.

[@leggedrobotics]

Depending on the task ;) but in general, you are right.

[@thinkingpro3905]

Excuse me, what robot simulation environment is used here?

[@jianweiliu6271]

Pretty sure it's Nvidia Isaac Gym

[@leggedrobotics]

Yes :)

[@ynnad7778]

Could this technology be extended to autonomous vehicles as well? I am imagining that we can also have scenarios where we can have an AV and we supply a curiosity state that relates to navigating some given environment, or managing control in a way that minimises the energy requirements from the battery. Is this a correct understanding? I am asking this since I am working on a presentation project related to locomotion modalities and curiosity-based navigation from your lab's paper

[@leggedrobotics]

Can be done. What is the presentation about?

[@ynnad7778]

@@leggedrobotics we're just discussing locomotion modalities and showcasing the curiosity mechanism, we're just explaining them and describing the potential use cases

[@sekkou_p]

ZABORGAR!

[@leggedrobotics]

We need to watch this movie.

[@KarlLew]

Put socks on the rear wheels and it will turn into Tom Cruise in Risky Business. 🎉😎

[@leggedrobotics]

I will check out the movie

[@transcendant3231]

Good, but expectation is nothing inside the boxes will fall out. The moment the boxes collapses and breaks all you have is an empty box or nothing at all. Equip it with a functional arm and hand. Maybe a kick stand for the wheels to keep it in place. Unfortunately, I wouldn't buy this if I had a company due to the amount of unpredictability and the way it handles boxes. This is great as a robot that delivers drinks and food while remaining on four wheels. It could work in an airport to handle lost luggage or delivery of luggages at an additional price. A carrier service or service for the disabled or seniors.

[@leggedrobotics]

All these points can be fixed over time but we have to start somewhere.

[@staticmin3]

recently the robot was "Programed how to stand up" 100% guarantee it didnt learn how to do it.

[@leggedrobotics]

What do you mean?

[@kzalesak4]

A task where standing up would be beneficial is sufficient

[@brianmccaig]

We're doomed😂

[@leggedrobotics]

Robot are coming … to help us

[@HOWYOUDOIN884]

It looks awesome but for some reason it seems evil to my brain.

[@leggedrobotics]

It’s a very nice robot and does exactly what you tell it. :)

[@yaosio]

You are too quiet I cant hear you.

[@leggedrobotics]

Haha

[@carl_aryee]

too spergyyy

[@leggedrobotics]

😂

[@TheAstronomyDude]

The loser at the channel @TwoMinutePapers stole your video and didn't even mention the source.

[@leggedrobotics]

You mentioned this on Reddit as well, right? :)

[@TheAstronomyDude]

@@leggedroboticsNope. I think a lot of people are fed up with that channel copy-pasting their work.

[@yutuniopati]

Now let's check for dog poop on this door....

[@leggedrobotics]

robots do not poo :D

[@yutuniopati]

@@leggedrobotics But they do drive on dog poo....