ViPlanner: Visual Semantic Imperative Learning for Local Navigation (ICRA 2024)

This video demonstrates how our fully-learned local planner can navigate complex environments by recognizing different terrains and their traversability.
Our paper introduces a novel planner design that combines depth and semantic information. It employs the imperative learning paradigm for optimizing the planner weights end-to-end based on the planning task objective. The optimization uses a differentiable formulation of a semantic costmap, enabling the planner to differentiate between different terrains and accurately identify obstacles.
Trained entirely in simulation, ViPlanner can be applied to real-world scenes in a zero-shot manner. Moreover, our experimental results demonstrate resistance to noise and a significant decrease in terms of traversability costs compared to purely geometric approaches.
For more information:
- Visit our Project Website at leggedrobotics...
- Read our Paper arxiv.org/abs/...
- Checkout our Code github.com/leg...

[@Charles-Darwin]

amazing work. I can't wait to see more.

[@milostean8615]

That’s awesome

[@dodeakim]

😍

[@marcosoliveira_accounts]

Cam someone explain how the path planning works for the real world? Is this some kind of pre trained path, and the robot are using probability to estimate the actual pose? Idk

[@leggedrobotics]

The pose is determined by the robot's state estimator. The predicted path is then followed using a PD-controller.