BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environment

We introduce BusyBoard, a toy-inspired robot learning environment that leverages a diverse set of articulated objects and inter-object functional relations to provide rich visual feedback for robot interactions. We also introduce a learning framework, BusyBot, which allows an agent to jointly acquire three fundamental capabilities (interaction, reasoning, and planning) in an integrated and self-supervised manner. With the rich sensory feedback provided by BusyBoard, BusyBot first learns a policy to efficiently interact with the environment; then with data collected using the policy, BusyBot reasons the inter-object functional relations through a causal discovery network; and finally by combining the learned interaction policy and relation reasoning skill, the agent is able to perform goal-conditioned manipulation tasks. We evaluate BusyBot in both simulated and real-world environments and validate its generalizability to unseen objects and relations.

[@zeyilucialiu820]

arXiv: arxiv.org/abs/2207.08192 Website: busybot.cs.columbia.edu Github: github.com/columbia-ai-robotics/busybot For questions, please contact Zeyi Liu lzylucy.github.io