Type 2 smoke detector disassembly with VLM-based action prediction

The video demonstrates how a vision language model (VLM) selects the appropriate disassembly operations. The workflow is exemplified on different devices, here we show a smoke detector disassembly.
The experimental setup for the implemented adaptive workflow consists of the following modules:
• Workpiece Input Module, where the smoke detectors and heat-cost allocators (HCA) are supplied.
• Robot Module with the Franka Emika Panda robot and a tool changer, enabling attachment of multiple end-effectors (a 3-jaw gripper and a vacuum gripper are utilized in the demonstration).
• Robot Module with the Franka Panda robot and the qb Variable Stiffness Gripper (VSG).
• CNC Module with a computer-numerical control (CNC) router for the removal of smoke detector housings.
• Vise Module with a pneumatic vise for HCA workholding while removing PCBs using levering.
• Cutter Module with a pneumatic cutter for the extraction of batteries from HCA devices.
The steps of the general disassembly workflow employing the modules mentioned above are:
• Utilize machine vision to detect locations and types of devices on the input module.
• Transport the object to the corresponding fixture based on the predicted step - a CNC fixture in the case of smoke detectors and a 4-jaw pneumatic vise in the case of HCAs.
• Smoke detectors are cut open using the CNC milling machine, while a levering operation is used for HCAs to extract the PCB along with the battery.
This video is part of the #ReconCycle project deliverable D5.6 and was prepared by the Department of Automatics, biocybernetics and robotics at Jožef Stefan Institute, Ljubljana, Slovenia.
Details can be found in
B. Kuster, M. Simonič, and A. Ude (2024) Vision-Language Model Based Robot Action Prediction for E-Waste Disassembly Operations, 40th Anniversary of the IEEE International Conference on Robotics and Automation (ICRA), pp. 539-541
#robotics #vlm #disassembly