Drone Visual-Based 2D Indoor Navigation with ROS

[@matlabbe]

Update: Bug at 3:56 doesn't seem to appear anymore in latest PX4 versions. On Noetic with PX4 v1.12.3, it could fly fully autonomous over 60 min without any sign of that bug. PX4 v1.8.2 was used in this video under Melodic.

[@ur-techpartner_de]

Hi matlabbe, awesome work. Are you planing to have ros2 branch. Since px4 officially supporting XRECDD agent using ros2 now . what are important changes we need to run this without mavros , using nav2 and ros2.

[@hariharanramamurthy9946]

hi when in launch gazebo i am getting error, ERROR [px4] Startup script returned with return value: 32512 ERROR [param] Parameter SYS_RESTART_TYPE not found. --> i have no idea about the error

[@hariharanramamurthy9946]

hi i read your question in d435 map error in rtabmap_forum, can you say why you used it as a "subscribe_scan" & "subscribe_scan", isnt the d435 a rgbd sensor, please answer this , and also can you share the repo used for simulation please , I am trying to implement same but ,I dont understand any part of rtabmap in documentation

[@bartoszziembicki6199]

Can "rtabmap_ros::VoxelLayer" and "rtabmap_ros::StaticLayer" be used also as layer plugins in global/local costmaps in Nav2 (ROS2)?

[@mathAI42]

I don't think so, we would need to port them to ROS2.

[@MarkRuvald]

Awesome! This is with monocular RGB sensor right? What compute requirements does this need? GPU or is CPU possible too?

[@matlabbe]

This is RGB-D, a depth camera is required to see obstacles accurately. GPU only used for simulator and visualization. All navigation stuff are done on CPU. To use on less powerful computer, you may have to switch the visual odometry approach used by a lighter one (ideally a VIO approach).

[@jonsundar]

@@matlabbe How do you think using a stereo camera in the place of RGBD effect the output? Great job by the way.

[@matlabbe]

@@jonsundar With stereo, you cannot "see" white/textureless walls, which are often seen in indoor environments. As a workaround, tape a pattern on the walls at the height at which the drone will fly, then it will be able to avoid them!

[@jonsundar]

@@matlabbe Thank you for answering that. I am just in the conception stage of a mobile application for building construction and infrastructure industry. Your tech will certainly be the future in construction applications👍

[@KensonLeung0]

This is awesome. How to make your apt as a gazebo world (how you generate the .dae file)?

[@matlabbe]

I scanned with Rtabmap for google tango, but it can now also be done woth rtabmap for iOS (should have iphone pro or ipad pro with LiDAR). You can then export in OBJ from the app. To convert to dae, I did it with Meshlab free softwars, open OBJ, then export as dae. Hope it will you getting nice textured environments in Gazebo!

[@KensonLeung0]

​@@matlabbe thanks for the instructions. i am using samsung s22 ultra (not tango supported but arcore with deptAPI). rtabmap on play store reported tango core outdated and terminated. I guess I have to build from source.. Do you have the build from source instructions for ios/android app?

[@matlabbe]

@@KensonLeung0 I don't provide the android app on play store for other android phones than Google Tango phones because it is too hard to support. I got a version on Huawei P30 Pro or Samsung Note10+ "kinda" working, having both a TOF camera, but since the TOF camera is not officially integrated in ARCore, I am not satisfied with the results and we have to re-code / re-test on every different phones, which is unsustainable (cannot buy every android phones...). It seems there is no TOF camera on S11 Ultra, I think Samsung stopped to put one in their new phones. It is why I recommend iOS now, Apple did a good job integrating the LiDAR camera, same API for all their phones. ARKit is also a lot closer to Google Tango, than ARCore is to Google Tango. If I remember well, I think Apple bought the company behind Google Tango back in the days... so ARKit is kinda Google Tango 2.0.

[@KensonLeung0]

@@matlabbe i just looked into the android code and i can see the effort you've put on catching the individual android models. I agree that the ios one is much more consistent and sustainable. Thank you i will work on the ios one.

[@zeeshan5735]

rosrun package_name Offboard Does not work...the terminal has output neither the drone starts floating Can u tell what could have gone wrong

[@matlabbe]

Just tested again on ROS Noetic ubuntu 20.04 and it works as expected. Make sure you are using the right version of mavros/px4 like in the readme of the example.

[@2612cristhian]

what type of rgbd camera do you use?

[@matlabbe]

In this example it was a simulated RGB-D camera.

[@MarkRuvald]

What drone did you use?

[@matlabbe]

This is a scaled down version (0.6 the size) of default simulated drone from PX4 project (www.arducopter.co.uk/iris-quadcopter-uav.html).