How to Control a Drone with the Keyboard using Python and Dronekit

[@immortalhamster1645]

Thank you so much man! I’ve been looking for something exactly like this! I’m try to learn what I need to do I can make a drone controlled in Vr. You helped a lot man!

[@olegmikulskiy]

Hi Tiziano, thank you for great tutorials. Did you try Gazebo simulator or just Ardupilot integrated solution?

[@joaobarreira5221]

Hi, Very nice video. Can I ask you a question? I want to start a project using python to control a Drone. for example play a bit with object recognition using the drone video camera. I am studying the hardware and since the budget is limited I looking for a drone with the following features: Programmable in Python (Through a PC or Raspbeery Pie integration) Could get sensor values in real time. Using a Pc remotely control a drone using python via Wifi or bluetooth Video Camera GPS (Optional) Can you point me a drone? It is not easy to find a drone.. best regards, John

[@radiof8]

awesome tutorials Tiziano!

[@justinkurtrivera2315]

Tiziano this is awesome I hope you will create another video similar to this

[@kevinoshea3687]

Does your programming works with Parrot Bebop 2.0 drone?

[@brockm2263]

another banger from the GOAT!!! thanks you save me sooooo much time!

[@heavenlypot]

Thank you so much! ❤️❤️❤️

[@rayanlobo5455]

This was really interesting. However, I wanted to write a python code to mimic the output of a remote control which would be controlled by keyboard keys and send it via a Raspberry Pi pin to the input of the Flight Control Board so that we would not have to actually interfere with the firmware and telemetry of the drone nor would we have to override the RC input

[@prandtlmayer]

Nice job. If you do any video and you want to share your results, just put it here and I'll mention it in my next video

[@rayanlobo5455]

Tiziano Fiorenzani Could you help me out with the code. I've written a bit but am unable to get a few parts

[@prandtlmayer]

Why don't you make a video or post a detailed question?

[@luizfelippeozorio6544]

To mimic the output of a remote control you can use Channel Overrides python.dronekit.io/examples/channel_overrides.html The problem here is that you are effectively overriding the RC input thus deeming any channel your code uses unusable on your actual remote control. You could write some logic to disable the RC override on a change on certain channel (maybe a mode switch) from your RC as a fail safe (use vehicle.channels['x'] to read the value of channel x). As an alternative, you could send your RC receiver input through your Raspberry pi and pass all input to the FC as RC overrides, enabling you to write an algorithm to combine both the physical input and your code's input to your liking. Another option is to use guided_nongps mode with set_attitude_target commands, dronekit has an example of this that is very straight forward. It is what I would recommend using if you're looking into autonomous flight or vision-based applications where RC input is not very important. Again, you could run your actual RC input through your companion computer to create blended outputs to your liking.

[@44anjan]

awesome tutorial Tiziano, Thanks!!. However, I've encountered an error-- "Link timeout, no heartbeat in 30 seconds, aborting". Any ideas on why this might be? What version of Python and dronekit are you using?

[@kavehghodstinat9500]

Hi, mavutil.mavlink.MAV_FRAME_BODY_OFFSET_NED command doesn't work with Rover, do you know any other command that can send velocity command for Rover? Thanks for the video by the way.

[@prandtlmayer]

Not an expert wit River, sorry. Ask on the community, but I strongly recommend you to open and study the code. As Obi Wan used to say: "use the source"

[@khtan5531]

Hi sir, for the condition_yaw() function, I observe that the drone yaws to the 'right' when given condition_yaw(45,1), but it does not respond when I enter a negative value for the angle argument. For example, it does not respond to condition_yaw(-45,1). How do I make the drone yaw to the 'left'?

[@Younn27]

Hi, to solve this problem, you just put a movement order before your condition_yaw. For example, you do in guided mode : take off then goto() and condition_yaw.

[@eeshvij6219]

This is 8 months late and you probably already solved this but the second parameter in the yaw function you mentioned is the direction. 1 is clockwise and -1 is counter-clockwise.

[@arpitsrivastava8431]

How to add install mission planner ,stimulator

[@prakhartiiwarii]

Genius! It works!

[@abdelz1617]

awesome video, thank you!

[@benjaminbarnes679]

Hi Tiziano first of all thanks for all your videos they are really helpful! I am having an issue with the DroneKit SITL - whenever I change out of guided flight mode the throttle cuts and the drone falls and crashes I was wondering if you had any helpful advice?

[@prandtlmayer]

If you set the drone in Loiter for example, it will expect to have a radio connected. With no radio, the throttle is set to zero and you descend rapidly. What you can do is connecting a joystick to Mission Planner and send it up as a rc. This will make you able to emulate the radio

[@benjaminbarnes679]

@@prandtlmayer Thanks for the reply. This seems to happen in all flight modes except for guided and RTL - I also receive a message on qground control that certain parameters are missing from the firmware could this be an issue with SITL? I have tried to reinstall everything multiple times and nothing seems to help. Everything was working fine until I tried to connect a joystick and emulate a radio. I believe something in this process actually caused the problem. Thank you again for the help

[@prandtlmayer]

In Loiter, stabilize and althold you need a radio. In waypoint, rtl and guided you don't

[@benjaminbarnes679]

@@prandtlmayer I have the same problem in auto flight mode and I never used to have this issue do you know if maybe there is some param which might need to be changed. I have tried resetting all the params to their original values and this did not help

[@lestergetubig9161]

What are the list of drones that can be controlled by this

[@omerfeyyazselcuk7325]

It's not about drones; it's about flight controllers. Any flight controller supports MAVProxy can be used. Any flight controller supports ROS can be used too, *but with different scripts.*

[@Rafaelnavarromobile]

Hi Tiziano, could you please explain why the keys up, down, left, are not working? Is there an update in the library. The "r" works fine. tks!

[@prandtlmayer]

I don't know, they should. Take a look at the code and if you fix it submit the merge on my GitHub

[@adamkraif3537]

Hi Tiziano, I managed to run the code and i get about 4 sec delay between the keyboard commands and the drone movement (when i click left or any other key it takes 4 seconds to affect the drone). any idea why?

[@prandtlmayer]

Wifi connection is the only thing I can think of

[@retrostark]

how to find tcp and udp from the laptop . please help me with that . i couldnt find it

[@narpat007]

Excellent !

[@leoakimono3631]

is there a way to connect radio to computer then control with keyboard and mouse and directly send signal? I am designing a method to fly FPV drones with keyboard and mouse. The keyboard is a special range keyboard I designed. I tried on liftoff simulator works really good. I never flied a real drone. And after 25 hours practice I break in all time world 23. So I really want to try it with real drones. Although I design the keyboard mainly for PC vehicle games.

[@prandtlmayer]

This was more of an example to show the ability to send command, but in theory you could. Maybe add some extra code to recover control with the RC

[@leoakimono3631]

@@prandtlmayer Thank you very much Tiziano. I will check the src code of ELRS to see what I can do.

[@Galaxy-hf7ms]

Why aren't more people like straight to the point​@@prandtlmayer

[@chencj4547]

Hello sir, thank you very much for the tutorial you provided. I can run normally on the computer of the ubuntu system, but when I want to run the code on the Raspberry Pi 3B +, I get an error: _tkinter.TclError: no display name and no $ DISPLAY environment variable, it seems that Tkinter can no longer build windows on the remote PC, is there any way to solve this problem? Or is there any way to replace it? thank you very much!

[@prandtlmayer]

You are trying to open a graphic window through ssh. You need x11 for that

[@dakshgarg775]

I know this is weird but can anyone tell how can i get the i IP on which i have to connect my script like the ip of the vehicle

[@prandtlmayer]

If you setup your vehicle you can set the IP as static. Otherwise, if you are connected on dhcp, you can use the hostname. Otherwise you can use a network scanner

[@sharoseali708]

i want to control servo through python script using drone-kit. ...kindly guide me please...

[@omerfeyyazselcuk7325]

You connect the servo to APM. Then you can control your servo via MAVLink.

[@MrMohamdMahdi]

Good video thanks. any way to arm the drone in Guided mode indoor ?

[@prandtlmayer]

guided_no_gps

[@MrMohamdMahdi]

@@prandtlmayer I tried Guided_no_gps mode but it has limitation and this code not working with it !

[@prandtlmayer]

That code was written when the mode did not exist yet

[@MrMohamdMahdi]

@@prandtlmayer Okey. Thanks.

[@grabitgo2724]

Please guide us that what is the set up for Windows!!!

[@antoniodibella6155]

Ottimo lavoro. Complimenti. Io ho un problema. Vorrei comandare in remoto direttamente i motori del mio rover. Sulla documentazione di dronekit ho trovato "set_actuator_target". Potrebbe fare al mio caso? Lei ha mai fatto qualcosa di simile? Grazie Buona giornata

[@prandtlmayer]

Con quel comando puoi settare direttamente i servo, anche se perde di significato andare a domani motori individualmente

[@dheerajchaudhary7132]

Hi, I was using these velocity commands for controlling pixhawk based copter. But as I give it up command, EKF primary changed:1 error came which led to drone crash. It happened to me twice. Earlier I was using these commands correctly for about a month. What could be the issue?

[@prandtlmayer]

This is mostly matter for the Ardupilot forum, and without details and log is hard to know. Just keep in mind that with the latest releases, the yaw rate input in the velocity message is read, unless you specify the correct bitmask

[@luizfelippeozorio6544]

Ekf is an algorithm that estimates your vehicle's state (or position, simply) taking in data from its many sensors. My guess was something is wrong with one of your sensors. Probably bad gps lock, check that your hdop is below 1 to ensure accurate positioning. If reliable positioning is crucial without good gps lock, consider adding a opticalflow sensor.

[@mcalokadarlene480]

Hi Sir, is it possible to add sonar sensor in dronekit

[@prandtlmayer]

Sure. Dronekit is just a wrapper around mavlink. You have a few choices. 1) you add sonars to Ardupilot. Follow the instructions on Ardupilot wiki. 2) you read the sonars with the companion and you send the readings to Ardupilot using mavlink messages. 3) you deal with sonars only on the companion and command the autopilot accordingly

[@prandtlmayer]

You want to bring your question on my newly born Facebook community page. Post your projects there

[@mcalokadarlene480]

Thank you sir, This sensor we need to use for the obstacle avoidance in SILS you any idea Sir.

[@jenniferliupl]

greating from Singapore.. awesome

[@kamwow9469]

Could there be a way to edit the code so that you can control how far it goes up and down using the keys

[@prandtlmayer]

Well, yes. Get the code on my GitHub and play around

[@Slothmother]

Anyone have any luck with getting these keyboard commands to work with a SITL rover instead of the copter? I am trying the script but am not seeing any movement out of my SITL rover

[@prandtlmayer]

I haven't tested with Rover, but I am sure you can convert the code to sending mavlink commands that a Rover understands

[@Slothmother]

@@prandtlmayer yes I believe so too but unfortunately have not cracked the code just yet. It looks like here github.com/ArduPilot/ardupilot/pull/4716 that only position parameters are used when sending set_position_target_local_ned to a rover. I have been trying that as well but am still unsuccessful. I am able to put the rover into GUIDED mode and arm it, so that doesn't seem to be the problem (I fixed the github link to be correct)

[@jeshwanthsaravanan]

Hi Bro I was build an drone without an controller and the cost of the controller is too high so can I use this program with raspberry pi 3 to control the drone "please take as a request Bro for my project"

[@prandtlmayer]

I wouldn't recommend you develop something without a backup rc transmitter. There are very cheap on line

[@jeshwanthsaravanan]

@@prandtlmayer Bro I'm having an "wireless key board "for that controller and the cost of the openpilot controller is too high so only I asked a software replacement

[@prandtlmayer]

If you don't want a transmitter at least be sure to program a key to stop everything and one for disarm. If you have a bug you need to override. Your choice.

[@jeshwanthsaravanan]

@@prandtlmayer thanks Bro sorry for the late reply

[@malakzitouni1924]

Hi Tiziano! Thank you I FOLLOWED the procedure and when i run the script of control with arrow keys end up with this "Error:autopilot:Disarming Motors"

[@prandtlmayer]

It's a little general. Consider that the vehicle disarms after a few seconds if it doesn't receive any command

[@malakzitouni1924]

@@prandtlmayer i use your script of control_with_arrow_keys, but doesn't work , PS: i can only work with tcp, if you can tell me how i am going to write in this command maybe that's the problem "mavproxy.py --master tcp: 127.0.0.1: 5760 --out udp: 127.0.0.1: 14551 --out udp: 10.55.222.117: 14550"

[@omerfeyyazselcuk7325]

I'd suggest you to watch "messages" tab on Mission Planner when arming - either via PC or radio controller.

[@4FALLEN8]

sir ,can u make a video using apm 2.6 please,it will very helpful and appreciated

[@prandtlmayer]

No, that board is dead. Do yourself a favor and get any 32 bit controller board instead

[@filippovannella4957]

Complimenti per il suo lavoro straordinario. Mi chiedevo se tramite un set-up raspberrypi+pixhawk e possibile costruire un interfaccia android con dei pad in maniera simile a quello che ha fatto lei in questo video. Grazie e buon lavoro

[@prandtlmayer]

Dovresti trovare progetti di GUI per android. Basta che ti costruisci un tuo protocollo di controllo UDP tra la gui android ed loscript python/dronekit sulla Pi ed avrai fatto. Se vuoi semplicemente controllare il veicolo con una stazione di controllo come Tower, basta che fai girare un proxy sulla Pi (mavproxy o mavlink-router)

[@filippovannella4957]

Grazie mille per la risposta! So come interfacciare la pi con tower e simulare python scripts su di esso. Quello che mi chiedevo é se la GUI android per il pad potesse essere di qualsiasi tipo (ad esempio questa ez-gui.com/manual/controlling-a-model-from-the-ez-gui/) e se fosse possibile calibrarla tramite uno script python/dronekit. Grazie ancora!

[@HieuNguyen-yt4sc]

I have a drone using pixhawk and wifi telemetry, how I can control drone using keyboard?(just pixhawk and wifi telemetry without raspberry)

[@prandtlmayer]

Build a script on the computer that sends udp messages to the companion. You can s be the button you pressed for example. On board you read the udp message and fly accordingly

[@HieuNguyen-yt4sc]

@@prandtlmayer what method I can use? i'm beginner

[@prandtlmayer]

I'm afraid you have to dig a little deeper and try for yourself. You can find a lot on my tutorials, on Caleb's course, on the Ardupilot website...

[@jenniferliupl]

@tiziano where can i get the codes ?

[@prandtlmayer]

On my github, see in the description. I've been to Singapore three years ago. Great experience, amazing food, fun city

[@lestergetubig9161]

What drone are you using

[@prandtlmayer]

Any Ardupilot compatible drone

[@fiqfik555]

can we change from udp to tcp?

[@prandtlmayer]

That is up to you. Udp is lower latency

[@fiqfik555]

@@prandtlmayer ouhh... okay thanks

[@lukeconnolly935]

Anyone get this to work with PX4 for a physical drone with no GPS? With optical flow sensor

[@prandtlmayer]

I don't think it's compatible with px4

[@kavehghodstinat9500]

Does Dronekit support python 3?

[@prandtlmayer]

It should

[@jeremiagedyjuliowattimena404]

is this GUI?

[@matkmichael5297]

How do I begin to code from python from scratch

[@prandtlmayer]

matk michael you do exactly as any of us did: studying and googling. Pythonprogramming.net is a good place to start

[@kophyo7461]

Thank

[@Boss-pu9uc]

Buongiorno! Complimenti per il video e la spiegazione fluidissima e molto comprensibile. Vorrei avere un suo consiglio: sto lavorando su un APM 2.6 (un ArduCopter) e vorrei generare a runtime, secondo un algoritmo, dei waypoints dinamici da impostare per la missione del drone. Il fatto è che Dronekit non lo eseguo su un companion PC connesso USB al drone, ma lo eseguo sul computer dove è eseguito anche il programma di ground control station (Mission Planner, per ora), comunicando con il drone tramite la radio 3dr di telemetria. Dronekit lo eseguo su Windows e, quando simulo virtualmente il drone o quando il drone è connesso USB al computer, il programma stabilisce la connessione al drone correttamente. Quando invece attacco la mia radio 3dr per la telemetria e comunico con il drone non più via USB ma via radio (come poi dovrà essere quando sarà operativo), ottengo un errore di timeout della connessione e non riesco a connettermi al drone. Ho impostato correttamente la baudrate (57600) e la COMport dove è inserita la radio. Come consiglierebbe di procedere? Se è interessato e disponibile a discuterne, le lascio la mia mail: [e-mail] Grazie e buona giornata

[@prandtlmayer]

Luca, innanzitutto dammi del tu. Se permetti ti rispondo in inglese, così magari aiuti anche altri con lo stesso problema (non sei l'unico). The question is about connecting an apm2.6 with dronekit running on windows on the ground through the 3dr radio. Ok, there are many things you'd rather do. First do not use an apm2.6 any longer. That board is no longer supported and some of the commands available in dronekit are not appliable to old releases like 3.2 and older. Second, 3dr radio just sucks. You'd better off with something more proficient like the rfd900, simply another level. Third, I'd run dronekit on the companion, like a pi zero w, 10 dollar and you have all you need and you just do all with the wifi. On the computer put a long range wifi dongle, create a hot spot and make the drone connect to it, pr you make the companion work as access point

[@Boss-pu9uc]

First of all, thank you a lot for the detailed answer. Secondly: let me recap to check if I got it. You suggest to replace the flight controller with a more suitable one (a Pixhawk would be ok? I never worked with any other flight controller, so I am trying to stick to the more versatile and simplest one), connect it (via USB?) through a companion PC like a Raspberry Pi Zero W, run Dronekit-Python on the Raspberry and send commands to it through a wi-fi connection. What I am trying to build is a system of multiple drones equipped with several gas sensors, which sense for gas leaks and autonomously navigate to reach the maximum of the leak, communicating with the ground station (for real time data logging) and possibly communicating with other drones to call for help. I guess it is possible to equip the Pixhawk with several analog and I2C gas sensors and communicate their data to the Raspberry, then let the Raspberry communicate them through Wi-Fi? Additionally: is there a way to have on a single ground station software on my laptop (e.g. Mission Planner) multiple drones connected to monitor their position and path? Thanks in advance, Luca

[@prandtlmayer]

A pixhawk is perfect, i even use the hobbyking's one and does great. For the connection with a companion I will post a video on how to do it, but in the meantime check the ardupilot documentation. It's very easy using either the usb or the telemetry port. I used both, no problem. Finally, i suggest you put your extra sensor and logics in the companion pc and let it guide the drone in guidance mode. You can setup a wifi network at the ground station and let all the drone's companion pcs connect to that. You can also visualize everything in mission planner. In each drone you run a mavproxy that outputs on both the local port and the mission planner ip. Follow my first and second video.

[@Boss-pu9uc]

Tiziano Fiorenzani Thank you! It happened that I switched to a Beaglebone Blue as a new flight controller, so now I have no need for a companion computer anymore, I run both ardupilot and a Python script with dronekit both on the Beaglebone board. Following your suggestion of using wifi as a connection, I am now adopting wifi to transmit to ground the data to a laptop running a custom Python script. I have a new question now: what device/long range dongle do you suggest in order to establish a really long range wifi network without the need to have a really bulky device? Thanks as always for your great experience and time. Luca

[@lestergetubig9161]

Can i control a naza mv2

[@prandtlmayer]

No

[@lestergetubig9161]

@@prandtlmayer Can i have a link that list the drone that can be controlled

[@hatemmahmoudi3880]

hello sir , thanx a lot for this awesome tutorial , it works for me fine but when i went to run it directly on navio 2 i get this "WARNING:dronekit:Link timeout, no heartbeat in last 5 seconds "

[@nauraizsubhan01]

awesomeness

[@lestergetubig9161]

Can i control any parrot drone

[@prandtlmayer]

If you can install Ardupilot

[@lestergetubig9161]

@@prandtlmayer thanks

[@АнтонАнгелов-ш5д]

Pasta, mana mia

[@prandtlmayer]

mamma mia

[@prithvirajkumar6825]

hey I've been trying to develop an autonomous drone using raspberry pi 3 b+ and pixhawk , I've successfully controlled it manually .When i try to control it with python script i wasn't able to execute simple_takeoff() function, but I was able to arm and disarm by drone using script. Can u help me with that .... Thanks in advance ..

[@prandtlmayer]

You can test the code with sitl and figure it better out. I can't tell with more details

[@prithvirajkumar6825]

@@prandtlmayer yaa I tried with sitl n the code seems to be working, but when I provide the same to the real drone it doesn't seems to react to that function...

[@prandtlmayer]

That function works, I'm sure of that. It might depend on a different setting that you have. Are you in guided?

[@prithvirajkumar6825]

@@prandtlmayer yaa the stone's in guided mode .... Is there anyway to gradually increase the throttle value just like the one we provide from our transmitter ?