Target lock and tracking on FPV drone | AI Computer Vision Target Tracking Tech for Autonomous Drone

[@timelapsecoder]

Curious if I can convert these software projects into real homing drones? I'm getting there, but still saving for hardware and logistics. SUBSCRIBE to support me. Ask like-minded friends to subscribe also.

[@GG-Portal]

yeap, if you live in Ukraine)

[@Oleksandr-b5e]

There are many teams in Ukraine that are interested in technology like this. We can provide you with the necessary equipment for testing and development, engineering resources that can physically implement this in iron. Ukraine needs this for its defense. If you are willing to discuss this topic in more detail, please let me know.

[@artempogorelov7145]

How we can get connected ? It's looking very promissing. Please, ping me, it's would help us much.

[@МаксимКиндрицкий]

@artempogorelov7145 how can you be trusted? how do we know you are not russian ?

[@timelapsecoder]

@@artempogorelov7145 Still needs a lot of work before becoming a real working software. Will keep on testing on more drone footage.

[@sakshammishra8291]

I would like to develop a civilian use case for something based on this type of tracking. Sounds like an interesting concept with commercial potential

[@timelapsecoder]

You can experiment with existing OpenCV trackers and some lightweight ML models for object detection. From there, you can sell drones that follow its owner, like DJI for example.

[@Vanchick87]

Hello. Tell me where to get this code? I want to experiment with this. Thank you

[@SolitaryElite]

its private lol

[@DmitryX]

​@@SolitaryElitethis is a Ukrainian beggar

[@jonothankaplan]

Test it on yourself

[@SS-gu2tx]

I can help you no problem.

[@sugarvenom2189]

@@SS-gu2tx How?

[@easonzhao4941]

Great job! Just wonder what initializations are done when you click on the target.

[@timelapsecoder]

No initializations, other than maybe basic ones like approximate size of target, but even this is not that important. The algorithm has to immediately figure out what it needs. This is the 'magic sauce'. :) It does not work correctly all the time, so not so magic.

[@atrefonasful]

Curious what ai accelerator you are running? Are you running off a raspberry pi5 with coral or like a Jetson?

[@devinbin]

Where do you get your video materials from? Can you share them?

[@timelapsecoder]

Random videos from the internet or youtube. They are easy to search.

[@aviator1472]

Can you share please videos that yoy used to test you system?

[@timelapsecoder]

i don't have them. just a bunch of random videos from the internet.

[@PeterA650]

I'm actively teaching myself flight controller development from scratch. I have already developed a homing sensor for a certain application and it needs to get airborne. I can't find any other way to get it done other than designing it myself.

[@timelapsecoder]

Nice. Writing PID flight control software would be a good learning project. But if you wish to avoid the trouble, would Ardupilot or similar open-source software help your situation?

[@PeterA650]

@@timelapsecoder Ardupilot is designed mainly for GPS-guided waypoint navigation. There will be no GPS signal where my sensor will deploy.

[@timelapsecoder]

@@PeterA650 Maybe you can find the core PID code from university labs working on indoor autonomous drones. They probably won't share their vision, path-finding and navigation algorithms, but maybe they already shared their PID.

[@v0id683]

@@PeterA650 Maybe a little late but maybe a interesting project would be a UAV that orients around multiple ground based IR emitters, they are placed in a way that the drone can use their orientation to determine the drones height and location. I have no idea if this would work but just a idea

[@WRDO]

Are you using goturn tracker 😊

[@timelapsecoder]

no, i made my own private custom algorithm

[@surplusdriller1]

do you have an git repo? me and a friend is also working on some ideas like this

[@shakhawansalih5601]

Why not making laser guided systems ? can you do that like in rockets?

[@timelapsecoder]

It should be doable in theory. Replace the camera tracker with a laser tracker, plus a lasing unit to mark targets. Beam riding is a little different.

[@omumbare8582]

hey can you share the code for the HUD only? i have rest of the things, just missing a good HUD

[@SKM_KB]

You managed to program a drone with computer vision object tracking but are uncapable of drawing some green boxes on an image with opencv?

[@gianluca1905]

Lol​@@SKM_KB

[@aviator1472]

​@@SKM_KByeah. Some people suck in UI design. Me included)

[@aviator1472]

May i ask you one question - did you use openCV template matching?

[@SKM_KB]

@@aviator1472 What ui? literally some text and squares 🤡

[@오이닝-f2i]

1. did you develop the algorithm to track and shoot them down? Can you post the process of creating such a system, I'd love to learn. 2. Did you fly it by hand? Or is it an automated tracking flight?

[@timelapsecoder]

1. just the algorithms to track. can't share, private research. can only share demo. 2. neither. these are pre-made videos. just testing the tracking on fuzzy videos. automated tracking flight can be easy or hard. if simple flight commands, easy. if rapid maneuvering, hard.

[@AndriiMatsevytyi]

@@timelapsecoder did you use opencv algorithms or constructed your own from scratch?

[@timelapsecoder]

@@AndriiMatsevytyi made my own. did not benchmark against opencv trackers, but just testing what i could do from scratch. i use opencv for supporting functions like opening video, showing images, etc. this can be done in PIL also, but i already had an opencv base.

[@devinbin]

Hello, where do you collect your materials from?

[@timelapsecoder]

Hello, if you mean the video, same as where we find random things... the internet. :)

[@MohammadIbrahim-tf8rd]

Great What is the tracking algorithm this time?

[@timelapsecoder]

PF with some secret sauce! :)

[@andreyl2705]

@@timelapsecoder can you recommend a good resource to read about it?

[@timelapsecoder]

@andreyl2705 i think szeliski's computer vision book is considered very good. also forsyth's (haven't read both so don't know which one is better). i am recommending them because they are used as reference in cv classes. very math heavy though. if you just want cv concepts, i am sorry i do not know a good easy to digest resource. if you look for these online, you would also find many related links to cv lecture slides from the best cs schools. check them out and see which one is to your liking. you might understand the concept even if you gloss over the math. you can also study computational photography.

[@andreyl2705]

@@timelapsecoder thank you sir, very comprehensive answer) Have a nice day.

[@RobinHood-qg2xz]

hhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh @@timelapsecoder

[@aviator1472]

As i can see every time it locating an object it update the previous template that was used to detection to a new one and until the end. In the end of the video this algorithm starts to glitching and it gets road part as a template.

[@timelapsecoder]

Yes. The jamming static causes problems. There are ways around it. The transmission static is not a problem if the processing is onboard.

[@fpvlive3346]

I see a different timing. did you speed up your video recording when the tracker activated?

[@timelapsecoder]

Haven't look at this in a while but I have a timer sleep when the tracker is off so the video does not go by in a blink. So I am guessing my delay estimate (to approximate 30fps) is actually slower than the speed of the tracker processing logic, so the video looks slower when the tracker is off.

[@marchik1]

Do you have a model/discussion somewhere in github posted?

[@timelapsecoder]

This is not ML-based so no models. The problem I am solving is to track anything, not just pre-trained objects.

[@Dynamitry]

The next challenge is to learn a drone to precision bomb drop to a moving target and fly away to save itself.

[@timelapsecoder]

Accurate free-fall physics can be hard. :) Better to have a motherdrone with mini-drones like this. The motherdrone has high-power processing capacity for target identification, selection and designation. The mini-drones would be like this, using disposable homing tech.

[@maheshkumar_2017]

hey can we communicate?

[@timelapsecoder]

only through here :)

[@mmaelectronics9322]

Great job my friend, can you share the code with me?

[@withlovearoundtheworld4496]

But sometimes your tracker loss the track of the object and jump to other locations, in real world you will miss the target.

[@timelapsecoder]

The tracking loss is often due to the static from transmission on these test videos. That is assuming the tracking calculation is remote. Transmission is subject to jamming. In real application, the drone will have very clear video that feds an onboard tracking system. It can still miss due to algorithm limits, but far fewer and smaller errors than the big jumps you see here. Also, I am still improving this system. :) Finally, small errors can be manually overridden. Military grade drones tend to have terminal manual adjustments to help correct final approach trajectory (see Lancet, Spike, etc.; the last second aim adjustments do not correlate with a vision system, so they must be human corrections).

[@withlovearoundtheworld4496]

@@timelapsecoder also you need to handle some delays from the time you select the tracker position in the ground system and the time it update back to the drone, sometimes due to link delay it may place the tracker on the wrong position. The is solution for this problem be have buffer of frames in the memory of the drone and run from past till current time and place the tracker in the correct position.

[@timelapsecoder]

@@withlovearoundtheworld4496 I am taking it further (not shown). Selecting a moving target is hard. But I can freeze the frame and select a non-moving target, then have the tracker catch up to the live frame. This of course requires a faster than FPS algorithm, which this does.

[@CJShadow82]

Є можливість поділитись кодом? Я займаюсь схожими задачами, але з ШІ моделями.

[@phamnhuthai6847]

I actually did this 12 years ago. The concept almost same with yours, But not self suicide drone.

[@timelapsecoder]

People like to watch, and The Algorithm promotes, videos that are like this. If I track something harmless like a balloon, no one will click. Maybe I'll track a cat next.

[@ДмитрийФидорко]

Hello. Your work would be very useful to Ukraine. Would you be willing to share your code if contacted by the official representatives?

[@timelapsecoder]

Working on developing this into maybe a hardware system instead of plain text code.

[@sedatbahadir112]

If you are not a Russian citizen, we can do a joint project with you

[@theSpicyHam]

you are ironman 0.1 lol

[@timelapsecoder]

mk0.09

[@garymyers6638]

I would like to speak with you. How may I contact you?

[@michaelzhehalo2330]

Its fake video

[@timelapsecoder]

It's real, bro. trust me bro.

[@michaelzhehalo2330]

you taking russian money to make fake videos@@timelapsecoder

[@phamnhuthai6847]

You’re not programmer, you won’t understand. Try to learn hello world app first

[@michaelzhehalo2330]

first video it's STERNENKO video(left up angle), he showed first using Target lock several days ago, before it all user controlled, including this one. It doesn't work like on video.@@phamnhuthai6847

[@bessembousselmi677]

hhhhhhhhhhh cool honey pot

[@timetimer6429]

Я уже написал программу, которая наводится по упреждающей метке, теперь буду стараться сделать такой дрон, для того, что бы внести свой вклад в победу над неонацизмом

[@bromine_35]

This technology is of great importance to both sides This future was predicted long ago

[@makskk6spm275]

fake and post FX.

[@timelapsecoder]

Fake=no, post=yes, but you are free to interpret however you want. Good luck to you.