PID Controlled Ping Pong Ball Juggling Robot With 90FPS Realtime Image Processing

This is a Raspberry Pi Ping Pong Juggling Robot. This time it doesn't use a plate but a real ping pong racket. I use Nema17 stepper motors for the movements. I bought some belt drive components and ball bearings. I used those to convert the axial motion of the steppers into linear motion. It's the same setup as in an inkjet printer.
The main controller of this project is a Raspberry Pi 2. It has a Raspberry Pi camera module NoIR connected to it. Because it is NoIR, there's no IR filter on the thing. I originally planned to use infrared light to brighten up the ball in front of the camera. But it didn't work well. The IR elements I used were way to weak. The infrared noise in the background (sunlight etc.) would overweight the infrared light getting reflected from the ball. Another problem was, that the IR signal gets picked up on the red-channel of the camera. This means that even if there is no infrared noise from sunlight, the infrared light's intensity still has to surpass the normal red light's intensity to be recognizable. So that idea got dropped. I ended up using the green-channel on the camera with green light from a RGB Power LED pointed at the ping pong ball. Sure enough, normal white light would've done the job as well, but I just happened to have a RGB LED laying around so I ended up with green.
Another important thing is this: You might be asking yourself "Why did you use a light source anyway? Couldn't you just use OpenCV and get the thing to recognize the white ball with normal lightning conditions?". The answer to this is: Probably not. At least not with a Raspberry Pi 2. The reason being that we need 90 fps real-time image processing to track that ball. The image processing has to be done fast. I didn't really use OpenCV for anything at this Point in time but my guess is, that it wouldn't work well, because I don't think I could get the processing rate up to 90 fps with OpenCV. Maybe I am wrong about this, but it was my gut feeling, so I went on to program a really simple algorithm that could take care of the image data really fast. The algorithm looks directly at the data stream, so no conversion to a different datatype (e.g. NumPy) is needed. This is one reason this algorithm is quite fast. The other reason is, of course, that it is a dumb algorithm. All it does is look for a clump of bright green pixels on a fairly black background and then compute the center of it.
After the algorithm extracted 3D coordinates out of the images it examines that data and, in the case the ball is going downwards and gets quite near to the racket, it uses that specific data set in a PID controller to get some useful correction values on the racket as the ball is moving towards it (the machine tries to get the ball into the center of the racket). The movement values are then sent to the Arduino Uno over the serial port. The Arduino Uno runs grbl (a program for running stepper motors very efficiently) and all it does most of the time is waiting for some new instructions from it's master; The Raspberry Pi.
Some lines about the tilting mechanism: The tilt in the racket is produced by different heights of the 3 belts on which the racket is mounted. The joint "belt to racket" is a rather elastic one. This is why a tilted racket can be accomplished without ruining the joints or other parts of the machine.
It's worth mentioning that the thing isn't working perfectly at all. A 2 hour juggling session is definitely not one of the things this machine is capable of doing. It is able to juggle the ball for about 2 mins. Then it makes some misjudgment and the ball drops. So it isn't great. But then again: It only uses one single camera to compute 3D coordinates of a flying object. using 2 cameras would improve the performance. But it would also make the thing more complicated.
The Stepper motors are Nema17 Steppers! It Also uses an Arduino Uno with Arduino CNC-Shield.
The Arduino Stepper Motor Controller uses the great grbl software! here a link:
github.com/grbl/grbl
And here's the complete Raspberry Pi source code (Python3):
github.com/T-Kuhn/PingPongPi
And here's a blog post about the thing:
electrondust.com/2017/10/28/t...
Music:
"Covered In Oil" by Broke For Free
www.brokeforfree.com

[@pinolux21]

You are an excellent electronic, mathematician and programmer. Congratulations incredible what you achieved !!

[@Peter-sj6yn]

Nice Video! I like the way the bounce height varies.

[@ElectronDust]

Thanks Peter!

[@KulbirAhluwalia]

Holy Table Tennis! Damn!

[@thezone363]

this is sick I might want to try this for a group project with some friends

[@MrKnutriis]

thezone363 how did it go?

[@helxsz]

wonderful work !!!

[@ElectronDust]

Thank you!

[@OussamaOusmoi]

Awesome !

[@3bschool995]

Nice bro keep going.

[@demej00]

Great project. Would be interesting to use a machine learning algorithm and see if it can break that 2 min barrier.

[@ElectronDust]

There sure is room for improvement. Although I am a bit worried about the data. I've got a feeling that the problem isn't so much the controlling algorithm (in this case it is a PID controller), but rather the data being too rough.

[@revimfadli4666]

@@ElectronDust what about machine learning to un-rough the data?

[@pedrorafaelvidalarias6517]

Great video! I would like to see this on your robotic arm.

[@ElectronDust]

Hey Pedro! The robot arm is way to weak to hold that racket. ^^

[@pedrorafaelvidalarias6517]

Yes but if a larger robotic arm is used so that it could manipulate the racket. Would it be possible?

[@ElectronDust]

Yes. I think there are some people doing that here on KZbin. If you use an industrial robot arm it shouldn't be a problem at all. ^^

[@pedrorafaelvidalarias6517]

I see. I hope you upload more videos about this. Excellent work. Thank you!

[@ElectronDust]

Oh. Thank you! Glad you like it.

[@ipadize]

1900: We will have flying Cars in the future! now:...

[@Chris-bg8mk]

What language did you code it in? C? Assembly?

[@user255]

Python.

[@user-ks8qn5sz5b]

In your Arduino code there are so many code we can't understand how you juggling the platform . in your readme file, it is also not mention that how to approch your code, can you please send me code of arduino because we are building same thing for our courser project

[@indiarobotschool9680]

Bro where u study help