PnP Calibration with Computer Vision!

So far, all of the component placement with the Index PnP has been dead reckoning; we THINK we know where things are, and so we place them blind. But now, we're adding in active, real-time calibration based on a computer vision pipeline built into OpenPnP! And our accuracy is about to go through the roof!
-------------------------------------------------------------------
Check out my Patreon!
/ stephenhawes
Join the STR Discord Server!:
discordapp.com...
Check out Index Machines merch!:
indexmachines....
Index Pick and Place Github Repo:
github.com/sph...
-- FOLLOW --
Twitch bit.ly/32q4pDG
Instagram bit.ly/2FrreOJ
Twitter bit.ly/2FArY3G
-- PO BOX --
Stephen Hawes
PO Box 44248
Pittsburgh, PA
15205
-- GEAR --
These are affiliate links so if you buy anything through them I get a small percentage which helps out the channel. Thank you!
PCB Holder: amzn.to/344qhVD
Microscope: amzn.to/348aSnw
Soldering Iron: amzn.to/2KlHDGs
Wire Strippers: amzn.to/2IESgDV
Cutting Mat: amzn.to/3oNCwOz
Flush Cutters: amzn.to/3ncFwnc
Digital Calipers: amzn.to/3gIzZSL
Leatherman Sidekick: amzn.to/3a4sQuY
Part Organizer: amzn.to/3a9uDyV
Soldering Iron Tip Cleaner: amzn.to/3a3wZiD
Oscilloscope: amzn.to/2W4zrNt
Multimeter: amzn.to/2JVp6Ba
Crimping Tool: amzn.to/37WymwP
SpaceMouse: amzn.to/3gCWR64
Tweezers: amzn.to/3a1auL5
Solder Wick: amzn.to/347kUVI
Files: amzn.to/37bMXp0
Dremel: amzn.to/3gIADj9
Headlamp: amzn.to/3gE5VYw
Pelican Case: amzn.to/3ncFpbg

[@olivercouch1651]

Thank goodness for the guys that wrote the vision system for OpenPNP! Standing on the shoulders of giants!

[@xxportalxx.]

Hahaha or someone who happened to spend an immense amount of time doing that one thing, most programming problems take a lot of time to work out, and after you're like 'well, thank god thats done, never gonna look at this crap again'

[@stephen_hawes]

absolutely. they've done some really incredible work! most of all Jason, the project lead: kzbin.info

[@remigiuszbloch]

I always thought it's illegal by law for engineers to be so cheerful and enthusiastic ;-)

[@ayanvaidya2727]

Yeah because they can change the world

[@xxportalxx.]

Hahaha he's young, they'll 'educate' that out of him in a few years tops 👌👍

[@alexanderfedorov7890]

@@xxportalxx. Yeah, Stephen always happy like Santa on Prozak getting laid In Disney Land ;))) Still fun to watch and follow his projects though!!! ;)))

[@xxportalxx.]

@@alexanderfedorov7890 hahahaha that description... idek what to say to that hahahaha

[@myetis1990]

actually the illegal thing is smoking weed before making video :) the result always will be cheers :))

[@pacmancoder]

Wow, watching this series from the beginning, and now we are here, pretty close to the fully working DIY PnP machine! Keep up!

[@kennorman3586]

I found that doing the bottom vision, the part/nozzle should be at the same Z as the board. This takes in account if your Z travel is not 90 degrees to the board.

[@olivercouch1651]

Good tip!

[@clanross62]

Indeed, getting everything (parts in feeders) the same height as the PCB made a significant improvement for me, did not think about bottom vision, great tip!

[@cryingfries8765]

The progress the Index has gone through is amazing. Good work, well done!

[@bytesizedengineering]

It's been so cool to watch your progress and success from the start!

[@siberx4]

It's interesting to see the use/motivation of vision cameras in a low-cost PnP like this. Amateur astrophotographers use a very similar approach where they'll have a second "guide camera" to measure and correct for tracking errors (by monitoring star positions in the sky) in their mounts while taking long exposures with their main camera. Like the PnP scenario, while you can technically get the accuracy you want just by having really accurate (and thus expensive) hardware, it's much cheaper overall to pair less expensive mechanics with a camera system and dynamically correct for errors. Very cool to see it used on a such a low-cost implementation!

[@enredao_electronico2737]

Nice !! Following you since day 1. Finally bought a 3D Printer which should be here In Dominican Republic anytime next week and build this awesome project!! Kudos

[@randycarter2001]

The top of the LED's is not flat. There is a discontinuity at the transition from the white body to the clear lens.

[@LaserUpgradesdotcom]

At work I can hardly even get some people interested in these things and you get excited about fiducials. I want to work with you when you are ready work in the industry. This is a very ambitious project!

[@AndyMcBlane]

Great video. Huge step forward, can't wait for the testing phase (and reliability fixes!)

[@SparrowHawk183]

Dude, awesome work! I had no idea this type of pick 'n place technology is so accessible for makers, wow!

[@chepossofare]

Man that's INSANE. Keep up.

[@JKTCGMV13]

What a creative necklace haha I've got the same unused part laying around, might have to steal his look

[@azyfloof]

I saw the thumbnail on my homepage and was like "Please be a new video! Please be a new video!!" 😍 I have been SO stoked for this project series, it's incredible! When are these gonna be available in kit form? :D

[@michaeld2438]

Looking good!! Keep up the good work :D

[@sammiller5509]

Damn, that thumbnail 😅🤣😂👍

[@Gooberpatrol66]

*Techpriest intensifies*

[@Gobhoblin126]

It might help to add a flat black surface under the head so the upward camera has a black background behind the part.

[@kentswan3230]

Is there a built-in way of using the bottom camera to calibrate the offset of the various nozzle tips under rotation? Additionally a way of chain calibrating the offset of the top camera to the tip rotational axis and the top camera axis to the bottom camera axis.

[@kennorman3586]

Yes, OpenPNP has the tip run-out built-in.

[@autonomousperson]

Try a black and white dedicated machine vision camera and some uniform soft lighting

[@daskasspatzle2396]

Absolutely amazing project :-)

[@johnl5235]

In an earlier video, you were concerned about glare due to the design of the lighting. Has that been a problem? If so, you should be able to fix it easily. Buy a circular polarizer just a little larger than the outline of your ring light. Remove it from the ring used to mount it on a camera, and mount it backwards covering both the ring light and camera lens. Circular polarizers are not symmetrical. By "backwards" I mean with the side that normally faces the camera instead facing away from the camera. If you lose track of which is which, look at your reflection in a mirror through the polarizer. One direction, you will be able to see your eye. That's the normal orientation -- your eye is on what is normally the "camera" side of the polarizer. The other direction, the polarizer will look like a black circle. The latter is what you want -- the side facing the mirror should face *away* from your camera & ring light.

[@juliankandlhofer7553]

Nice! Can it detect the arrow on the bottom of LEDs and the line on Diodes?

[@UhloMuhlo]

Nice! Are you going to publish some OpenPNP configuration that works best for the Index?

[@fischX]

If you building that thing by hand you better calibrate it after that specifically for that machine.

[@BrainSlugs83]

I saw the purple servo horn you were wearing and decided to subscribe. Is there a story there?

[@detprest8975]

Took me a while to understand WTF you were talking about but I just loved the energy and passion and now I know a little about PNP. Dope!! Just curious what your 9-5 is?

[@danp762]

Very Cool. 👍👍 Thanks for sharing!!

[@Belginator]

Great work dude I work in an electronics Rapid prototype centre. you are not far wrong. use a hardstop

[@FyJonas]

leaving a comment mostly for the youtoube algo to pick you up better, great stuff you´re doing here though! :D

[@ElectricEvan]

Yea it's smooth on openpnp now. I was on the mailing list back when they added that and the debate about exactly how it should go was painful.

[@mikesnapper9001]

I want to build this thing so bad, but it just doesn't make sense when i always only need to make one copy of a board i'm making usually

[@MM-cp4tw]

This is really impressive. I have a question. If the suction nozzle places a component on the board, what is the chance that when the nozzle releases pressure and lifts up, that the component will move slightly? Anyone know?

[@nikkismith8750]

Really impressive accuracy! I don't use anything smaller than 0805, but will the Index place even smaller components? and what's your goal for minimum pin-pitch on ICs ?

[@alsiqueira1]

Hi Pro Favor I would like to know the specifications of the lens you are using in your computer vision?? Excellent content

[@TheAnimystro]

I'm begging you to dust that glowtie board lol. Looks good mate

[@Factory400]

It looks like double sided tape was applied for the placement test so the parts stay where the machine places them.

[@TheAnimystro]

@@Factory400 Ah nicely noticed thanks :)

[@MD_Builds]

>.> umm the leds are 180' the wrong way round.... but other than that this project is amazing.... making me wonder if i should build one! (bad thought bad thought)

[@TheRemo176]

Why do you have a servo arm as a necklace ?

[@micultimy91]

I hope you clamped the PCB down when you did the calibrations, otherwise you have to perform that process again, this time CLAMPED. Clamping is inducing some bending in the panel that will change the visual offsets. Also, put a stopper for the panel on that conveyor. You'll thank yourself later for that :) Cheers!

[@gustinian]

Watch his previous video...

[@randyrandom5702]

:D new Stephan Video

[@zlatanjakic7717]

How do those really fast machines do this at such speed, or do they just not use vision?

[@mateoarv]

Nice! Do you think you'll get down to 0402 components?

[@shibbleswentworth]

Why aren't fiducials a crosshair to get a bit more information in one image?

[@kennorman3586]

Vision can be fickly at times, so KISS is the best route at times. I found going with a 1 mm silkscreen dot work better than a fiducial.

[@hansdietrich83]

@@kennorman3586 depending on the manufacturer, silkscreen can be quite a ways off. Better to do it in copper, which will always be realtive to the pads

[@bhavikbhansali]

Hey Amazing work. I just want to know that how to detect the polarity of the components? It's automatic or you need to manual give data to the PNP?

[@GroovBird]

I was about to give up life and oh.... a new video by Stephen! Cool!

[@connoredwards7013]

I wish we could spread Stephen's charisma like we've spread Covid.

[@Gooberpatrol66]

10 years from now: "I made a DIY CPU fab!"

[@clanross62]

Why are you "dropping" the components onto the board ?

[@CooperDuper3000]

Best youtube channel on the planet.. well maybe after James Bruton's channel and of course Cody'sLab. ^^

[@gufonero9405]

a pulley with a spring should do the job....p.s. love your work

[@jamcdonald120]

0:00 me: how can you calibrate a PnP transistor with a camera? That doesnt even make sense! 1:00 oooh, Pick n Place machine.... ok aaaaaand im done with this, good by

[@derrogers]

Why are you wearing a Servo-Horn as a necklace? Just curious...

[@cwbh10]

oh man, lifetime and validation ;)

[@rudygunawan1530]

Yeah, this concept has been applied to our automation liquid handling machines from few years ago.

[@hammerhead3044]

Hey Stephen I enjoy watching your videos and learning some great stuff from you. If you don't mind I have some additional questions I will like to email you. What us your email?

[@UndeadChew]

yeeeeehaw!

[@batugunduz3950]

I minimise the video to give a like multiple times during your videos. Such a shame you can only like videos once!

[@borgquads4669]

Why is his neck so horn-y

[@willsalazarramirez5139]

i am chiclayo peru 🇵🇪 SOUTH AMERICA 🌎