12.1: What is the Kinect? - Kinect and Processing Tutorial

In this video I discuss how to get started working with the Microsoft Kinect in Processing using the Open Kinect for Processing library. I look at the different versions of the kinect -- 1414, 1473, v1 vs v2, and discuss in basic terms how the kinect itself works. I discuss how the kinect works in the context of computer vision applications.
All examples: github.com/shi...
Open Kinect links:
github.com/shi...
shiffman.net/p5...
Contact: / shiffman
Next video:
• 12.2: The Depth Image ...
Learn Processing from scratch:
www.youtube.co...
More about image processing in Processing:
• 10: Images and Pixels ...
Help us caption & translate this video!
amara.org/v/Qbti/
📄 Code of Conduct: github.com/Cod...

[@ChaoteLab]

5 years in the making! #shiffmanforpresident

[@deepankverma]

Apparently, IR is the rightest one. But never mind. This series is what I needed for long time. Thanks for bringing it up.

[@TheCodingTrain]

+deepank verma ah, thanks for the note, I should add an annotation to correct.

[@mrsloth-qw8pt]

You are awesome!

[@matheusd.rodrigues429]

His shirt has a green stripe, it distracted me for longer than I'd like to confess haha

[@QuarktaschemitSenf]

A lot of nerd knowledge, humble and funny. Liked and subbed

[@repunknante]

you are awesome!! very good teacher, you are very funny!! I am just introuducing myself into processing

[@QuinnWaters]

these videos are so great!! thank you very much!!!!!!

[@svenkraker]

This is great, just what I needed!

[@Picnicly]

thanks for this!

[@actmyage]

Hi Daniel, I realize this is an older tutorial but I'm hoping to find some resources to get OpenKinect working on my Windows 10 machine. Are you aware of any communities who are still actively using this?

[@abdullahjhatial2614]

did you find any solution? i also wanna use in windows 10

[@alokkr029]

when you have too much knowledge you start laughing unnesserly.

[@polyclickkkkk]

I like how your tshirt has the same green as the green screen :D

[@stormtuper]

OMG YES!! I WANTED TO DO A PROJECT WITH KINECT :D

[@studiobits2atoms706]

Thank you for this amazing tutorial. When I start with "import org.openkinect.processing.*; Kinect2 kinect2; void setup(){ kinect2 = new Kinect2(this); kinect2.initDevice(); }" I have a kinect v2 and I'm getting this error... "64 windows Init Kinect2 [Freenect2Impl] enumerating devices... [Freenect2Impl] 11 usb devices connected [Freenect2Impl] failed to open Kinect v2 @1:30! [Freenect2Impl] found 0 devices No Device Connected! Cannot Find Devices EXIT closing Kinect v2 : " but Kinect does work with microsoft Kinect studio. Any suggestions?

[@TheDoggyChannelVideo]

Daniel, i just bought the book "making things see" and with this video(s), and your super hard awesome work, im finally in on kinect's world... thanks a lot!!!

[@TheCodingTrain]

+TheDoggyChannelVideo so glad to hear!!

[@Rednesswahn]

So the new Kinect does it like radar. Those waves also travel (near) to the speed of light.

[@moazalhady7895]

First it was a great video but , Can the kinect work outdoor ??

[@cookieenick]

found something at last THANKS

[@s22407707]

I came here because I'm wondering how Hawk-Eye in Tennis works. Looks like a good tutorial for dummy.

[@INT_MAX]

[Freenect2DeviceImpl] failed to open Kinect v2 @1:18! This happens to me under Windows 10. I can use my Kinect just fine with the official SDK, but I can't for the life of me get this tutorial to work in 2017.

[@samuellorette1590]

Hello Daniel. Thank you very much for all these very interesting videos. Do you also use the Kinect for the green background subtraction (I think this is called ChromaKey?).

[@luisrey8986]

Excelente! 👍

[@kumareshr5975]

You'r tutorials are Awesome please do for c# and app development also request :)

[@bullseye6763]

My self, and only my self i love.

[@ChaoteLab]

1414, por vida!

[@zeetangled]

Thank you so much!

[@nanibrisque]

Thank you.

[@helenalexandra6821]

Hi Daniel, I am curious if there is a an open kinect for processing skeleton example for mac? Is it only for pc?

[@sher.5027]

awesome

[@MysticalLoveAndLight]

I have a question for you… I have a Kinect SLS… I have ?no idea what model it is ..I bought it off of eBay… My question is when I look at my front room or any room without it I obviously know how my house or the room is set up ..if I look at the room through the Kinect The room is flipped or mirrored another words if my TV is on the right side of my fireplace when I look at the area through the connect tablet the TV is on the left side why is that ? is that normal for everyone or is there someway for me to fix it ? for it to show exactly how the room actually is when looking at it through the tablet… Thank you Blessed Be

[@aerotman2003]

How does the infrared projector work? It seems silly to project IR, in other words it sounds like you’re adding flash to a camera that’s already over exposed. How does the projector help the sensor and not mask the already present IR wavelengths in it’s natural setting?

[@1322441]

is it same for windows users also??

[@mazmaxman1]

I need to know how the calibration done between the IR CAM and the RGB CAM. Its important to me to know how to calibrate between 3d point cloud and image.

[@linchen7258]

Sad, it seems that we cannot install "Open Kinect for Processing" anymore. It says "Could not find a library in the downloaded file".

[@methodwritingmedia]

I had this issue too Lin. I'm using Kinect 1414, Windows, am stuck because I can't install the library. Does anyone have any suggestions in here please? :)

[@methodwritingmedia]

PS have got the Kinect to work with 1.8 SDK examples but can't do C, C# coding

[@adventurer_omi]

hello..i want to start working on kinect with rasberry pi please help me from where i start..

[@qzorn4440]

Great video information. How would this work with the RockChip NanoPi M4. Thank you.

[@MrSDNeo]

Nice video, but no instruction on how to hook up the Kinect to get the damn thing working. I like everyone else seem to be having an issue finding the Kinect. Although I can see it in device manager. Windows 10

[@videosdemo100]

Hi, I want to conect my camera kinect to my computer, I added a PCI card USB 3.0 (with chip renesas - that is a requirement), but the computer doesn't recognize the camera. The configuration test detect that USB port is not supported. What should I do? the power adapter is 12 volt, 1.67 amperes. Can you help me solve this problem?

[@bondell1840]

if i usewindows, should i install the open kinect processing too?

[@chitralalawat8106]

how to import kinect library into python spyder.. ?? Do i need to have kinect camera to run this library??

[@alexgloba]

Buenas tardes quiero molestar con una pregunta porque mi kinect 1473 no me funciona con processing como hago para que funcione, me puedes ayudar.

[@pkfoodzhub]

Sir I have alot of complication in installation of kinect library ..kindly tell me steps how to install libfreenect and freenect2 packages in python . Thank you

[@ambientsoda106]

What is the Version 2 kinect as current 1473 seems to be version 1 and the newest kinect us version 3...I'm confused...

[@CaptainChainsaw2]

Hi Daniel, I found a channel called "Really Haunted" and the videos document some sort of haunting. A Kinect is used and appears to be using the infra red sensor and the Kinect displays a stick figure for an invisible entity. How can this be possible? Surely if it can see something and place a stick figure on it, it should also be able to show the raw footage of the invisible entity? Would love to hear your thoughts on this.

[@katiahebbache1936]

Hi, please for SDK which one i will use SDK V1 or SDK V2, knowing that I'm going to use a kinect for Xbox 360 for a windows system PC. Thank you Cordialy

[@liuyanliu6761]

Hello Daniel, would this work on Mac too?

[@user-fs3bo8ff3c]

Please what I can make with p5??

[@1322441]

sir i am trying to follow a book arduino and kinect project bookbut i am not able to complete the hello world project its showing some or the other error please help..

[@chitralalawat8106]

great video..but what is its use??

[@wenguang6597]

Hallo Daniel, thx for your Video. Can you tell me how can i see the Programm in SDK, which makes people-detection ?

[@varshabalaji1875]

Is there a library for RpLidar A1 . Please upload a tutorial or share a github link

[@sagaragrawal240]

SIR CAN U PLZ GIVE ANY IDEA ABOUT PROJECT DETECTING OBSTACLES BY ACQURING IMAGES WITH KINECT AND CHANGE IN ROBOT MOVEMENT

[@kseniag3337]

Why are the kinect examples not working with Processing 4.0? is there an easy fix?

[@elfpimp1]

I have windows 10, I don't have an Xbox one. Which 360 kinect should I look at getting for 3D scanning? Looking to scan objects for 3D printing. Thanks in advance.

[@eleazarherrera9268]

Thanks for the video Daniel! !How I can use this library on windows 8 and processing 3?

[@hemasreejonnalagadda]

I am working on this for a project. I need help in working with kinect. How can I contact you?

[@seankhoo2849]

Will the kinect 360 and also following the tutorial work in 2019?

[@calebhawkins3868]

Just tried this, the older kinect 360 does not work well with windows 10. I would recommend using the Kinect for Xbox One

[@bondell1840]

@@calebhawkins3868 are you using windows? am trying to do this too..but using 360 one

[@calebhawkins3868]

@@bondell1840 I ended up getting the adapter for the usb. I'm running windows ten on the surface laptop 2. I think it may have been the adapter but It wouldn't recognize the connect. the kinect for xbox one worked great though.

[@calebhawkins3868]

@@bondell1840 There may be some things I missed trying the 360 one though. I found that you have to go to your comp settings and switch the camera and mic

[@THEcucufate]

So, basically not available for windows and this whole cool tutorial will not apply for me and Processing?

[@ahtiolavi]

Hi! I've been testing this library with the Kinect v1 and my Macbook, and it has worked well with it. However, as I got my Kinect v2 and the adapter for it, I can't seem to get it work. The code is fine but all I get is a black screen. The Kinect's Xbox light is not turning on. Is this a common problem or am I just missing something out? The Kinect should work okay and as far as I know, my Macbook does have USB3 ports. Thanks!

[@ahtiolavi]

It seems like I get the same error as some in the comments below: "No device connected". The forums didn't have an explanation for this.

[@TheCodingTrain]

frustrating! it works for, not sure what could be the problem?? does it work if you try other kinect applications?

[@ahtiolavi]

+Daniel Shiffman I don't think it gets any power at all. I tried connecting it to my Windows 7 PC, it detects a connected USB device with the sound notification but it doesn't show up on the devices or light up the Kinect's power light. Someone in the comments mentioned her device woke up after connecting to Windows 8. I unfortunately don't have xbox either but the first owner of the device said he tested it and it worked.

[@ahtiolavi]

+Daniel Shiffman Some say it would require eg. Bootcamp, but that can't be it, right?

[@NzedY]

did Dan ever do a p5.js tutorial for this?

[@sportmenthol3696]

Noobie question here will i be able to run your library on windows 10 "Open Kinect for Processing" using processing 3 and kinect 1473 ? I'm planning to use this for my Thesis.

[@rileyrfitzpatrick]

Anybody else getting `A library relies on native code that's not available. Or only works properly when the sketch is run as a 32-bit application. ` I'm Windows 10, up to date Processing, just installed usb drivers with Zadig (I think). Any ideas?

[@AlexAranda45]

does anyone know a way to reveal another image under what the kinect V2 sees?

[@fernandolobina6459]

is there anyway of making this data avail to use in quartz composer?

[@xesigndesign]

hello there. is there a library for integration with Leapmotion ? thank you

[@AliyAkhbar]

Hi shiffman! how i do change color of getRawBodyTrack/getBodyTrackImage?

[@eatmykwad]

do you need a driver for the kinect to do this on mac ?

[@mrb34]

Can anyone advise what hardware I need to connect the Kinect v2 to my macbook pro? The Kinect I bought appears to have the USB 3 Type-B connection, and all the adaptors I have found seem to be for P.C. only... Any help would be very much appreciated!

[@seereen2004]

can we have the code that you used to get these 4 images together? ... thank you

[@mauriciobasseni4515]

How could i fix the next problem? : Failed to open camera subdevice or it is not disabled. Failed to open motor subddevice or it is not disabled.Failed to open audio subdevice or it is not disabled.Could not open device. I have windows 10, 64bits, kinnect 1414 (V1) I did tried everithing, but the problems still. i already did installed libusK, libus32bits, microsoft SDK 1.8 (compatible version, win7, win8) and, try it with some differents versions of processing. i dont know what to do, Help please !

[@alexischavez5014]

If I have ubuntu linux, this library I can do? or do I have to install more. . I was trying and so far did not work

[@alexischavez5014]

as for example the library open_NITE

[@joelborja4786]

@@alexischavez5014 did you ever solve this?

[@alexischavez5014]

@@joelborja4786 No, I did not try again either

[@surajbarthy]

This isn't working now with Processing 4. Is there something else we can explore?

[@TheCodingTrain]

Hi Suraj! I dont have one to test anymore but I would love for this to still work with Processing 4. You could also consider using ML models like google.github.io/mediapipe/solutions/pose instead!

[@AmelNGR]

Hello, is there a way to load .xef file on processing ? thank you

[@stephenderosa2412]

Do the libraries work for Microsoft SDK?

[@fasiltilahun8782]

hi danile, i wan tnks for ur tutorial,please show me the step that develop arduino control with processing the radar controler(fishfinding) with source code

[@AtariXcore]

Did you write code to switch cameras when you move from the one location to the next?

[@TheCodingTrain]

I wish! It's a feature of OBS (open broadcast studio)

[@danacozzi]

Hi Daniel! How are you? I need to do a transforming mirror, and I will work with kinect 1473 and processing 2.2.1 . Specifically you would need to capture the skeleton of the user, any recommendations? I was told to try the library simple open ni , but when I test one of the examples (User) processing window just looks white , ie not shoot me error. PS: Sorry if this poorly expressed , use the translator because I do not speak English . From already thank you very much

[@TheCodingTrain]

if you are on windows look for the official microsoft SDK, also ask at forum.processing.org.

[@g43s]

can i have a link for the kinect v2??

[@anuj771998]

Is there any python library for Kinect ?

[@ambientsoda106]

I'm really pissed as first I could not get the kinect to work after installing the 1,8 v1 drivers, then the lusb and even the 2.0 drivers; then at some completely unrelated times it magically started working...now it stopped working about a week or so ago, so I got a new camera and found the same issue and I'm using the 1414; so I am pissed because i built a dissertation project where the kinect sends data to an arduino after processing it and enhances a artficial environement....and I found old post where others have had the same issues...

[@jackpot1377]

Hello everybody! I just bought a kinect 2 and an expresscard-usb3 adaptator to be able to plug it on my computer... So I went to download the drivers and now I realise that it's made only for Windows 8 and above! Does someone have been in the same situation and find a solution to install it on Windows 7? I realy don't want to upgrade my Window's version... Thanks!

[@arieframadhan5113]

is this tutorial work on windows 10 ?

[@donato5391]

Hello!! where can I find processing and other programs for the management of the kinect? I use mac osx 10.7.5 thank you

[@BenHall289]

google

[@skiobaldo]

Hi, i want to know if it is possible to use Depth information to erase background from the Registered Image from kinect v2.

[@skiobaldo]

Omg, i figure it out! it's really simple: // Threshold the depth image int[] rawDepth = kinect2.getRawDepth(); PImage rgb = kinect2.getRegisteredImage(); // store the calibrated rgb image for (int i=0; i < rawDepth.length; i++) { if (rawDepth[i] >= minDepth && rawDepth[i]

[@tsiougos]

is this library able to run on linux?

[@TheCodingTrain]

+Melchizedek yes, it does!

[@tsiougos]

+Daniel Shiffman wow fast response. thanks mister Shiffman

[@alexdw5]

Hey Dan, what do if we have a v1 Kinect? I've tried changing to "Kinect2" variable to "Kinect" to no luck. Any suggestions? Thanks, really appreciate all of your tutorials!!

[@TheCodingTrain]

+Alex Wagner All of these examples should work, however there are small syntax differences. I'm making a note to post versions for v1 kinect. For now I would suggest looking at these examples to start: github.com/shiffman/OpenKinect-for-Processing/tree/master/OpenKinect-Processing/examples/Kinect_v1

[@alexdw5]

+Daniel Shiffman thanks a bunch, Dan! Will do!

[@louiserobson9135]

Are there extra steps to installing the openkinect library other than doing it through the Processing software? When I run an example code I get an error: A library relies on native code that's not available. Or only works properly when the sketch is run as a 64-bit application.

[@BRRRRRRRRETTTTT]

same here

[@BRRRRRRRRETTTTT]

i changed some of the code in the examples where it was erroring and got it working

[@carlamagica]

Does someone knows where to buy a kinect?

[@nishankpriydarshi3984]

ebay or ur local computer store, can find a second-hand kinect there

[@bknxxenacion]

How to setting Kinect with Mac OS High Sierra ? Some one know how to setting with this os?

[@jingxinwang947]

Hi,daniel, im going to buy a kinect now, and i guess mac will work with kinect v2 ? Because i have done some kinect hacking things with my mac before, will that influence the performance of the version 2 in the future?

[@TheCodingTrain]

+Jingxin Wang It shouldn't as far as I know!

[@ahtiolavi]

+Daniel Shiffman How does the Kinect v2 work with mac? I read that it needs a different adapter than v1, but that it requires Windows 8? Or what do I need besides the Kinect itself?

[@jingxinwang947]

+Ahti Nykänen I have successed it by downloading the openKinect library in Processing, and connect the Kinect to the computer and plug it in power before running the code in exemples. Strangely the kinect didn't react the first time i connect it to my mac pro or windows system, but after i download a windows kinect driver and connect it to the windows system, it works both with windows and mac afterwards.

[@ahtiolavi]

+Jingxin Wang But it does need the adapter with a power cord, right?

[@TheCodingTrain]

+Ahti Nykänen Yes, you need the adapter and you also need to make sure your mac has a USB 3.0 compatible port (most new ones do.)

[@matthewloewen8644]

Why us the kinect vs a webcam?

[@Finite8614]

did you watch the video? 0:4:05

[@sriramanvijay]

Hey Daniel , Awesome videos . first of all i want to thank you for these tutorials . I am very new to kinect . i am using kinect v2 to make an air drum and finding it very difficult to use openni libraries . I am using Windows 10 and processing 3.0.1 . can you please help me with this ?? i would be really grateful to you

[@TheCodingTrain]

+sriraman vijay You may need to use the windows only pv2 library github.com/ThomasLengeling/KinectPV2 I would suggest posting a thread on forum.processing.org for further discussion.

[@tqtff7064]

+Daniel Shiffman Thanks for your tutorials, I have a question, how to delete the contour of objects in depth image, as you can, the contours are in black.

[@tqtff7064]

+Daniel Shiffman Because the black contours will lead to errors in distance measurement.

[@hrodvithit]

Green stripe on the shirt is very dirstracting

[@InkDropFalls]

still bigest issue of all is getting it to work on windows 10 ...If you can solve that part then coding will be easy...:P

[@faisalikhlaq837]

After searching for almost a month with no use for Kinect V2 Tutorials that cover basics as i am totally new to Kinect development. I came upon this video and its for Java and i am working with C# and WPF. :/

[@TheCodingTrain]

+Faisal Ikhlaq Was this helpful to you?

[@faisalikhlaq837]

Yes this video was helpful as it covers basic principles about Kinect V2 working. Simple and easy explanation (Y). Can you plz refer the same kind of tutorials for .net framework. Thanks...

[@TheCodingTrain]

+Faisal Ikhlaq Right now I am focusing on Processing and p5.js, but I'll consider other frameworks at some point in the future as well. Thanks for the feedback!

[@faisalikhlaq837]

+Daniel Shiffman Waiting anxiously for the tutorials and thumbs up for the awesome work.. :)

[@black-pill]

only kinect v2 or one yah ~

[@lemorlenny]

Who said the italians gesticulate? :). Thanks anyway.

[@andrewkiminhwan]

this was as close as we get to a MTV CRIBS haha

[@TheCodingTrain]

+andrew kim hah!

[@koobanhee]

I want to use them on my graduation media art work . I plan to stick some graphics to people's body when they move. before they move, graphics draft around screen But I don't know how to write code on processing help me :(

[@stefanmisic7405]

Learn Java

[@Ikpoppy]

6:35 60.96cm