How to do Stereo Vision and Depth Estimation with OpenCV C++ and Python

[@NicolaiAI]

Join My AI Career Program 👉 www.nicolai-nielsen.com/aicareer Enroll in My School and Technical Courses 👉 www.nicos-school.com

[@nilutpolkashyap]

Last week I received an OAK D kit and then today I got to know about your channel (best combination ever).....your videos are so awesome....I instantly subscribed and became a big fan of your work....keep shining

[@NicolaiAI]

Thank you very much! Really appreciate it and hope that u will learn a lot. Definitely enjoy ur oak D kit, it's so cool

[@omarmurajia1642]

How the aggregate cost computation algorithm ?

[@schnebastian]

do you know, what kind of metrics or procedures autonomous car use to detect obstacles and the distance to the obstacles?

[@NicolaiAI]

They use deep learning. Tesla uses deep learning and neural nets for depth too, all the other guys are using lidar

[@schnebastian]

@@NicolaiAI so only Tesla uses the camera to detect obstacles?

[@NicolaiAI]

They all use cameras for obstacle detection but merge the data with lidar for depth also. So they create a 3d map of the whole environment

[@schnebastian]

@@NicolaiAI do they use sematic segmentation to detect the obstacles? do you also have a video about that?

[@NicolaiAI]

@@schnebastian yeah they use segmentation for the road, sidewalks and so on. I don't have videos about that yet unfortunately

[@janaeshkrish]

HI can you combine both Camera Calibration and depth estimation as a single vedio? That would be greate if you do so ..

[@fairozashour3339]

I want to get the steroe camera, but I don't know what type or version was used in the video. Can you help me?

[@NicolaiAI]

I'm just using two budget webcams as a stereo camera in this video

[@fairozashour3339]

@@NicolaiAI I want to use the steroe camera like you used it in the facial recognition video,Can any steroe camera get the same results, and can I use two separate cameras?

[@fairozashour3339]

@@NicolaiAIthank you know i inderstand

[@hamzaomari7052]

Thanks a lot, this is the best video I've seen about all the process of a simple stereovision system.

[@NicolaiAI]

Thanks a lot Hamza! Really appreciate it

[@SAINIVEDH]

Please make a video on deep learning based depth estimation works. Thanks

[@NicolaiAI]

Yea that's a really cool topic, that's certainly a video in the future. Thank you 🙂

[@nigellim8272]

Isit possible to get a pdf copy of the notes you shared in this video? so that i can refer to it for my project

[@NicolaiAI]

Some of them should be on GitHub somewhere

[@BlueWolf-ry2os]

I'm planning to use a single camera to detect various points of interest such as banks and pharmacies (specifically, to detect their logos). I have already trained a YOLOv8 model for this purpose and aim to calculate their 3D coordinates . Can YOU help me with a code that can calculate the 3D coordinates of detected objects in the image using a single camera? I'm considering treating two consecutive images from the camera as if they were captured by two separate cameras, thereby making stereo vision applicable. Additionally, the coordinates of the camera are always known via GPS

[@ShadowD2C]

Im kinda doing something similar, did you find any good solution?

[@QuarktaschemitSenf]

Hey Nicolai. What is a decent sterovision camera (maybe with usb) for a decent price atm?

[@NicolaiAI]

All depends on project, accuracy and price

[@QuarktaschemitSenf]

@@NicolaiAI Ok, more general. Do you have a prefered brand / shop for this kind of equipment. ? My goal : Detect and track objects on a custom trained yolo algorithm (by the way , how to do transfer learning e.g. add a custom class to coco nameset?) and track the distance of the objects. Price for stero-cam

[@NicolaiAI]

@@QuarktaschemitSenf u won’t get good real life results with Midas. But if u are looking for an easy to work with camera u can check out oak-d

[@QuarktaschemitSenf]

@@NicolaiAI Ok, ty for that Info Nicolai. Highly apreciate your interaction with your viewers. But lets say i have a good Stereo-Camera and want to use it for example for collision prevention. Can i do this with python/opencv without tracking certain objects like red ones. Would i need any additional things like reference points in the fov?

[@NicolaiAI]

@@QuarktaschemitSenf yeah u can do that with opencv without object detection. Basically just create a depth map from ur stereo camera and then avoid objects being close. Then u will also know the position of the objects. So it will more be avoid everything close than avoid certain objects

[@raihankhan5160]

Thanks a lot for explaining stereo projection in simple words rather than math and vectors :D (most computer vision books do) I wanted to ask, if the cameras were placed in orthogonal positions and not in same axis, can we apply the same formulas and techniques to find the depth estimation and corresponding 3D reconstruction to get 3D coordinates of an obstacle/detected object? Thanks in advance!

[@AbdulkerimBiz]

bro did you get any informat,on about this topic

[@SealionPrime]

Great video, clear and to the point!

[@NicolaiAI]

Thank you very much!

[@Bala92635]

I am bit confused with stereo vison approch and Structure from motion (Photogrammetry approach)

[@NicolaiAI]

What are u confused about?

[@Bala92635]

@@NicolaiAI whether these two approaches different or uses same algorithm two calculate 3d point

[@NicolaiAI]

@@Bala92635 yeah they are different, but u can create point clouds with both. I'm general u will get a more complete and better point cloud by using STFM since u have more views points and more information to create ur point clouds. I'll make some videos I'm the future about structure from motion

[@Bala92635]

@@NicolaiAI Thanks sir... Waiting for next SFM video

[@letianwang5141]

Awesome tutorial!

[@NicolaiAI]

Thank you very much!

[@salmansaleem4343]

Love this tutorial. Is it possible for you to share the code? Because the link you mentioned here does not refer to any page or repository on Github.

[@NicolaiAI]

Thank you very much! I have updated the link in the description

[@salmansaleem4343]

@@NicolaiAI Thank you so much for your quick reply.

[@satishsinha2221]

Hi, I recently explored some deep learning approaches for stereo and I am very new to it. Can you please help me by telling what’s the difference between stereo matching networks like GANet and stereo depth estimation networks like MVSnet. Thanks

[@faizaali895]

Thank you for a very informative video! I have written a program that already gives me the position of the object that I want to track, so I already got its x-coordinates from both cameras. Is it then necessary to create a disparity map of the whole frame in view or is it enough to just calculate the disparity of my one target point?

[@NicolaiAI]

If you already know the point you want the disparity for and you don't need any other points later on, then yes you can just calculate it for that point. I would still recommend you to calculate it for some of the neighbor points aswell for more precision. Would you mind me taking a look and maybe make a video about it so others learn from it aswell. If not that is totally okay and thank you for ur question and attention 🙂

[@faizaali895]

@@NicolaiAI I see, interesting! I don't seem to get the right value so I wonder if it matters if the target object is non-stationary? So the depth of it changes with time? The idea is to throw the target object (being a red ball) and calculate the depth while it is being thrown. Yes, I wouldn't mind sharing the code; however, it works in real-time if that's okay. So if you don't have a red ball I guess, the code won't be able to track anything :)

[@NicolaiAI]

It might be a bit of a problem if it's non stationary, but I think that it would help to calculate the disparity of an area instead of just a couple of points then :) can u send me the code to my email? Thank you - [e-mail]

[@watoysland1137]

I also tried to measure distance of object from camera. But it behave in different lighting conditions. How to get rid of this problem. I also want share result with you so kindly share email.

[@watoysland1137]

Faiza which stereo camera are you using?

[@fiNitEarth]

Thank you for this interesting video. What do you think about the approaches to use neural networks to compute depth from stereo pair images? Supposedly they're faster, but it would be interesting to hear what a pro thinks about it

[@NicolaiAI]

It depends on many things, but I'll prefer neural networks when u want to create full depth maps and not only the depth to couple of objects. Or if u need really good accuracy. The neural networks have become so strong and the are improving almost exponentially

[@milindpandya2230]

Loved this tutorial ✌️

[@NicolaiAI]

Thank you for watching, glad that you liked it!

[@ahmadrezayousefi-r1w]

thank you

[@souravsahoo1582]

i am immature in this subject........................but what you mean by depth of the image...i didn't clearly understand?????

[@NicolaiAI]

Depth in stereo vision is the depth or distance to the objects 🙂 so u have 3d information instead of only 2d

[@souravsahoo1582]

@@NicolaiAI like 3d information on 2d screen 🤔😐

[@NicolaiAI]

Yes i also have a video where i actually do it in code with a small project. Tracking an object and calculating the distance to it with stereo vision. So i have 3d information from 2x 2d cameras

[@souravsahoo1582]

@@NicolaiAI ok i definitely check it bro👍 Please make more tutorial on computer vision ..i am currently grad student in computer science..and take this subject as elective 😭😭

[@NicolaiAI]

@@souravsahoo1582 thank you 🙂 if you take computer vision as an elective course then everything in this computer vision tutorial here will cover more than enough for you

[@zainahmed4172]

20000 views?! Tillykke!!!

[@NicolaiAI]

Haha tak Zain!

[@zainahmed4172]

@@NicolaiAI det er velbekommen. Men nice video, og over 6000 views, du er populær. Men jeg kan rigtig godt lide hvor tydeligvis du forklarer i dine video. Stort pros til dig.

[@nealrowe6169]

Love this tutorial tbh

[@NicolaiAI]

Glad that you like it!

[@speak_charlie]

👍🏿👍🏿👍🏿

[@speak_charlie]

Do u have IEEE membership?

[@speak_charlie]

If yes i need a help

[@NicolaiAI]

Unfortunately I dont