OpenCV Python Neural Network Autonomous RC Car

May 2015
Self driving RC car:
OpenCV neural network - Steering
Haar-cascade classifiers - Stop sign and traffic light detection
Ultrasonic sensor - Front collision avoidance
Raspberry Pi - Data streaming (video and sensor)
Arduino - RC car control
BGM:
[Hunter x Hunter 2011] Original Soundtrack 3
28 - Holding A Card File üm-ün
Apologize for the ad, at this moment, the bgm cannot be changed or removed.
Source Code:
github.com/hamuchiwa/AutoRCCar
zhengludwig.wordpress.com/pro...

[@jessex8535]

Imagine how cool it'd be to have a whole bunch of them driving randomly on a little paper roadmap following laws and such.

[@funkymonk4688]

I support this idea 1000%

[@JeevayPakistan]

It was only responding to static objects. So I think there is long way to go. Unless they have already done it.

[@pedrorneto]

@@JeevayPakistan Tesla and other companies already does this well, but there is need to many improvements indeed.

[@Ghryst]

"obeys the law" ... "is autonomous"... you fucking millenials dont know wtf yr talking about

[@akashchoudhary8162]

@@Ghryst In this case, autonomous only means free from any manual driver control, not free from anything and everything. It could still be following traffic laws and be autonomous.

[@tanvirchowdhury1518]

never wear a green outfit infront of this car

[@randhir4278]

Tanvir Chowdhury lolololololoooloo

[@science_mbg]

but it recognizes rectangular traffic light shape and recognizes green light later. Not all green things. It should be under that shape

[@ThatGuyDownInThe]

@@science_mbg what is with programmers and jokes flying 3,000 miles above there heads

[@science_mbg]

@@ThatGuyDownInThe coundt get it that he might be sarcastic my bad

[@ThatGuyDownInThe]

@@science_mbg now I feel mean :/

[@pedroam7218]

Fake. There is a small person inside the car

[@HaiPham-nv3yb]

lol i'm dying

[@warclown9419]

😂😂

[@rocking1833]

Oh! Are you in that car?

[@kermitinmountain6371]

🤣😆😂🤣😆😂🤣😆😂

[@TintuVlogger]

😁😁😁

[@Grensom]

Great Job!

[@deepL0]

Im your follower and like you

[@cybercodes369]

This game is amazing . Check it out kzbin.info/www/bejne/g33Gpmapd7Cbmqc.

[@benitokim1793]

@@cybercodes369 Why should I trust a Microsoft Spy?

[@cybercodes369]

@@benitokim1793 ok then keep sucking 😎

[@henson2k]

Perhaps the only self driving Range Rover out there ;-)

[@landroverfriends]

I am studying Embedded System Development and I would find it awesome to go working for Land Rover. So maybe in the future there will be ;)

[@TheGrayWolf81]

Lucid Gaming No please don't work for that awful car company. I'd rather work for a company which makes reliable cars.

[@landroverfriends]

TheGrayWolf81 Whats the problem you had with the brand?

[@TheGrayWolf81]

They're just unreliable cars. My friend got a brand new one not too long ago and it's already having problems. Any other person I've known who has one has had problems with it (although some older models are nice and don't really have problems). I've never had much luck with English/British cars :/ But there are probably other people who love them and haven't had any problems yet. If I were to buy or do anything with a car, it'd be a Toyota or any other Japanese engineered car. They're the most damn reliable car I've ever owned and seen.

[@Jadanti6]

Why do you have a verification badge

[@toadiri]

congrats for this project dude. it's really brilliant and thx for sharing your work. u rock !!!

[@Rether16]

Wow awesome work! Seems simple to most people but there is a lot of logic going on behind the scenes! Love it!

[@tarunalakhramsing7861]

The Car is a paid actor

[@kakulmeow]

Can confirm. I was the camera

[@Tymczaq10]

imagine making few of these cars and making mini city, then upscale this

[@animee2003]

It takes more time

[@geryfabrega6749]

The impressive part is that this was made 5 years ago before all these easy to use libraries.

[@KevinMsyah]

How did they managed to build it back then??

[@nareshramini5984]

They mentioned Opencv

[@NaturalKiller]

People like you who perhaps do not have all the technical and financial support that could have, are the ones who truly make the future since with less they do much more, the truth my brother I congratulate you

[@darshank9659]

Wow first I was getting old videos in recommendation and now I am getting the same video that I watched years back giving me LOL and nostalgic feeling altogether.

[@TheDiamondGames]

Whoah! That is impressive! In fact, incredible!

[@raghudatheshgp6231]

Nice work Wang it's good to see that you are sharing your knowledge with world.

[@centaurirobotics87]

centaurirobotics.in/learn/

[@OmkarShinde22]

This is awesome! Please make a detailed video series on this project.

[@tdharmendra3204]

Good work! Way too good to see some of the common sensors act too smart!

[@apoorvpathak5378]

This Guy: I finally did it. Stackoverflow: [Leonardo DiCaprio DRINKING MEME]

[@Jaxenormus]

PewPewZee LawL I'm not sure if you were joking or not or if you are a programmer but trust me, this was a joke lol and is probably correct

[@dragonlordsaviour7005]

nice

[@TanveerAhmed-yi9tq]

Wang Zheng bro can you share the paper ? the link you have given is not working

[@illuminated2438]

I absolutely love the fact it's a red Range Rover. The most awesome vehicle out there and quite possibly its best color.

[@MuhammadBilal-lp8xx]

i am computer vision engineer myself but this work that too 5 years past,,, this work of your astounds me

[@raz0229]

Incredible Peace Of Artwork There!!!

[@AniketDEV]

*why don't you make an udemy course on this topic!!*

[@kressckerl]

Because udemy is a platform that isnt to creators advantage, find a different platform

[@jayamankrah4328]

Sound pretty good

[@hdang1997]

@@electronicsengineering7375 Thanks mate!

[@centaurirobotics87]

Hey, check this tutorial series to make your own autonomous mobile robot centaurirobotics.in/learn/

[@HalloweenJoni99]

There is a github repo linked in the description. Just read the code

[@jajanito]

question, how do you measure the distances to the obstacles or the traffic signals? Do you use distance sensors or analyze the image?

[@icarus7471]

Great job brother I am truly amazed. I also thought of this kinda idea but didn't give it a shot.Great job man ,great job !!!!

[@171Manoj]

Excellent work , image processing at its best . May I know how did you measure the distance between the car and Signal post ??

[@wangzheng7526]

Measure the distance between the camera and the bounding box of the detected object.

[@EliteCubingAlliance]

Distance is one of the easiest things to do

[@AryanKapur0605]

@@wangzheng7526 Can you please make a tutorial on this

[@shiva9989509341]

please make an instructable

[@anubismva]

Shiva Kumar Read his paper.

[@harshitsomani1717]

Where can we read the paper

[@tehdanny682]

@@harshitsomani1717 It's the last link in the video description.

[@woolfel]

Pretty cool, I was thinking about doing something like this back in 2015 and did some testing to see the feasibility of building autonomous racing robots with raspberry pi. The performance of python with openCV was too slow to process 30 frames per second based on simple benchmarks. I thought about off loading the image processing to a workstation, but decided to wait until raspberry pi got a faster processor. With raspberry pi 3 B+, I think it has just enough power to do it all on board without offloading the processing to a desktop or workstation.

[@JONOVID]

a good Autonomous car project. keep up the work!

[@truthinck8853]

I'm curious to know why you chose mono vision instead of stereo vision?

[@Aterrebgnuoy]

He probably wanted to use that Raspberry Pi which only supports one camera at a time. Therefore, he used the ultrasonic sensor to detect depth instead.

[@freezerbdn8358]

Aterrebgnuoy you can use 2 camera using USB

[@Dxeus]

Distance Measurement Raspberry Pi can only support one pi camera module. Using two USB web cameras will bring extra weight to the RC car and also seems unpractical. Therefore, monocular vision method is chosen. zhengludwig.wordpress.com/projects/self-driving-rc-car/

[@user-ef2dj8rt8e]

There was no need for stereo vision. Stereo vision of for depth sensing and wider FOV. The ultrasonic sensor does the depth sensing while the FOV is good enough. So no need for stereo vision. FOV- Field of view

[@lrw0475]

That's an incredibly great piece of work. I'm wondering what the car will behave when it sees a car with a red light or a "stop" sign on the back?

[@md.mushahidulislamshamim7335]

Is the RC car steering is controlled in a definite angle (ex: 1 degree, 2degree etc) or it is just left and right?

[@inquisitortr7930]

Can you tell me how you select steering angle. As i can see you just have 4 option at the end of the softmax layer (left, right, straight, stop). But how you denote steering angle.

[@fengyouliu8937]

1:43 the fact that it knows that there is space in between the obstacle and itself is pretty great, I though it had to wait until the obstacle was out of the frame.

[@haroonrehman8156]

It's using Ultrasonic sensor to measure distance between obstacle and the car itself

[@lalala213]

I read the description of your project on your wordpress and have a question if you don't mind: For your training dataset of input image and output direction input, did you hand label each of the images to have a certain output? For example, if the track is a slight left turn, did you take, assuming 10 fps, 30/40/50 images and hand label each to make a turn when you feel it is appropriate to make a turn? Thanks and great work!

[@ThingsMihirDoes]

I am curious to know about this too.. Is the training data hand labelled or you trained it by controlling the car via RC..

[@lalala213]

I just found out via his github and it confirmed my suspicions. He trains manually, saving a training image whenever he makes a keypress.

[@oggy112]

Nontheless still an impressive piece of engineering. Whatever you do in life there's room for improvement, he can improve the training.

[@mohsensalimy66]

hi, what was the processing unit called ? handles following : multiple tasks: receiving data from Raspberry Pi, neural network training and prediction

[@HarikrishnaNalla9]

Great work!! Appreciating your efforts..

[@centaurirobotics87]

Hey Harikrishna, now you can make your own autonomous mobile robot. Check this out: centaurirobotics.in/learn/

[@riteshmaheshwari5882]

stuart little was there in the car.

[@alexdeshift6733]

I am buying ruspberry. Looks really cool. Like.

[@ismailaliyazicioglu61]

Nice GTR and very good job too..👍

[@sidneybernardin6484]

Finally, technology and Hunter X Hunter combined.

[@SpotTiger]

Very impressive! 🤔

[@colegartner3339]

This is the task that many beginner python programmers feel like they're ready for after making a "Hello World!" program

[@researchitechindia]

Wow a plend of neural and sensors it's great to watch

[@AhmedAymanM]

Why did i Just start loving OpenCV and going to learn more about it?

[@guaren6611]

:o I remember I saw this many years ago, and it was my inspiration to do an AI course and learn a lot. I never got to do this though, hope I can soon!

[@odiljanandith3817]

There is a MIT OCW course on AI. Course Number 6.034 It is FREE. 🙂

[@Bisofication]

That R34 though

[@davidpossoc271]

Very nice project. Congratulations!

[@melihcelik9797]

Hey just wondering, how did you gather enough data to train this model? Did you take corner pictures?

[@Qimchiy]

Now you can recreate that GTA SA mission with the RC helicopter carrying barrels.

[@hardikshettigar1644]

That was in vice city 😂

[@hardikshettigar1644]

@Hot Dog Oh, one in VC was to carry bombs.

[@geronimowilliamson2173]

I will learn how to do this for my car someday, thanks.

[@harsh-haria]

are all the calculations and the decisions taken on the raspberry pi or on the server pc(which is displaying the data)?

[@AhmetOmerOzgen]

Arduino, sonar range finder, camera, opencv, pwm, lots of effort knowledge and intelligence. Nice

[@annaanna3260]

Imagine having like 20 of these cars you could simulate road traffic with it

[@TobiasLeininger]

They would all interfere with each other as he uses an ultrasonic transducer on the front that can't handle that. And the car doesn't look for other traffic at the intersection. And so on ...

[@reyou7]

Can you make step-by-step video tutorial for this? This is just amazing.

[@kratikapatidar336]

yes plz send step bystep this projcet

[@pperez1224]

Sure , you just have to pay the price , you beggar!

[@joelapin953]

kzbin.info/www/bejne/fZ_Rd6xmqquYaac

[@reyou7]

@@pperez1224 🙄🙌🤬

[@HarshKumar-hy1ug]

that was really cool... awesome work

[@centaurirobotics87]

centaurirobotics.in/learn/

[@jaydeeppatidar5434]

can you please tell what was average time to process single frame taken by camera. what i have done so far it takes too much time to process the data using HOG detector. thanks

[@aureliobuarque192]

what are you using to supply the raspberry pi? A battery?

[@wangzheng7526]

yep

[@manikamishra9495]

Aurélio Buarque thiydd

[@barraolivier2582]

No, no, no. An R34GT-R will never be an obstacle in front !

[@nicethobababro8262]

True

[@nguyenangthang7473]

hello! I want to ask about the distance from the Street light to the camera, how can we measure that approximately? Thank you!

[@gurribuy]

what will happen in low contrast situation, will the system perform as good.?

[@devansh8095]

Me: still Print("hello world) Output: Error EOF while scanning Computer shutdown Edit : oh need to edit this ... Guys I am trying to make a dumb joke and u guys are replying that" this is correct this is wrong"....... I know

[@kennydiaz3080]

print() can’t be capitalized. And I did get the joke.

[@devansh8095]

I know guys what the error I am just kidding. I am also a python programmer, And about 'p' it's is becoz yt comments

[@EigenA]

Wait I got it........... you are passing a double float. Please convert to cell array and import fun functions. Can you believe I failed my programming course work?

[@lowrhyan567]

def Print(text): print(text)

[@abhishekkk95]

Plot twist : A guy was controlling the car using remote

[@RaduGavry]

Please, can you show how you trained your haar cascades for the stop and traffic lights?

[@praveenkhanna783]

can you please help me on making host and port connection between computer and rasbperrypi

[@codingblaze4611]

Which camera did you use for getting the video stream for predicting the driving angles ?.Can you please tell me.

[@ParadiseQ]

I found this project interesting. However, it seems that there is no motor control from the Arduino or Rasperry Pi. I guess you are transferring the wireless signal back and using the remote the control the car...

[@RachelPecio]

there is motor control for RPi ;)

[@israelgoytom6085]

NO You are not right

[@Chrrvn]

So why there is ultrasonic sensor in front of the car ?

[@gianlucagrasso620]

Can i use a normal webcam instead the raspberry one?

[@israelgoytom6085]

yes, but you should change the code in Streaming

[@brhanudegene5037]

have you tried the code??

[@pjpassero]

This is like the best thing I’ve seen

[@oka-yoke]

How do you make image processing? In PC and get the results through Raspberry Pi or directly inside raspberry pi?

[@91722854]

dayum computer science rules, I a mech eng student and seeing these just made me feel that I'm ignorant. btw I am truly interested in machine learning ai and robotics, hope to work with u in the future if we ever get the chance.

[@lissobone7635]

I am just so happy that there is no Windows.

Congratulations for your awesome job ;)

[@none4882]

Hi I just wanna ask you that Is it detect only red color? or detect traffic shape something more?

[@YurkerYT]

Damn! you are good, Google should hire you to collaborate on their Autonomous car project. What's next? Autonomous drifting Nissan r34?

[@wangzheng7526]

YurkerYT Haha, good observation! Maybe 86 as it's easier to drift... ?

[@Ak-yz6td]

Yea they would hire if he did it 30 years ago

[@ManojFB]

Wang Zheng would you please share some tutorials. I am Python developer and I am interested in this field. please share some useful tutorials..

[@rizeadhikari5073]

Bro once you learn raspberry pi learn opencv machine learning and neural networks you will able to create this and more than this don't learn to make only a make your base strong you will able to make thousands of projects

[@rey1242]

Except that good dont meddle in such basic things, and instead research more professional and complicated works

[@husseinalaa47]

+Wang Zheng Thank you Wang, Would you tell me you training code that you used to train you haar classifier? I am working on same project and this is my training code: opencv_traincascade.exe -data hussein -vec sam.vec -bg neg1.txt -featureType LBP -numStages 6 -miniHitRate 0.999 -bt DAB -maxFalseAlarmRate 0.5 -numPos 900 -numNeg 2550 -featureType LBP -w 35 -h 62 -mode ALL -precalcValBufSize 1024 -precalcIdxBufSize 1024 and this is my makesamples code: opencv_createsamples -info pose.txt -vec sam.vec -num 998 -w 35 -h 62 but my classifier is still wont work well. last quesion: How did you calculate minHitRate, numStages ,FalseAlarmRate, numPps and numNeg? Thanks again

[@wangzheng7526]

+Hussein Alaa Here is the training code for stop sign (on mac): opencv_traincascade -data data -vec samples.vec -bg negatives.txt -numStages 20 -minHitRate 0.999 -maxFalseAlarmRate 0.5 -numPos 800 -numNeg 400 -featureType HAAR -w 20 -h 20 -mode ALL -precalcValBufSize 5120 -precalcIdxBufSize 5120 You need more stages to improve the result, I chose 20 stages (in fact, at stage 16, the acceptanceRatio is 400 : 8.33416e-05, which means the result is good enough, a couple more stages won't help much). Also, you might want to try HAAR featureType. LBP is good if you have high quality samples. The advantage of using LBP is not very significant in this stop sign case as it didn't take very long to train. Keeping your sample size small will reduce training time dramatically. 0.999 and 0.5 are the common values for minHitRate and maxFalseAlarmRate set in training. You can read some articles about adaboost to dig more. Below shows the final training stage status of stop sign: . . . ===== TRAINING 19-stage ===== Training until now has taken 0 days 0 hours 16 minutes 1 seconds.

[@pranavsridhar5241]

Hussein Alaa I’m also working on a similar project. Can you send me the documentation for your project? I’m having problems with Numpy.

[@Disheet]

where can i run server program in my pc? What is a procdure for starting only live streaming?

[@nikk797]

Brilliant Work, Dude!!!

[@MrJohn1966elliott]

what about keep left or right ? Like Australia is left drive, scan to left edge of road. ………. American is right drive.

[@Crucizer]

This isn't funny.

[@ahmedzia8539]

dude, try tweeting this to Elon Musk, he gon be pissed

[@Aditya-hd8qb]

Why will he be pissed? He would be happy. SMH

[@muzamilalishahmohammad6744]

Dude musk is not that type of guy 🤦‍♂️he would be happy to see everyone innovative and do something gud

[@muzamilalishahmohammad6744]

@@Aditya-hd8qb haha yes 💯

[@Aditya-hd8qb]

@@muzamilalishahmohammad6744 haha yes 💯

[@milkywaythecommenter3369]

This just simple OpenCV even though it's pretty hard. Don't compare it with Tesla's self driving feature where they use their own library with a REAL car with more advanced recognition system, i'm sure this dude's version is full of buges like OpenCV is known for. But Tesla has bugs too but relatively less, way less and don't forget the other cars and that this dude used Rasberry PI and Arduino. Tesla made their own I believe. Plz DONT woosh me 😭😭

[@AmCanTech]

Epic, Thank you for releasing the source code!

[@devintham1335]

Can anyone tell me is arduino uno necessary in this project in order to control the RC car? Thank you.

[@eldhokurian5781]

Please make a step by step video on how to make this

[@birdost4872]

Yes such tutorial will be a big step for the losers like us

[@limitless1692]

step 1 learn python step 2 learn aboun artificial neuronal networks ( the algo behind it ) step 3 code it , make mistakes , find them and solve problems easy !

[@zes3813]

Bir Dost no such thing as l /w, just you are loserx or step by othersx, stupidx

[@RobDawsonjr]

Looking at the source code, i believe its a Haar Cascade

[@elprofessor7000]

@@limitless1692 i know python ....and some of it frameworks...i want to do AI but i don't where begin

[@awaisahmed5489]

PLZ SAHRE WITH US YOUR PROGRAMMING AND FROM A to Z, HOW TO MADE THAT CAR

[@Cothebanchuabiet-09]

Is such automatic line detection in dispute with the signboards because the vehicle following the lane and following the sign is different?

[@belalnaim2005]

Well done, I could only dream of making something like that

[@___OmerAJ___]

great , it needs speed limit.

[@mamertvonn]

"Mom, can we have tesla at home" "We already have tesla at home" Tesla at home:

[@ankushsingh9871]

can we use the same raspberry board for both controlling rc car and performing camera processing?

[@bastiannenke7101]

Respect. I don't understand much about programming but i bet it was a lot of work. And it's awesome

[@avy0010]

It's 2018 Now that car is flying

[@shamirafridi4734]

It's 2020.. This car is in space now

[@richakumar9703]

Its 2020 june, the car has travelled to parallel universe and came back

[@sinapi6031]

In Soviet Russia, the car drives you

[@prateekpanwar646]

LOL IDK Why Russian drivers are considered very bad

[@sinapi6031]

@@prateekpanwar646 I guess that joke could be switch to Communist China drivers now In Communist China, the leadership drives you

[@matveynelson2489]

пошел в жопу

[@timurbadretdinov9555]

@@matveynelson2489 АХхахаах Бум

[@matveynelson2489]

@@timurbadretdinov9555 ?

[@lahernandezr]

Hello, your project is amazing, great job

[@giordano3176]

This absolutely revolutionary

[@yogeshkawar]

background music is lit..🔥🔥

[@twinklingstar2082]

Actually this project is quite simple. It has been just elegantly displayed.

[@shoemakerleve9]

Okay then explain the control system architecture, how the CV and image processing works. Could be easy but I'd like to hear!

[@emreipek4485]

Good working bro. Inspiring video. Thank you :)

[@derschwarzejulian7201]

So he used a Hunter x Hunter Theme to describe the epicness. Smart!

[@harrymakongwa1147]

Me:Watching a Remote controlled car yet it's called autonomous at 1am . Tesla entered the chat . Elon Musk is typing ...... Elon Musk: I need you at my Tesla Gigafactory in Storey County,Nevada

[@Soumya16]

😂

[@r.taufikutomoiswanindrakus8538]

imagine elon musk copy his code for tesla car

[@DavidJJJ]

Elon Musk has some slightly better hardware and software than a crappy Raspberry Pi and open source computer vision software.

[@Sanjar50bay]

Great job! Excellent! One moment... In real life if there is no stop line, you should stop near traffic light, in one line. But it's a small issue)) Good luck!

[@ayanangshudas3620]

Hi Wang Zheng, can you please tell me which Arduino model did you use.I have an Arduino Uno.Rev3 Will that suffise?