Tesla Autopilot: A Computer Vision Perspective | Jitendra Malik and Lex Fridman

Full episode with Jitendra Malik (Jul 2020): • Jitendra Malik: Comput...
Clips channel (Lex Clips): / lexclips
Main channel (Lex Fridman): / lexfridman
(more links below)
Podcast full episodes playlist:
• Lex Fridman Podcast
Podcasts clips playlist:
• Lex Fridman Podcast Clips
Podcast website:
lexfridman.com/ai
Podcast on Apple Podcasts (iTunes):
apple.co/2lwqZIr
Podcast on Spotify:
spoti.fi/2nEwCF8
Podcast RSS:
lexfridman.com/category/ai/feed/
Jitendra Malik is a professor at Berkeley and one of the seminal figures in the field of computer vision, the kind before the deep learning revolution, and the kind after. He has been cited over 180,000 times and has mentored many world-class researchers in computer science.
Subscribe to this KZbin channel or connect on:
- Twitter: / lexfridman
- LinkedIn: / lexfridman
- Facebook: / lexfridman
- Instagram: / lexfridman
- Medium: / lexfridman
- Support on Patreon: / lexfridman

[@frednicholson]

The computer doesn't have to be better than a human in 98% of all conditions. If it can be better than a human in 75% of conditions that make up 99% of accidents, that's probably good enough. Thus, if the elephant falls out of the back of a truck and the Model S bisects it, that ok, because 100 inattentive drivers were saved from going off the road while checking their smart phones, or whatever.

[@mohammadessawy8879]

good point

[@rkalla]

Bingo

[@AverageJoe3]

The problem with this is the human psychology. Many people dies because of doctors do some stupid human error, like giving wrong sedative or dosage. Imagine going to a human dentist knowing that there is a 0.1% chance of a human error with wrong sedative dosage resulting in your death. Then you have a second option, go to a AI dentist, he has only 0.01% chance of malfunctioning, decapitating your head and ripping your spine, resulting in your death. Which one will you choose? What will make the headlines? Statistics are clear AI dentists are superior....

[@rkalla]

@@AverageJoe3 I have been decapitated twice at the dentist.

[@duplexdown]

so true. perfection is impossible. how much better does the computer have to be before it gets adopted without any human supervision? also might be inflection point where the more computers on the road, the more predictable the other cars are

[@gamble8008]

Idk if I’m smart enough to comment here but, I like that you asked the question twice since the first answer was focussed on Highway driving and we all care more about city.

[@joethompson9124]

Personally, I'm not particularly impressed by how he frames the problem. Interesting to hear nonetheless.

[@laplacesdemon45]

When was this recorded?

[@AdamWood]

Lex is really good at driving a point. ❤️

[@atomspalter2090]

nice video!

[@kipling1957]

Piaget is very salient on this aspect learning.

[@raddaks2039]

I really like his points here (as an avid Tesla fanboy who also uses Autopilot a lot). I think the future of machine learning is, as he says, a lot more than just neural nets. Neural nets are great for certain things, but they're a small piece of the puzzle when it comes to full-blown AI as a whole. I think he purposefully left his answer as open-ended here as he could, and that's what I would do, too. Time will tell. It could be that Tesla just simply has enough data to be able for a purely neural-net approach to just work. Autopilot is still in development, and rate of improvement doesn't seem to be slowing. I don't have terribly high hopes, but I'm keeping an open mind about it.

[@vuththiwattanathornkosithg5625]

Watching this make me realize one thing. If our natural system of behavior are fundamentally a quantum system so we can some what able to predict and take action accordingly to situations (to the best of the individual abilities according to their life experiences and training) therefore, wouldn't the Neuro Network which will do computation of the data need to be either mimic or is a quantum system it self?

[@listerdave1240]

@5:00 I think the skateboarder example is not specifically a machine problem. I think any person who is not familiar with skateboarders would have more or less the same problem. It is more a matter of the machine being 'younger' and having less experience than its human counterparts rather than something intrinsic to the machine itself.

[@meenoomeenoo4578]

U just said what he said in a different way.

[@daniel_960_]

The ai can collect the data from all the world. With a fleet like Teslas. While a human is limited to his own experience.

[@nroose]

Let's hope the new methods of learning that involve active experimentation by the computer will be entirely non-destructive experimentation! Sometimes when people do experimental learning, there are negative effects!

[@KaiHenningsen]

A lot can be done (and, in fact, is being done) with simulations for that. That's one way of learning that AI often does a lot better than humans, for example by speeding things up (doing a lot of experiments in a short time), possibly even parallelizing it.

A self driving car would not need to emulate a human driving. All those examples are beside the point. At best it would need something more akin to the level of a horse with some basic added rules about traffic regulations. All the unknowns are outside of typical traffic laws anyway (truck opens up , skater does weird patterns) . Simple avoid hitting something as best you can, try to stay in the road , don't swerve into oncoming traffic. Basically everything a good horse would do too. Doesn't mean there won't be any accidents and it will be super human.

[@n4rzul]

Agreed. People will also adapt to AP especially if it becomes more mainstream. You don't see kids skateboarding on railway tracks and expect the train to stop? The same will start happening for AP. There are other solutions to these problems too. Pressing a button at a pedestrian crossing for instance that sends out a wireless signal to cars that a pedestrian wants to cross. When there are more cars on the road that use AP and they communicate conditions to other APs, such a system could become very powerful indeed. It is about time that the way we use roads in general changes. There are far too many deaths on them at the moment with regular drivers and were not even discussing drunk drivers. Humans in general are *terrible* drivers.

[@LazerEyez]

Good point

[@llothsedai3989]

Assuming all the road miles on earth, 65 million round that up to 100 million divide by 3 billion autopilot miles driven in Tesla, that's 30 times a road driven per mile, that's the opportunity space from a blank map to determine base reality. Some regions will be more mapped then others globally so that concentration of miles driven, so it's more concentrated on certain areas of course where they drive. That's the opportunity space for intelligent software or ai to make sure that it performs the right action from a global picture, thats top down. Second from a top down approach is all combines of weird events in say a 50x50x10 Meter cube (your localized point in space), this would be the GPS for inaccurate spacial location and object detection. The bottom up approach is you want to replace the driver ultimately and those metrics, miles per accident/disingagement which you want to minimize. So to start with you want have senses in the spacial deminesions as inputs and differentiate between objects (cameras), then the ability to sense distance between objects,and 4 outputs (left/right/stop/go). Having a self accurate model of yourself is also important for knowing your position in space or for close distances (combination of an accurate dimension of the car - a cad model really and your relation to near objects (ultrasonics)). That's the basics of it. The rest would be the learning piece (the hard part really). This should get you to being able to drive with nobody else on the road anywhere you want in an effectively static world. Hardware seems to be improving at Moore's law, redundancy if a device breaks that it's not catastrophic, reiterate on failure and hardware improvements and that's should get you from anywhere you care about point a to point b in a static universe.

[@XXX_JiaYangGuiZi]

Finally, see an expert knows what he is talking when compare AI (self driving car) and human learning to drive. Amazon CEO compered AI’s learning ability to humans was so idiotic, but I didn’t heard anybody came out to point it out. It is so obvious to scientists human have been learning their whole life, and also millions of years of learning when it comes to driving a car. AI just started and has so little information feeding to them so far.

[@davefx7949]

"and also millions of years of learning when it comes to driving a car" - What? People have been driving cars for only around 100 years. These machines aren't starting from zero. They start from where we (creatures that have evolved for millions of years) push them too. They then eventually go far further than we could ever have dreamed.

Dave FX - I guess what Andy meant is that what we humans learnt thanks to millions of years of evolution is used to drive. Recognizing objects, understanding their interactions, etc.

[@mikebailey2970]

While I respect the technical content of his answer, its true humans have mastered the visual skills needed to drive at an early age, however on the other hand humans on average can't maintain focus on a specific task like driving for say 10 hours or hundreds of hours at a time. Autopilot/computer 100% focus on the data at hand is constant their modeling is getting better and will improve dramatically in the next few years.The fact computers don't get distracted like humans is immense. I think the most significant aspect concerning Level 5 is how many lives statistically are enough to say the system is 'good enough' for widespread adoption. Pragmatically I would hope the number would be be such that Autopilot/computer is 10x better than human drivers. From what I have seen I don't think Tesla has that far to go.

[@D_HongKongVideos]

Doubt it. All it takes is for AP to make one mistake after 5 minutes and your life is over. Level 3, never mind 5 needs a new paradigm. ML can only do so much. Any time a new thing comes up, computers are confused. Eg overturned cars, people wearing orange clothings confuses ML as large cones.

[@mikebailey2970]

@@D_HongKongVideos being 'confused' because ML has not seen it before is far different from being distracted. Being distracted would be being unaware of one's surroundings like humans can be at times. The examples you reference are the long tail of the problem, which may never been solved. However once AP has encountered something 'new' is it unlikely it will be 'confused' when encountered again. Humans forget, computers essentially do not. True while AP may make one mistake and your life is over, if AP saves thousands of lives over the period of a year then there is a moral imperative to allow AP to drive rather than a human prone to distraction.

[@D_HongKongVideos]

Mike Bailey I’m using Nassim Talebs fat tail issue There can’t be too many fat tails otherwise AP is unusable. I drive a Tesla, drives like a 7 year old and damn right dangerous here in Hong Kong. Phantom braking, acceleration and AEB warnings. It can’t cope with busy streets. Ok for highways

[@mikebailey2970]

@@D_HongKongVideos I respect your opinion regarding AP current suitability. However my primary premise was simply and I still think it is worth repeating.; Statistically how many lives have to be saved using AP or another Level 5 system before it is approved to be used on public roads.

[@D_HongKongVideos]

Mike Bailey firstly, I’ve been saved a crash by a car’s AEB. My point is that current version of AP which is level 2 is very poor. I think Elon said the code will be completely rewritten. Obviously if tesla claims its FSD can do level 3, I will be impressed but I certainly won’t be an early adopter. I’ve been in the AI/Expert systems for 30 years, I know it’s pros and cons

[@networm78]

I agree with the guest. IMO, relying on big data for machine learning is slow and inefficient. Not to say that it is useless, but it should only be used for reinforcement, not for core learning. Humans learn by inference. For example, we know, even before the age of 1, that collision is bad. We experience the various states of matter: solid (objects), liquid (water, rain) and gas (smoke, fog), and the effects of coming into contact with each of them. We know that the faster we move, the less reaction time we have. We know that anything with wheels generally move faster. Humans don't learn through overwhelming examples, but through generalisation of physical rules (i.e. physics) from just a handful of experiences. Literally anyone with zero driving experience, but with decent motor skills could drive a car forward and avoid obstacles in their first try as long as they go at a speed they are comfortable with. This approach doesn't necessarily apply to all domains (e.g. learning to play the game Go), but for cases where we need to interact with the physical world, we are tackling it the wrong way.

[@n4rzul]

I'd argue that most people's knowledge of the physical world and physics, in general, is severely flawed. Just look at the following distances that *most* people adhere to. Humans take massive risks even when they *know* better, they underestimate the real risk and reason - "It won't happen to me". People KNOW that using your phone while driving is a terrible thing to do, but they *still* do it. Once ML and big data understand the risks associated with a particular action, they take the utmost care to avoid it, unlike people. This is why eventually AP will be far, far, faaar superior to human drivers. Humans on a daily basis *completely* disregard what they learned in drivers ed. They are aware that they are *breaking* the rules, but they value convenience and getting there faster more than their own and others' lives. That is immoral and we haven't even touched on drunk driving yet. AP *will* and *should* eventually become law on public roads.

[@networm78]

@@n4rzul What you're pointing out is where humans are deficient, which was not the point I was trying to make. I don't think anyone would disagree that humans make mistakes far more often than machines. What I was trying to say is the ability for humans to learn and adapt with little data far exceeds that of machines. If the machine could learn the way a human learns, then we would still eliminate the human problems you mentioned. The thing is, deep learning is only part of the solution. It needs to complement symbolic AI, not replace it entirely.

[@Andre-ff4hp]

So in driving school, you are told 72km/h is 20m/sec,your reaction time is around 0.4 sec, +more time mechanic do it its job, so when you driving faster watch in your way longer distance bcz in next 10 m you can not do much. If you have opportunity for test try driving faster in the empty road then you push the breaks or pull hand breaks u ll be surprised, distances u paste in start stop reaction +stopping, lwas in joining, did it when (car-school teacher left me alone. And it was (now l m not proud of it) reason I succeeded to run(drive) away from po lice (even l had done nothing wrong) What if self controlled driven car be stopped by the police, would police arrest the car. (added comment)

[@daniel_960_]

Don’t pull the handbrake while driving lmao.

[@listerdave1240]

@@daniel_960_ Why not, it's great fun.

[@Andre-ff4hp]

@@daniel_960_ hy, on the closed road, empty and line is Wright try to, stay in line, or stop at 180 degreas or at wet grass like football field , drive slowly,and have a nice day.. my friend stops his car on the car roof.

[@daniel_960_]

From my understanding vision itself is mostly long solved. It’s not inferior to lidar. This kinda shows it: kzbin.info/www/bejne/m4W8lIB4a76bm5o What’s the far bigger problem is how to react to/ in certain situations. Huge learning and data required for that. But Tesla has it. It has all the data it needs with their fleet. The problem is collecting the useful and necessary data and making the ai learn efficiently. Because solving driving is a huge problem with many edges.

[@n4rzul]

I think Elon's main point is that using computer vision is the most cost-effective and economically viable way to solve the problem. Even if Lidar and other systems are superior, if they cost too much, they simply will never see commercial use and that would be very sad indeed.

[@WaddaFundraiser]

Humans are bad at driving, from ages 16 to 25 insurance premiums are sky-high. One human driver can gain one life-time's worth of driving experience by the end of their life. An autonomous driver starts with many human life-time's worth of driving. The age of 16 is not about maturity of visual systems, children are driving all kinds of vehicles at much younger ages: bikes, go-carts, dirt bikes, etc. Are visual AI systems enough? Yes, when those visual systems are feeding more complex conceptual models, as is the case with Tesla and others. The visual input gets converted into a 3D model containing objects that are tracked moving in time. The visual part is just the input, there is a lot of flexibility with what to do with that input.

[@LUKE390425]

Skateboarders get hit by a lot of cars

[@heltok]

AI will not be able to drive because computer cannot do X. Computer does X. But it will not work because computer cannot do Y. Well computer already does Y. But how about Z? Over and over again... These guys are so used to being the smartest person in the room that they struggle to imagine that someone else, a computer, a team of computer scientist, Teslas autopilot team etc could be smarter than them...

[@NominalTopic]

Take home message: Tesla owners need to drive around skateboarders more often.

[@elon_alerts]

You picked the most central message here I see 🤣

[@SzTz100]

Anyone who has worked in the Machine Learning field knows very well there are edge cases that are intractable. I think Elon is too focused on the business and not the science of this. Time will tell.

[@KaiHenningsen]

Don't forget, humans have their intractable edge cases, too.

[@SzTz100]

@@KaiHenningsen Yes, but whereas humans are still allowed to drive, computers will not be allowed to drive at the same level of accidents. Elon has already stated this and has also talked about the march of 9s. In any case, this all depends on Tesla's AI team and their largest gathering of driving data since cars were invented. I personally don't think full autonomous driving will be here for another 5-10 years but would be happy to be proven wrong.

[@thomasstanley1547]

Hate to say it but this guy might be one of those dummies that Elon mentioned in the 2nd qrtr earnings meeting. I believe he is severely underestimating the power of exponential learning.