The Truth Behind Why Tesla Removed Radar w/ James Douma

[@IraqemUp]

James Douma is an incredible asset to the Tesla community.

[@mac5658]

Wow Dave! This discussion really cuts to the heart of the matter. Manhole cover statement was presented simply and clearly for mass understanding. Must watch episode. I bet this becomes an often repeated explanation regarding the radar removal. Thanks for being a conduit for access to all of these great minds. (Yourself included)

[@DrewS777]

Your interviews took your channel to a higher level. Especially Interviews like this.

[@cbmusgrave]

This together with Karpathy's talk on removing radar are extremely valuable. Thanks Dave for bringing James on again!

[@jamesenglish9736]

It's always great to hear James talk with his deep understanding and explanations. Thanks!

[@jiaweishi7226]

Love Douma, can't have enough Douma...

[@dogoargentino7834]

Thanks Dave for sharing these conversations it’s really helpful.

[@hanweiz84]

Please continue having James on the channel. He makes complicated ideas easy to understand

[@LiguoKong]

Can’t wait for the other episodes, Dave. Thank you for the great conversation. James is awesome as always

[@romanvasyliv9216]

Finally, James is back !

[@johnmcbirnie3203]

Breaking it into multiple episodes is a great idea. A 30 minute episode is very digestible. A 90 minute episode less so. James can get into the weeds because he knows so much. I really enjoyed this vid.

[@endeuinable]

These episodes with James are super valuable and educational and should be on every single Tesla investor "to watch" list. I think Dave's intervieweing style matches perfectly with James's educational and technically rigurous explanations. Really looking forward to the rest of episodes of this series!

[@guslevy3506]

Every restaurant owner on Earth has figured this problem out at least a thousand years ago. Too many cooks in the kitchen counterintuitively slows down the process and efficiency.

[@CampTeslaFun]

These episodes are very helpful. Thanks for the careful & complete explanations, Dave & James!

[@beck99o]

I love the point about Tesla realising that Radar is hurting them and is barking up the wrong tree. This realisation has come through having million+ cars on the road testing the system, collecting examples, training the system. Other companies may take years to get to this point and possibly come to the same realisation. No matter how good you get Radar+Vision working, there will always be scenarios where they disagree and the safest option would need to be taken. This leads to 'phantom' breaking where the user is left confused/annoyed unless of course it saved their lives. The fact that Tesla-vision can replicate the job of Radar is a breakthrough and means they do not have to work around that problem anymore. Of course this all depends on their solution working as expected, but with their expertise I certainly wont be betting against them.

[@tedshep]

I see James Douma, I click play.

[@lifuxiao]

Thanks Dave and James! For James, every glimpse on the ceiling is a great answer!

[@fatcat220]

Awesome video! I watched AK's talk, but this episode is a great companion to the talk. Definitely will watch both again.

[@mlee1308]

How does vision work in foggy or heavy raining situations???

[@ParaSpite]

How does it work for human drivers?

[@antonpershin998]

IR

[@ken-mb5cp]

Douma is great!

[@cybergigafactory]

I like the long videos more but hopefully in these snippets there will be more information as a whole. Thanks Dave for giving us such great insight with James. I love every interview from you two. Can’t get to nerdy 😅🥸

[@bones23jones]

KZbin should allow us to like videos twice. 👍👍

[@monosplatanos]

Great stuff, have been eagerly awaiting this series of discussions with James since you first mentioned it Dave. Thank you both!

[@Raysliquidity]

Thanks for this discussion Dave & James! James you really are able to take very complex topics and explain them in an understandable way - a true sign of intelligence. Really looking forward to the rest of the series

[@JoeVirella]

Was waiting for this as soon as I saw the Karpathy talk! Thank you so much!!

[@Alvin-my6wj]

Great discussion. Sensor vs noise problem is complex and why Waymo and others are geofenced. Tesla vision with distance map makes a better path.

[@joekkl]

Vision is so good, it can replace radar. That‘s the golden nugget of the story. Thanks Dave and James. This is an awesome episode!

[@Sandwich137]

Lol ya go try and get a high end camera to balance the lighting under a bridge and the 30 ft after it. Cars need both radar and cameras for balance

[@billdvorak]

Are the sonar sensors still there. I never see them mentioned as still in or out.

[@antonpershin998]

@@billdvorak sonar is slow and low range.

[@mrnegative48]

good to have James back

[@retiredbitjuggler3471]

Once again, kudos to the Dave and James tag team! Your give and take banters are always very entertaining and full of insights. I find myself rewatching these episodes because I gather so much knowledge pertaining to the mystical realm of FSD. I reserved a Cybertruck with FSD quite a while ago and when I did I knew nothing and had my doubts about FSD ever being fully attainable. However, sessions like these have opened my eyes into a world that I now believe will be truly disruptive and attainable within a reasonable timeframe. I just want to thank both you and James for the great discussions and the effort you put into them. They are very much appreciated!

[@mhfs61]

Very clear explanation and educational, James. Thank you 👍🏼👍🏼👍🏼

[@Turismo2929]

Shout out to James, even before watching I gotta thank you for your insights. Making some popcorn 🍿 and checking out this show now.

[@fjdkfjdk]

As someone that did a bunch of vision ml, I agree. The logic for keeping radar makes no sense right now. In ML, data quality is way more important than quantity. So, increasing quantity at the cost of quality is absolutely terrible. Fundamentally, in order for radar to help, we must increase the quality. This was being done by cross checking against the vision system. But the whole point of radar is to help you in situations where vision doesnt work. If people disagree with the radar removal, imo, they must suggest a way to increase radars data quality without help from the vision systems. It might be possible to use radar effectively, but having a backup system that depends on your primary system is just a bad idea.

[@entspannter_hase]

James Douma is my favorite guest on any KZbinr's channel. Great video. Dave, can you up the volume or compress the audio signal more? The video seemed really quiet to me.

[@HubertHeller]

Had to look up CIPV: "Closest In-Path Vehicle"

[@adventurenature6456]

Welcome James!!!

[@tlow5766]

Fun fact: the reflection issues mentioned by James do not only occur on radar. They also occur on lidar.

[@nickfosterxx]

I get it, vision only. But I'm still puzzled by the camera locations. To detect range (parallax) changes wouldn't it be better to have the widest possible stereoscopy, ie a camera at each side of the car looking forwards? And for right/left detection at obscured T-junctions, have them as far forward as possible - ie within the headlight cluster? (And include a mist-and-wipe 'blink function' for bug and mud splats, snow build-up etc.)

[@ronmc3]

Intuitively, that would seem to be the case. But for humans and other hunter animals, our eyes are only fairly close together. If stereoscopy was super important, we'd have evolved eyes farther apart. Prey animals have eyes on the sides of their heads; for them, all round vision is more important than stereoscopy. So stereoscopic vision is probably only useful for short-range, static depth perception. Long-range depth and approach speed of moving objects is detected by changes in a single image: detecting size changes as an object moves directly towards or away, and observing how an object moves across a stationary background. Most people have a dominant eye, and people can drive satisfactorily even when they have only one functioning eye.

[@nickfosterxx]

@@ronmc3 Makes sense. Thanks for going to the trouble of a detailed answer Ron.

[@NYProfessional]

Would it possible to get someone from Mobileye to comment on radar / no-radar?

[@z4zuse]

Is it possible companies like mobileye or waymo have a better radar stack?

[@simoc24]

If vision and radar disagree, what will the car do?

[@xumba007]

They have many cameras but not smart enough.

[@mjr7991]

Dave…great vid…more please👍😀

[@bobbiehoward780]

This helped my meat computer to understand the problems with radar. The photographs help more than the graphs.

[@okanaganteslaguy1426]

Thanks for the clarification on fantom braking! Good by radar! Very soon FSD BETA 9 ! 💎🙌😎

[@danmack145]

This might be a stupid question…but are the cameras inside the cab of the car? Or what keeps the cameras from being covered with dirt, snow, rain?

[@jpmcsweeney7156]

Dave, great very informative, I was able to follow the dialogue with understanding, thanks.

[@sanjubajaj229]

Taking the radar off is going to be like removing the training wheels from a kids bike.

[@pwells2389]

‪Was another great collaborative piece. Thanks Dave and James. I’ve heard mutterings that newer, far more precise radar is available and would mitigate many of the problems Andrej and James refer to. Does this have any credence?‬

[@flarierza33]

Amazing discussion. Thank you!

[@hadtobe4502]

Still not convinced taking radar away. Can camera only systems see through heavy rain, thick fog, dense smoke, bright oncoming vehicle lights at night, bright sun during the day.

[@chadwickwood9843]

Same. Maybe don't fuse them anymore but have a poor weather mode where TACC can still function with radar and stop using vision

[@dlwatib]

CIPV = Closest In Path Vehicle.

[@rolandkertesz4196]

Thank you!

[@Sandwich137]

Short answer to ship more units without delay

[@Bobswatercooler]

Great video! It looks like Tesla has removed a HUGE failure mode of their system. A big step in the right direction!

[@mallow610]

take a shot every time they say "right, got it, or interesting"

[@cyrusadamrevilla3851]

Been waiting for this! Lets gooooo!!!

[@BFArch0n]

It's funny that Tesla always said, humans can drive with no lidar, only cameras, so cars can too...but they still had radar in their system....not just cameras.

[@scanspeak00]

Why no mention of the biggest safety issue of vision based systems? - its poor performance in bad weather.

[@alanmay7929]

Yeah these people are just focused on money and even forget the chip shortage is part of the problem of no radar.

[@scanspeak00]

@@alanmay7929 Prediction - lots of accidents when Winter arrives in the US.

[@mukamuka0]

Really great explanation! thanks for bringing this up man

[@GautamAtri]

Good Morning 👍🏻

[@thelongame]

Can we make everyone’s Capcha selection data feed into and train the NN to improve labelling and edge-case detection?

[@JohnSmith-lx1xe]

Great Interview

[@integra8502]

Phantom braking is still a problem without the radar !

[@ibnorml]

Concerning people's trust in radar, that mostly comes from movies about the military. The difference here (ignoring the inaccuracies in the movies) is that trained pilots are a hell of a lot smarter than any AI at the moment, plus believe it or not the radar return from an aircraft at 30,000 feet is a much simpler signal to interpret than a mass of ground clutter from a radar in a car.

[@rogerfroud300]

@23:57 - I guess tall commercial vehicles could have their height characterised in the system to prevent them from driving into low bridges, something that happens a lot in the UK.

[@antonpershin998]

Just train NN to recognise signs, problem solved.

[@christopher_rubicam]

Now I have a much better understanding of my Model 3 and all the annoying phantom braking on city streets. That said, I use it all the time and am fully satisfied with FSD purchase. Lane keeping and start/stop/follow is so much better than manual control.

[@woolfel]

people are missing an important thing. Tesla needed radar in cars to get to this point. The radar and sonar data is used to train the vision system for distance and velocity model. Those distance annotations are used for off-line training in the datacenter. If the new 4D radar pans out, it should be a huge improvement and filter out most of the noise. Vision isn't full proof either. If a seagull poops on your windshield, you're screwed until you turn on the wipers. There's always going to be a need for multiple sensors. The old radar and sonar have major limitations that are well documented. The newer generation of radar and lidar are working to fix that.

[@OrbitTheSun]

If you are following a car and it suddenly brakes, radar detects this immediately, even in poor lighting conditions. - Brake lights are unnecessary for the detection.

[@frankfortino1405]

I have a question. How do other cars that use RADAR, deal with these RADAR errors?

[@sc0Obs]

He mentioned Volvo would set lower limits on speed to avoid phantom breaking.

[@QuietJugung]

Do other cars using radar have the same problem with stationary objects?

[@chrisheath2637]

Really good explanations.

[@W020-j9o]

A question for James about sensor physics of different sensing methods for a future discussion: If we assume that any FSD method will have recall vs. precision tradeoffs, how do different sensing options (Lidar, radar, vision/other) compare? Lidar (time of flight) vs radar (doppler/signal strength), vision. (pixel-to-pixel resolution of a camera-based image. Which one is best, based only on the signal to noise ratio of the detection method? Which one looks best for the future?

[@xumba007]

Don’t think this vid or channel is for this kind of discussion. Reddit is better place.

[@W020-j9o]

@@xumba007 Comment accepted, with a qualification. I wanted to address James directly because of his depth of understanding of the whole system. If I understand him correctly he is saying that they were trying to control something with an input from radar that is not reliably measurable. I would like to hear James's thoughts on how vision compares in the same way. This is a first principle approach.

[@pauljoyce32]

This is a great video. I'm interested in "Tesla Vision". This is by far the best "Idiot Guide". James simplifies the subject matter well. But, like only an expert in the field can, provides more information on the issues involved.

[@bpg786]

FortNine did an excellent video on why removing these sensors is a terrible idea and likley resulted in several deaths to other road users.

[@geemy9675]

if radar is completely incapable of detecting any stationary object from the stationary object noise, how could the radar be confident saying there is no stationary object, and why would the AI trust it over the vision detecting a truck. it seems like an oversimplification

[@nobodyimportant_23]

What about lidar? Would it have the same issues as radar or it's different

[@simoc24]

They did a whole series of 6 or 7 videos on Lidar several months ago :)

[@arvijeffery6859]

Lidar would see the fire truck easily. Also, lidar would see the white truck, against the white sky background, crossing the road left to right (or vice versa) very easily. The only negative thing I've heard Andrej say about lidar is that it is expensive. Yes, it is. Also, mean time between failure is probably not a good number, because it is a delicate optical instrument with moving parts. However, Tesla people, from top to bottom, should openly admit (if they agree) that lidar provides useful info in challenging edge cases. Also, LWIR sensors are becoming cheap and reliable. They can also do great things, particularly when there are obscurants in the air.

[@chrisheath2637]

Active sensors (like LiDAR, and radar) do have additional problems over passive sensors like vision. An active sensor sends out signals, and analyses the reflection from the object. Reflections can be weak, and , as James explained, you are trying to spot the important things amongst a mass of unimportant things - (signal to noise). Also, other cars may be transmitting their own signals, and you have to differentiate your reflected weak signal, from their stronger transmitted signal. That needs more and more processing as more cars use the system - (over time the situation gets worse). As has been pointed out, human vision is incredibly acute when it's focussed, and quite adequate when it comes to driving. With faster processing, and multiple cameras, it's easy to imagine (in this case) Tesla vision easily outperforming a human in virtually all conditions. There would be some extremely rare (non-driving) cases, like a car-jacking, or a riot, where a human could take action, based on being a human, and a FSD computer could freeze ( ie the human beats the FSD computer because of psychology). But I suppose even those actions could be learnt...if the neural network is trained...

[@jiraphat2200]

10:00 Dunning-Kruger effect of radar sensors

[@jamespeterson7100]

How come all of the other car companies that use radar arent having as many problems as Tesla?

[@archigoel]

Because their & Tesla's goals are very different. Other cars want to give good AEB...Tesla wants FSD. So for Tesla, it has to fix all the edge cases where radar fails.

[@fillthedao]

Simple: no other company has so much data sample to find so many edge cases, nor they have enough vehicles deployed to collect new cases, and none is so advanced in AI to make any difference - this just shows you how far tesla is n FSD. Consider only viable "competition" (ugh that's such a stretch) nowadays - Waymo. Not vertically integrated (cant rely on own subcomponents like own custom made chips or super-specialised sensors... damn they don't even produce their own cars oops), limited area of tests (geofenced area of operation), relies on radar, lidar and vision in hq mapped envs (tight control of the area, avoids dodgy locations with tough roads)... its like comparing a calculator to an general intelligence. And that's the closest player to be consider somehow a competition... the rest just plays with this concepts in the sandpit.

[@fillthedao]

@@archigoel yeah excellent point, for other companies that's also just a marketing feature - like better speakers or heated seats. For Tesla FSD is the goal, not an addition.

[@alp5725]

You are just asking a baseless question. In reality most of the other companies have major problems(with the exception of Subaru that only uses cameras). Problems like AEB not working consistently or at all in some cases, or like James pointed out with Volvo where the system only attempts to work over a certain speed.....please do some research.

[@fillthedao]

​@@alp5725 yep its for other manufacturers to "have no problems" if they are doing nothing ^^

[@jorgesmith2000]

Question: would the problem of integrating radar be the same problem of integrating lidar with vision?

[@dlwatib]

The failure modes are different between lidar and radar, so no, not the "same problem". But lidar does have problems, primarily it quickly degrades in bad weather and it's not as data rich as a camera in the first place. So snow, rain, fog, etc. turn the data signal into noisy mush that can't be used.

[@rolandkertesz4196]

@@dlwatib Also a hellla lot more money (and ugly imo).

[@webspiderc]

How about the situations like foggy, heavy rain and snow ? Does vision based system handle those situation well ?

[@patricklegault6383]

are you able to drive in theses conditions? if so it should be able to

[@alanmay7929]

@@patricklegault6383 radar works way better in those conditions

[@casperhansen826]

@@alanmay7929 you are not listening to the experts, radar is not working better in any condition

[@vhoy7899]

Hi James. I missed you. Hello Dave. Now let me hear what you have to teach

[@wolfgangpreier9160]

I can confirm those problems. I have a new Enyaq, VW MEB Platform, runs just fine. The automatic cruise control works "better" the Teslas. Much more smooth and mor determined, breaks earlier and hits the gas much earlier. Does not keep too large a distinace to the cars in front. But it has one major MAJOR flaw. I had 2 occurences that it did not break at all behind a smaller lorry. Not even trying to get slower, instead it showed 130 km/h inside the city and wanted to accelerate into the lorry. VERY dangerous. I predict that VW/Skoda/Audi have the same problem descerning between radar and visual as everybody else. Some work better some not so much. Lane assist btw is easier then Teslas - it always works as long as you don't hit the brakes, and at the same time much worse - when it does NOT work you do not know that it stopped working, it does not show any different sometimes it shows it has stopped, sometimes not even that and no audible alarm whatsoever. But it steers you into the barriers, oncoming traffic etc. pp.

[@rolandkertesz4196]

What do you mean by "Lane assist btw is easier then Teslas"? I am curious.

[@wolfgangpreier9160]

@@rolandkertesz4196 When i activate it the car does not pull the steering wheel out of my hands to find to correct position in the lane. BTW the precision of the Tesla AP is much MUCH better. But after activating it just gently nudges the steering wheel. Even in tight corners which Tesla does not allow to steer on its own. Apparently because of a wrong interpretation of some obscure UN/ECE regulation. When leaving the lane you do not recognize anything. No strong hold back no sound, no flashing lights or other type of information. It just lets you do it. In the new lane it automatically takes over without further ado. Gently as before. Easier then Tesla’s not as accurate and much more dangerous because it never tells you when it stops steering. You are just making contact with the next car, barrier, house, lorry…

[@trent_carter]

I can confirm, by experience, that The VW groups tech package enabled lane assist will kill you. It ping pongs from left to right on a regular basis then tries to take your life. VW’s lane assist is a four year old sitting in your lap steering the car. They’re doing their best but eventually they will kill you. Tesla’s current system using hardware 2.5, is a 16-year-old driving a car. Yes they will eventually crash as all 16-year-olds do. Besides this difference; Tesla’s system is updated frequently and the 16-year-old will eventually turn 17 and 18 and 21 and 25 then 30.

[@QsViewsFinance]

Makes you question the future of LiDar companies if Tesla begins to dominate roads....

[@fillthedao]

if you consider how e.g. Waymo's LiDar is wired to the high-res pre-made scans of the area and piggy back to get lock on location to help them navigate its kinda no-brainer that this approach is a dead-end. Its a double-whamy for them - core "advantage" that not only costs more but is unscallable and cant resolve all ege cases... nasty ^^

[@importon]

@@fillthedao who said anything about "lidar only"?

[@stargazertress777]

We think cameras are better than sensors because sensors only sense but cameras see.

[@patricklegault6383]

and we have 2 camera working daily for years slowly degrading (eyes) we drive and i certainly don't have a lidar or radar in my pocket to help me steer the car :p

[@Moonwalker088]

Gracias Dave!!

[@danielhull9079]

Radars, Cameras or Lidar, etc. All need a high level ASIL platform and Architecture. With hacks available we get self driving lawn mowers at the best for the masses.

[@Steph1]

TL;DW radar is a bad thing to have from the moment your vision is sufficiently awesome

[@philliptemple4534]

Tldr; you can't have different sensors creating different world models and treat them all as equal ("fusion"). Until radar can be a reliable redundancy check for vision, it has to go bye bye until it can. Phillip.

[@reignstalk4376]

Wow what a surprise

[@lvrt8371]

He is just saying that the radar is not good enough, they should change product. The rest I don’t see a problem

[@slmille4]

These examples are rather cherry picked, since they all are on well lit roads where vision does well. What about in dark/snow/fog/rain conditions where radar has the advantage over vision…

[@denes123s]

One example was in dark..

[@slmille4]

@@denes123s nope

[@denes123s]

@@slmille4 19:10 you’re welcome

[@slmille4]

@@denes123s Nope, still plenty of ambient light and street lights.

[@mac5658]

Do you envision radar or lidar identifying a stop sign vs a speed limit sign vs a trash can (for example) in the dark/snow/fog/rain? It is not capable of doing this task unless used in a preplotted, static environment. I would require vision processing to exceed human capabilities but there will always be a restriction to it’s capabilities

[@ribe3434]

Great video helping understand the amazing FSD tech.

[@andyonions7864]

Consider if your olfactory nerves were wired into your visual cortex. How does visualizing smell help any? Edit: Lidar is a greater anchor than radar. Only it costs a hell of a lot more.

[@waynerussell6401]

Synesthesia is a ~1:100 powerful biological adaption associated with abilities in art. music , maths, memory...

[@andyonions7864]

@@waynerussell6401 I'm not so sure it's useful in cars though.

[@sirousmohseni4]

James understands AI and sensor technology very well and can communicate it with interested people. Right guest for this topic.

[@langforc]

I’m not convinced. Cameras alone cannot deal well with high contrast situations like in and out of tunnels, under dark bridges etc. Forward facing lidar can deal with this regardless of light. I believe Xpeng is the only manufacturer with built in forward facing lidar.

[@importon]

Dave is cheerleader. He will only come around to your point of view if tesla recognizes it first. you must be new here.

[@stevenhill3136]

If Tesla ends up removing radar can they turn it off remotely or will it have to be removed manually?

[@mulindejohn]

Even if its not turned on. The computer can ignore the radar input. But am pretty sure the car can cut the power to it through OTA by tesla.. remember Even the mere seats are connected to the computer.. they have alot of control on all the components in the car.

[@wildernesscapes]

I view this as a bullish thing, that they've developed neural nets advanced enough to solely rely on vision. Do other carmakers use Neural Nets as well? I'm also confused by the difference between Dojo and neural nets.

[@TechScottBrown]

Honda recently announced that they’re ditching Radar as well! hondanews.com/en-US/honda-automobiles/releases/all-new-11th-generation-civic-sedan-fully-revealed-in-production-form-with-sporty-design-advanced-technology-cutting-edge-safety-features

[@alanmay7929]

Simple answer: chip shortage you're welcome.

[@simoc24]

Karpathy says vision is 100x better than radar, what does he mean by it? 100x more accurate? 100x more easy to use? 100x less confusion? That part I need some explanation please and thank you.

[@mjr7991]

Do other car Mfg have same issue with radar? Seems like you don’t hear about that.

[@brunosmith6925]

Other car MFG's all rely on tech from NON-car makers. No other car company is working on self-driving. Some are invested in self driving development companies, but the only thing other car makers know about radar is how to spell the word "radar".

[@rolandkertesz4196]

@@brunosmith6925 Yes, that's why I am so bullish on Tesla. Their cars never ever could drive themselves, tesla has the hardware to solve autonomy.

[@TechScottBrown]

Honda recently announced that the 22 Civic will be using cameras and removing radar from the suite hondanews.com/en-US/honda-automobiles/releases/all-new-11th-generation-civic-sedan-fully-revealed-in-production-form-with-sporty-design-advanced-technology-cutting-edge-safety-features

[@Da-King-of-Swing]

James voice and speech pattern sounds alot like dat of Edward snowden

[@kvsingh007]

james douma you are great... Explaining things in deep as a engineer. Thanks Dave for the interview

[@joonbeomcho]

We want TESLA lawn mower!!!

[@LewdCustomer]

I'll need a Tesla handyman robot too. Doesn't need a face, just work as expected.