Tesla FSD 13 Escapes Narrow Dead End + Camera vs Human Eye Discussion w/ James Douma

[@KrisTC]

3:50 in medical imaging the scanners capture like 10-12bit greyscale. Radiologists are trained to see much more color range. But they still can’t see 12bit so they have a tool called window/level that allows them to adjust the range that they can see so they can differentiate between shades. You could probably do something similar with the raw video footage at that point. Pause it adjust brightness and contrast etc to shore that the passenger is visible. But the car can see all that without adjusting.

[@635574]

Usually if it's greyscale you can tonemap for better contrast, not sure if it's used as well

[@Mrbfgray]

You two are unbeatable in this.

[@edhill8568]

A great new feature of reversing is great when backing out of parking spots. FSD sees behind my car much better than I can when I'm looking back and forth as I back up manually.

[@djsUltra]

I used to think that theres no way the model could be better than a human, because its trained on human driving. Recent videos have started challenging that assumption though...

[@edhill8568]

Backing up in reverse in a complex parking lot is one example where FSD exceeds what a human can do. FSD camera has a very wide view. To see that you have to go to Camera Preview in the Service screen. The regular rear camera view you see in the car does not have the same wide view.

[@lukasbeckers2680]

Take human driving, remove mistakes from the data. Train on it => superhuman performance

[@austinzizzi1142]

It’s training on millions of humans all at once

[@djsUltra]

​@@lukasbeckers2680 Yes, I think you're right. But I think it's less obvious how that works when it comes to situations like slowing down in anticipation that someone might open a car door even when there are no human perceivable visual queues. For the slowdown to be trained into the model, the absence of the intervention scenario is not enough. There would need to be some scenarios in the dataset where the human just happened to be driving slow before a person opened a door, even if the human didn't actually perceive it. Though there is certainly more at play here than just this, even the prospect that this kind of evolution is happening is incredibly interesting to me as it mimics our own evolution in a way.

[@azureactivedirectory]

ChatGPT is arguably smarter than a lot of humans being trained on human data, so is AlphaGo/Zero.

[@wizard-469]

I hope AI5 comes with a front bumper camera. It would be as confident as when reversing close to obstacles.

[@KendallHall]

Plus it needs a second camera pointed slightly differently in the windshield or on the hood so that it doesn't get blinded by the sun. Mine gets blinded so often it's ridiculous

[@EdwardMillen]

@@KendallHall is yours HW3 or 4?

[@thesolarfutureenthusiast1102]

We all know there are at least 50 shades of grey.

[@freddydad1]

but you can only see 36 of them 😂😂

[@catbert7]

@@freddydad1 Thank god. Imagine if we could detect more details in that book 🙈

[@formytots0128]

Thanks AIDRVR for this great video. Everyone should a James Douma in their car as a guest when testing FSD. Instant teaching moments as unique scenarios and events unfold during the drive. Thanks JD and James. :)

[@jooptablet1727]

People don't understand how much more detail can be gleaned from a digital cmos capture as opposed to the human eye. And then 36 times a second or whatever the refresh rate is... It is already superhuman. It just needs to learn more. And it is...

[@VoxelLoop]

People talk about how humans have an 'infinite refresh rate', but... We don't process that data 32 times per second! Considering an 'average' human reaction time is around 250ms, that'd be 4 frames per second in terms of vision-to-control for a human. (Nevermind the fact that the car is looking in all directions not focused on a specific one)

[@qqleq]

No. The human eye has a much better contrast range than a camera. Video even has a way worse contrast range - even the modern camera's - than classic film. The guy in the car says a lot of sensible things about it, because indeed, it is very complex, but in general, the human eye does better. However, the human mind is not processing everything and our attention is automatically directed to certain parts of what we see. And these parts are not the dark parts inside a car we think is just parked. I'd say that is the bigger advantage of the camera/computer: having attention on everything.

[@thesolarfutureenthusiast1102]

@@VoxelLoop Assuming an 'average' human reaction time is around 250ms we could still react more than twice in 500ms as our frame rate is greater than 4fps. You can start your next reaction before your first reaction has been executed.

[@jooptablet1727]

​@@VoxelLoop"we don't see with out eyes, we see with our brains"

[@635574]

Also it has many caeras and bottlenecked by evolutionary compromises like blindspot, lag, compression or your brains processing.

[@jonathanmelhuish4530]

I know you're probably spending days editing this perfectly but personally I would pay real money to watch the complete, uncut conversation with James. Unlike most Tesla commentators, his AI knowledge runs deep, so I soak up every word. 🤩

[@ProXcaliber]

What this has shown me is that Tesla should have included a front bumper camera on all models. It would just improve its confidence in going forward closer to objects and help in detecting things below the hood line. I don't see any downsides to adding a front bumper camera, with the exception of added cost. I hope they eventually do this and allow for retro fits to the models that don't have it.

[@alesksander]

But that skews all learned info about old trained model till now, because that's quite big change. Is even needed, thats the big question.

[@catbert7]

They should have added at least two cameras to the front bumper, for seeing around blind corners. Massively foolish that they did not, as the cost would be minimal and the safety increase would easily outweigh it.

[@ProXcaliber]

@@catbert7 I disagree on that to be honest. I feel the same thing could be accomplished with a wide angle single front bumper camera. The car already seems to do very well on blind corners.

[@nicollo3672]

They would just get dirty all the time. The current front cams are behind the windshield which can be wiped.

[@nowsc]

… Thanks for the video with James. I wonder if we could get James to talk some more about this business with the dynamic range of the cameras - - I’d be interested to see some more detail on that, especially what he said about what happens when there’s a strong light from the front, light from the sun.

[@EdwardMillen]

Yeah that would be interesting. From what he said in the video, it sounds to me like the actual sensor pixels in automotive cameras must work more independently from eachother than they do in normal cameras. I know HW3 can have issues being "blinded" by the sun in some way and HW4 is apparently better on that, but I think that's more of a lens flare sort of issue which they've much improved in HW4 (based on the images I've seen from both)

[@quedokid]

The Tesla view is 360 degrees with full capability of interpreting the 360 view to whatever it is capable of comprehending. Humans have peripheral vision which is nearly 180 degrees, but only comprehend what is directly in front, not comprehending most of the 180 view present. Big win for 360 view cameras, and the computers unbiased comprehension of the entire 360 view.

[@KendallHall]

With v13.2 I've found myself hitting the juice to coax it along less often, although with the new unpark and start FSD from park features, it needs me to hit the juice pedal sometimes to get it going and feeling good about what's in front of it, especially on a sloped driveway.

[@marcuss_aureliuss]

They’re yappin about how the camera could see a person in the car but didn’t the Tesla just slow because the hazards were on in the other car?

[@bratr]

That's what seemed more likely to me as well.

[@EdwardMillen]

Yeah I think that's entirely possible too (and perhaps more likely). Can't really tell either way in this case (need more examples of what it does in this situation, and what it does in the same situation minus the person in the car)

[@EdwardMillen]

2:44 Yeah, I've noticed that my Model Y can sometimes see people better/from further away than I can (e.g. if they're dressed fully in black, at night). And that's on HW3, in the UK without actual FSD!

[@JorgePacker]

It would be nice if you could do a "10 best FSD manoeuvres" video, or something like that! It would be a cool video with your commentary to show to friends and family! Cheers

[@RubenKelevra]

3:00 my eyes can only see 32 shades of grey? Wtf? I've generated a neutral grey step latter and I can see 20,000 shades of grey easily. Not even counting color tints of grey, just contrast.

[@Mark-kt5mh]

That's just a gradient (context) illusion. If someone took two neighboring shades out of 20,000 and asked you to identify the darker one, you would not be able to do it

[@Eyex97]

Because what they are saying is not true.

[@random_n]

@@Eyex97 It is true, but it's lacking nuance and context. The human eye has much less sensitivity to grey shades than to colour, but the contrast in immediately adjacent shades allows you to pick lighter vs darker easily. However, if you have to pick two matching non-adjacent shades of grey from a 32x32 grid of assorted shades, you'll find it's very difficult. The human visual system is chocked full of tricks that mask weaknesses or help us perceive objects and changes, but it does not necessarily deliver the truth. The "checker shadow illusion" is a simple example. You'll see that your eyes can lie, but also find that the lie is more helpful to understanding the scene correctly.

[@Eyex97]

@ Yes, definitely, you are right, the topic is very nuanced. I just don’t think it is a very smart move to automatically attribute the intelligence of a self driving car to the superiority of camera perception and the weaknesses of human eyesight. If we follow this philosophy, we can loose our desire to learn from nature and biology, and I think these ADAS systems could benefit a lot from applying the mechanisms of human perception in computer vision.

[@catbert7]

@@Eyex97 Your premise that anyone in the conversation implied natural systems to be inferior is false. They were discussing differences in capability between the two and explicitly acknowledged that human eyes are superior in some ways and inferior in others, which is true. This does not imply that they do not think there is anything to be gained from studying biological sensory processing.

[@kurthui]

I would thought based on its 360 image, it can drive backward as good as forward. so it can just backing out that deadend.

[@Resist4]

Tesla should do a video showing us what the cameras actually see. I’m curious about the contrast data.

[@catbert7]

Given that we can only see with our eyes, that is technically impossible. Best we can do is second-order analogies to what the cameras see.

[@Resist4]

@ not true, they could show what the cameras see.

[@TheGreatestJuJu]

I can see 50 Shades of Grey *too easy, but it had to be said*

[@Cuddlebear316]

0:06 when the guy laughs and says "it's almost like, NoOOOooOoo..." it sounds like Chris from Mr Beast

[@timwildauer5063]

More drives with James please

[@ldsmike88]

I respectfully disagree. What was the whole point of this video?

[@timwildauer5063]

@ do you seriously not understand why the commentary is so valuable?

[@ldsmike88]

@@timwildauer5063 yep. Not trying to start a fight. I just enjoy his regular videos better. I'm a software engineer and hobby photographer. I've written software that processes photos to gather data that humans regularly miss. I find his regular videos more entertaining.

[@timwildauer5063]

@ that’s why this is on the clips channel, completely separate from the main videos.

[@catoftruth1044]

do more narrow streets with lots of pedestrian and motors passing by.

[@zuplex8933]

One handicap of the human eye/ brain is the selective perception

[@catbert7]

It's actually a strength, to cover other weaknesses - namely that we lack the processing to pay attention to everything.

[@catbert7]

Yeah, it's funny how oversimplified the conversation on cameras vs eyes is for most people. It's not one being better than the other. Another area the cameras excel in is fog. Fog that I can't see 5 feet in appears almost clear on the camera footage, which is a huge advantage in some places.

[@watermelon1221]

saying the human eye can only see 30 shades of gray is bullshit

[@18837dc]

Can you prove it otherwise?

[@watermelon1221]

@ I can't provide links, youtube takes them down, google it... The burden of proof is on the one making the claim, not calling it out as bs, so you prove it by providing me a link that doesn't get taken down.

[@MarkCupLee]

I don't think the model is taking more then 8bit of the video as input.

[@HalfInt]

I wonder, how much FSD can already infer from the cars' and pedestians' movement. I am not so good with facial expressions, but quite with people's movements. Works with humans and also their cars. For example, I usually somehow see when people want to take a turn, long before they reach the intersection or use their turn signals. Might be that they move over sightly, or how they accelerate.

[@ebbyjones3177]

this IS the future. This is amazing. Thanks Elon!

[@bmet001]

low audio

[@happycamperinc.]

Fsd V13 Loves 3 or more point turns.😂

[@TeslaElonSpaceXFan]

😍

[@card_craft]

Your really do torture that thing huh!? It didn't want to!

[@happycamperinc.]

Baby AGI Achieved.

[@aguyfromnothere]

I have noticed it cannot back out of a drive way with curves. With FSD 13.2 I have basically had no interventions once it gets going. Speed is still wonky and its not smart on freeways in hurry mode (it goes to left lane as freeways diverge or come together which is slowest location not fastest).

[@alesksander]

hmm maybe lack of trained data on that area. Afterall i guess u need start somewhere heh.

[@patricksquires77]

& Just for your comfort cause it could have just backed all the way out - who cares other than what feels/looks normal.

[@esam-long2]

WOW

[@leesanghong80]

Dl21b