Teledyne Flir: Thermal Camera Systems

[@joshgray1331]

He is a great sales / engineering representative. VERY hard to find a person that can handle both. Well done!

[@LJ-jq8og]

AWESOME Tech 💪

[@BenBrand]

"Cars don't seem to see motorcycles" a few seconds later, "I used to ride on the dotted line and that"

[@stewartstarling2309]

Sandy your guest is extremely competent and hit on some excellent points. Adas systems in AEB and pedestrian and cyclists detection , and animal detection small and large, this is very exciting I’m looking forward to the future of driving. #nerd.

[@DerKatzeSonne]

18:21 - the image basically blends between standard RGB camera images and thermal images / FLIR. It's simply a marketing video showing what the difference is. And Sandy goes excited "THAT IS BRILLIANT" on the standard RGB image and when the thermal image comes back he's like "no, not really that one" lol. Or mabye that's really some morphed video we couldn't see from this angle but I feel like I'm right because the salesmen was weirdly silent at that point.

[@christophermcgrath8847]

Its a patented technology that FLIR has called MSX, basically blends data from both thermal and visible to enhance with detail like lettering etc. that you may not always pick up on a thermal camera.

[@mj-pg3uj]

Seeing the developing sophisticated technology and its application which will result in safer transportation is awesome. TY for being here on the KZbin.

[@yrj77]

I don't know how I missed this video but I'm glad it popped in my feed today. I would hope some auto manufacturer would jump on this technology.

[@aussie2uGA]

The real question all car guys want to know: can the flir system spot a highway patrol car's idling engine hiding behind a tree?

[@eubikedude]

Not too well if it's a Tesla patrol car probably ... although it might see the occupant/their heater where the windows are.

[@markplott4820]

Aussie - a ELECTRIC INTERCEPTOR gives off NO HEAT when IDILING. and POLICE can use ACTIVE COOLING to FOOL FLIR Sensors. there are ALSO Meta Materiesls that BLOCK heat signatures.

[@markplott4820]

@@jeffalvich9434 - TELL that to a MAIN BATTLE Tank covered in Meta Materiels , will look INVISIABLE to a FLIR system , while the MBT sets its Anti Armor Missile in your sights.

[@craigduncan4826]

Yes and it can see through women’s clothing as well

[@sjwright2]

Literally every reply above is made-up bullshit. FLIR cameras are readily available and anyone can see for themselves what they're capable of.

[@demonchicken]

"you can see someone who is a hazard to themselves" This guy is using all the marketing approved terms :p

[@ryanthompson8256]

Greetings from the U.s please i know this information might not be for everyone but if you are experiencing loss in your trade /invest i advice you work with Helen Howard pratea on google she is an expert advisor

[@kulaengineering]

Great stuff. Thank you for that interview/presentation.

[@NTNLabs]

The cam&flir holder on top of the vehicle is 3d printed, neat! :)

[@RayNLA]

The driving portion reminded me of the TV sitcoms when they left the stage set and drove somewhere

[@nodave77]

I would love to be able to buy a thermal system that would display on the radio/heads up or even a mounted screen.

[@Wavefront101]

Flir already makes them, pathfindir. $2.5k though which is quite expensive for a 320x240 30hz imager, although a lot of cost for heated front window, and waterproof casing. There are some cheaper chinese ones around half that price.

[@deeplato5647]

*Very Cool* John Eggert presents as one of the most intelligent, balanced and thoughtful guests that have visited ML

[@timmylassie6763]

i am a retired aerospace engineer and have spent a LOT of time working with FLIR and combining FLIR and RGB video for use on unmanned vehicles for search and rescue. the application he discusses is VERY similar to my environment. We are LONG overdue for a technical advance in the field of IR imaging - he is actually proud to announce they have finally reached the VGA level of resolution - very sad..... Your channel is a must watch for me - keep up the great work. btw - thermal imaging is virtually useless for this application during most of the daylight hours - perhaps it would work at those times in the winter in north dakota during the winter.... this technology hasn't really changed since i worked with it five years ago - thermal imaging only works well at night when it comes to identifying living things that have temperatures well above the ambient temp of the stuff around them. LIDAR is much better for this task than thermal at this point in time..... have him demo his system during 8-5 business hours between april and oct. don't take me wrong - I LOVE thermal imaging at night - it is the ONLY way to go.

[@MunroLive]

Thanks for watching

[@airheart1]

It has its uses.. mostly only at night or in cooler locations.. in any warm locations in the day.. a persons body temp and most everything around it will be a mess of similar heat that is not very easily distinguished without very high res. All this for an AI in a car to interpret and act upon, is not easy. Once again, Tesla going for the simplest approach, seems the most likely to work across the broadest spectrum. It COULD be enhanced in certain scenarios with FLIR or Lidar.. but all in, it adds massive complexity to the system to determine when and where to utilize that extra sensor.. added costs.. added weight.. added maintenance and repairs.. or just teach AI how to use all its cameras better than just a human with two pointing one direction, and the cars will still be 10 times safer than any average human driver. It has to be basic vision for now, in my opinion. The day AI is super human and can learn quickly on its own and just accept a new vision mode and figure all out on its own.. great.. that sci fi shit is ways off. For now, 4, 6 or whatever normal cameras should/will be enough to manage driving better than humans, once AI is sufficiently trained

[@timmylassie6763]

the test drive was with issues - the cars parked only showed up because they did this at dusk and they were still warm from temperatures an hour ago - in another hour they would be begin to fade to invisible as the temperature change slowed to it's low overnight..

[@danubiosalas4231]

@@timmylassie6763 trusting IR at 5AM/dawn seems a dangerous idea.

[@bob15479]

Lol this channel has become marketing faster than any other channel I’ve ever seen

[@sjwright2]

It's sad.

[@MM_3857]

Would it be possible to integrate those camera into the lights as it is already plastic and maybe have a stereoscopic view of the surrounding area?

[@connelly6375]

"that thing shines like a new dime" does he just make this stuff up or does Sandy have a huge library of these?

[@paullester2535]

great demo and interesting insight

[@ilusnforc3]

I'd be curious about your thoughts on a notched windshield like how cell phones were doing notched displays for forward facing cameras.

[@JensStragier]

Windshields are already notched where all the cameras and sensors are (behind your mirror, if you look from the inside)

[@ilusnforc3]

@@JensStragier I meant notched as in physically cut around because Sandy mentioned that thermal cameras don't work through glass so if there were a notch physically cut out in the top of the windshield the camera could be mounted directly on the body. The notched you're talking about is just sticking a camera to the inside of the windshield, that does not alter the shape of the glass though.

[@eddiegardner8232]

Adding another sensor to have to coordinate with visible light sensing from the cameras might be a hard sell to Tesla, given their current direction of simplification and removal of hardware.

[@johnnysved277]

Cameras can be used for ir-detection. No new sensors, just a wider spectre.

[@eddiegardner8232]

The sensors would likely be different from the installed base of cameras already in the field, with backward compatibility problems to deal with in the FSD software.

[@seb_gibbs]

I've always thought that a new camera technology needs to be developed just for AI. We base current cameras on our 3 colour (rgb) visual system, but there is no reason why we cant give AI a new 6 colour visual system, IR, R, G, W, B, UV (W - white rods are sensitive to specific wavelength). No need for separate IR cameras at all. ideally, wavelengths 900nm, 570nm, 540nm, 497nm, 430nm, 350nm.

[@wngimageanddesign9546]

Most of today's digital camera sensors cover IR, R, G, B. There is an IR filter added in front of the sensor. White is merely an uncolored photosite. Which they can do. UV is another level of shorter wavelength sensor which may add to the cost to implement.

[@LeesChannel]

BTW that stone Sandy mentioned was quartz.

[@TennisGvy]

Man his "kids left the house" already in 2015? This guy only looks early 40s

[@LeesChannel]

They'll never do it, but they should have two sets of cameras spaced for stereoscopic IR imaging, and wide angle IR cameras for the sides and rear.

[@Will-be-free]

I don't think IR gives any advantage in populated areas. Either you will make the car react to anything that is warmer than ambient temperature, or you are only going to react to things that the vision system recognizes. It would probably be too much noise to be useful. Sure for the human eye, warm bodies in the thermal image pops out, but for an AI that looks at every pixel, maybe the difference is not that great. However if you drive at night on a small road, it would be useful to spot people walking on the side of the road and wild animals. Out in the countryside you can be fairly certain that anything that is warm is something you should avoid.

[@danmobile]

Even in a city, at night time it's a big advantage IMO. I think if you're driving down any street and someone walks in front of you at night time, IR will spot this way before any camera.

[@stephendavis6261]

This feature needs to be on Cyber Truck

[@A2an]

As usual, really interesting. Precisely, pick up the kid behind the car we aren't able to see👍 Liked it a lot 👍

[@sin3rgy]

Did they answer what happens when ambient temperatures are at or above body temp? how does it know?

[@markplott4820]

you can Recalibrate the Sensor on the fly. FLIR is useful such as in the FIGHTERFIGHERS Helmet Cam / Sight. and in LAW ENFORCEMENT.

[@beatnik1855]

Right, if it can't see me cus I have a coat on in the winter, then whats the point.

[@charlesrovira5707]

@@beatnik1855 If you're like most people, you're probably *not* wearing _thermal camouflage _*_over your face._* (That stuff's not available to people outside the military anyway. If somebody's wearing it, he doesn't want to be seen ... at all.)

[@charlesrovira5707]

It works with lasers. (Like one of those touchless temperature sensors.)

[@lapaleves]

@@charlesrovira5707you mean like a hoodie from behind?

[@zodiacfml]

6:17 timely. about this part of the video, i've been thinking the value of thermal imagery. however, thermal would be rather poor for objects or parked vehicles. thermal appears to be a good backup or secondary system

[@sjwright2]

It's not even a "good" backup or secondary system. Thermal imaging stops being able to spot pedestrians reliably if it's a very hot day. It can't see road lines, it can't read road signs, it's expensive, it's fragile, and the technology has severe export restrictions. I'm struggling to think of _any_ situation where thermal imaging would be worth the effort.

[@zodiacfml]

@@sjwright2 that is the reason this could exist with visual like the Tesla Vision as it can't be relied on all the time even if the AI is as good or better than humans since there are also limitations with vision. if i remember it correctly, export restrictions only apply to higher resolution thermal and not vga.

[@sjwright2]

@@zodiacfml There are vanishingly few scenarios where machine learning on visible light imagery can be improved with the addition of thermal imaging. Certainly not enough to justify the high cost and high complexity. There are simply too many scenarios where thermal imaging is too unreliable to be useful. And there are too many scenarios where thermal imaging would be highly misleading.

[@dhiltonp]

@@sjwright2 Unlike fusing 1D radar+vision, fusing 60hz vision and 60hz thermal vision is simple. There are plenty of scenarios where regular vision can be confused, and plenty of scenarios where thermal vision can be confused. For the most part, those scenarios don't overlap. It's a near perfect scenario for sensor fusion.

[@sjwright2]

@@dhiltonp I agree it's simple relative to the processing of machine learning. Although I should note that FLIR themselves do an utterly terrible job of it with their "MSX technology". I also agree that you can find some narrow scenarios where thermal can occasionally disambiguate visible light. But it's never reliable, because it's affected by many factors which don't plague visible light-such as ambient temperature, thermal reflectivity, heat soak, and sensor calibration cycles. (That last one is a doozy. Oh I'm sorry, did you say you wanted a sensor that can run continuously? Sorry, nope. This is FLIR. You get a sensor that takes itself offline every ten seconds.)

[@shawntilluck]

Somehow I would not be surprised if Sandy got a pet deer,...for testing purposes.

[@wngimageanddesign9546]

I've always thought infrared would be potentially used to pursue the "99999's" for level 5. But one question is how effective is FLIR when operated in extreme temperature climates? ie. AZ desert temps of 115 F, with roads radiating over 160F.....a human body is going to blend into the background. Pedestrians and cyclists will essentially become stealth.

[@gondorian1006]

Works fine at 120°+ in my experience. In high heat bodies show up as cool spots.

[@johnbuchman4854]

Sandy has a "steak" in this technology...

[@nanoadrian1]

You have to watch till the end to get it 😂

[@yorkchris10]

Would have been nice to see edge cases where camera would have failed.

[@DerKatzeSonne]

If they coulnd't even come up with this in the marketing video I don't have high hopes. Honestly, I really thought this thermal imaging technology was promising but seeing this presentation I'm not convinced anymore.

[@snitox]

I kinda would buy the magnets if it had like a graphic of the top 4 most efficient engines of our time.

[@chrischatterton2460]

I hope Aptera Motors is watching.

[@wrohr15309]

Every time they say Teledyne and i think of Cyberdyne from the Terminator.

[@RobertGoley]

Wait till they attach tech to self driving cars... Will be more fitting...

[@2nd3rd1st]

18:50 What strikes me most is that they are talking like it's most important what information the driver can see on the IR camera feed. But the whole point of autonomy is that the driver wouldn't have to pay attention to the road at all anymore. Where is the importance of augmented reality and heat vision when seeing them is irrelevant?

[@bobwallace9753]

I think there's an assumption of driver assist. There was mention of using it with a heads up display. In poor light conditions it should be possible to 'put a box' around a human/other animal in the shadows and display the box on the inside of the windshield. Interesting idea, but probably not worth developing since we're so close to self-driving.

[@fredbloggs5902]

I agree, the only potential really is improving FSD and I’m doubtful about that.

[@yourcrazybear]

@@fredbloggs5902 Seeing people in the dark will save lives. That matters.

[@TimothyWhiteheadzm]

For those interested, Tesla dropped radar because although it has a fairly high accuracy rate, it also has quite a lot of bad readings, which made it difficult to clean the data and since they felt they could get just as good or better information from the cameras, it wasn't really adding anything. When its readings disagreed with the cameras, it made sense to ignore it anyway (due to the bad readings problem), so it was being processed but essentially ignored, so might as well just remove it. Thermal would have the advantage of adding greater visibility in certain conditions, but the reality is that humans can drive without thermal vision and low light situations are a very small part of the self driving problem. Thermal cameras may aid in object identification, but they may not make a significant difference. Yes, a self driving car needs to know the difference between a pedestrian and a plastic bag, but it also needs to know about bricks, traffic cones, and many other solid objects that are not living and know how to distinguish what you can safely drive over and what you can't, with similar reliability or better than a typical human. You don't want your car to say 'Oh that's just a fallen tree, not a pedestrian or deer, go ahead'.

[@zxggwrt]

We were just talking about the problems you get when radar disagrees with cameras. That's how we get this UFO nonsense, basically.

[@gibby7616]

Sweet Stuff!! Just bought me some #Teledyne shares!!

[@budgetaudiophilelife-long5461]

🤗 GREAT TO SEE ALL THE TECH BEING USED TO MAKE IT SAFER 😎💯…and happy to see you had a good meal with all your hard work 😋 KEEP IT UP 👍 AND WE WILL KEEP TIPPING THE PEOPLE 😍😍😍

[@FlipBoxStudio]

Anyone whose been in the military long enough will get the vibe that the guest may have been an officer in the military. Or has worked under the government as an SME in some form. Definitely not infantry related though.

[@cappybenton]

Wunderbar. Excellent cowboy boots and an excellent guest.

[@seb.]

With global warming, how would termal distinguish pedestrians when ambiant temp is the same as body temp?

[@energiewender143]

As I understand it, the car‘s camera vision gets enhanced by the addition of the flir cams in the infrared spectrum - flir is not a stand-alone system, I hope. But you‘re right to ask, as one can imagine such a scenario in the dark with roads still hot from the day‘s heat.

[@martin_mue]

The information quality level of this video is outside the visible spectrum. ;-)

[@pensansproductions3278]

How about icy roads? Spotting those ice patches under trees etc is another great advantage of flir vs current ambient temp only sensors.

[@derekwallin2624]

The issue is thermal cameras see temp gradients. Ice is typically the same temp as the surroundings so would show as very similar

[@markplott4820]

tie in the FLIR sensor with a MICROWAVE transmitter to DETECT and Break up the ICE.

[@pensansproductions3278]

@@derekwallin2624 I do agree with that statement. however it is also possible to see areas of the road such as over a stream or river or hidden in the shade by trees where ice patches could form vs where it can also see the road is above freezing. It might not see ice patches where the road is sub zero anyhow but it can pick out areas where the road is liable to freeze. its shocking just how good flir cameras are. I fitted one to a drone and could see pheasants in a hedge row clear as day 200m away where even looking with binoculars it was not possible to spot them. same if I stood bare foot on the workshop floor for 30 seconds...the heat signature was still a foot print 5 minutes later with a perfect foot print profile, granted the correct temp range had to be carefully selected but it was shockingly good!

[@pensansproductions3278]

@@markplott4820 probably some legal issues on being able to emit that many KW of microwave energy in front off the car to do that.... however pre cooked road kill could become a "thing" if it wasn't lol

[@kayodetitus5252]

Awesome video from Munro Live. I like these videos show casing new technologies and how they can be used to solve our currently problems and make the road much safer.

[@rexwitherspoon7176]

Isn't this unneeded as Tesla can use lower resolution non-thermal cameras (1/5 the resolution of NIO) for full self driving?

[@carexpert5945]

Just trust Elon. No need for LIDAR, Thermal, high resolution cameras, HD Mapping, redundant sensor modalities; things every single competitor is exploring and using. Just trust the almighty Elon with

[@bluetoad2668]

@@carexpert5945 I know you're joking but there is an interesting dichotomy here - the multi sensor approach actually makes things difficult for an AI solution. This is counter intuitive of course, but so much is when you get into AI. Don't be so fast to jump the conclusion that Tesla (and at least one other) are wrong.

[@2nd3rd1st]

@@carexpert5945 Does CAR in your name stand for Cynical Angry Regressive? I'm just glad you don't lead the FSD efforts. You'd spend billions on legacy tech, try to map the whole planet, and make the cars so expensive nobody could afford one.

[@carexpert5945]

@@bluetoad2668 no it doesn't, plain wrong. The redundant sensor modalities check each other, just like Tesla's 2 redundant computer chips check each other.

[@bobwallace9753]

I think the question is whether adding a forward looking IR camera might add function at a reasonable cost. Would being able to look into fog, the dark, etc. be valuable enough to justify adding a camera? Might it be possible to create a camera with a 'retractable' IR filter and use it as the narrow field of view (telescopic) Tesla camera? Pull back the filter when conditions are such that the narrow field camera isn't providing much information and use heat signals rather than visible light. IIRC, at one time in the early days of consumer digital, Sony made a digital camera that could capture IR images.

[@amkarkare96]

At 30:34 you can see a bike being detected by FLIR on the left lane going in opposite direction. You see it in next scene in the back glass.

[@richardrigling4906]

What are the impediments to TESLA adding an IR video sensor and integrating multispectral imaging into their vision system?

[@rockon7848]

Installed base of 2 million vehicles without IR. This would be the first magnitude of scale legacy problem for Tesla. It is superior tech and only needs software update for the increased spectrum. Plus the IR camera. He just dropped a couple of bombs. . " I can't speak of specific companies but most OEMs are looking at it," and " most high end FSD vehicles already have IR cameras" Not sure why Tesla would keep IR hidden athis point but if Sandy is talking about it, Tesla has been all over it from the beginning. Be interesting to hear what Elon has to say.

[@richardrigling4906]

Would it be possible to replace 1 or more current cameras with multi spectral cameras? Still a retrofit, but it adds IR to the current visible.

[@rockon7848]

@@richardrigling4906 Actually if they don't already have them installed, there should be an empty spot where the radar was located. But this is not news to Elon by a long shot. If this is that beneficial they will make it happen. The moves Tesla made with the upgraded models is a testament to their confidence in FSD going live and soon. The current beta is looking for the things the radar did bring to the table like Hyper's monorail columns and the light pole Sandy mentioned. It's much easier to integrate a widening the electromagnetic spectrum covered to include IR as the merged pictures showed. A software update fix. Radar has a different format so to speak. Tesla has missed much so far, as they say Don't bet against Elon.

[@richardrigling4906]

@@rockon7848 Agreed. We are speculating from the outside on potential avenues they could take. Like SpaceX, TESLA has no shortage of engineers willing and able to say "Why can't we just . . ."

[@mickpeterson5940]

It's good at picking up travelling ICE vehicles but seems to have difficulty picking up (tagging) 'cool' parked vehicles from thermal - how will it go picking up & identifying moving electric vehicles in various weather conditions given the thermal signal of electric vehicles won't be as pronounced as ICE vehicles?

[@Tigerratcat]

Good point. A parked EV should be fairly cold or close to ambient

[@watos77]

How do we get the camera inside the car, special windshield glass (IR transparent)?

[@rhamph]

Cut a hole or notch in the windshield, glue in special IR transparent glass.

[@DavidSpiers]

really interesting video

[@MunroLive]

Glad you think so.

[@amantedecapaldi4037]

Thank you I was asking about this for so long

[@WillN2Go1]

The road test looks like the hardware is robust. Some of Sandy's concerns may be more about information processing than visual wavelength captured. I can think of two problems. Deer. As I'm sure Sandy knows they aren't running through the woods and just happen to run across the road right in front of a vehicle. They approach the road, perceiving it as a open ground and a potentially dangerous space, then they see the approaching vehicle as a likely predator and so they do what's worked for millions of years, they freeze until the perfect time to leap across the path of a 20-30 mph predator - except it's a car traveling at 40-60 mph. If I was a cougar instead of driving a Cougar I'd try to turn and just go ass over tea cups. With a Cougar it slides right across the hood and through the windshield. Ideally that FLIR system spots a shape, perhaps it's height off the ground - recognizes it as a deer, and alerts the driver, or automatically slows the car. To the driver none of what the FLIR has picked up looked like a deer. This is a Labeling problem. I'm guessing that a lot of intersection collisions, someone running the light, or any vehicle heading directly towards another. It's difficult to judge the speed of something heading directly at us. If a machine can indicate an approaching vehicle that's going 'the wrong speed' i.e. it's not slowing down and alert us to that. This sounds like maybe a radar solution. I'm planning to buy a sailboat in the next couple of years and head off over the horizon. FLIR systems for boats are about $25k. Could be money well spent. It's high on my list. Good video. Thanks.

[@michaelrasch107]

Have you ever seen a deer? the answer is no. if you had you would not have made a statement that is totally false. But at least you can make a long post..

[@thisApex3D]

You can drive car hence pure vision if all you need. Period. It odds not anymore how to collect data, it is about how good your NN is.

[@danmobile]

At night time, can you see someone wearing black that steps into the road with a camera? Maybe, maybe not. Can you see someone with IR? Absolutely.

[@RealHIFIHelp]

I would like to see some Nissan LEAF videos, if it's possible.

[@ebb2421]

if windshield glass is an issue for flir, what are the camera lens made of?

[@scottm4894]

Silicon on low cost, germanium on high performance

[@cybertater5236]

You would just want a glass/material that doesn't block IR. It's not rare, just often you want to block IR/UV because people only need to see visible light.

[@Wavefront101]

@@scottm4894 This is the problem with these cameras, the germanium windows. They oxidize and the lens becomes pitted, more so for the flir marine cams. They are also quite expensive. If you own a flir automotive imager and want to preserve the lens, you need to clean it daily with distilled water.

[@ReevansElectro]

There is a time to talk and a time to shoot. - Ely Wallach.

[@AdityaMehendale]

Are FLIR cameras intended for sale outside the US still software-limited to 9FPS due to silly regulations from the US gubmint?

[@mikes2221]

I do not find any information what a radar device in a car really sees. Is there any video out there to show us what a radar sees and what it doesn't see?

[@sjwright2]

A horizontal plane of distances.

[@DerKatzeSonne]

Just a couple of points, mostly of moving objects (and if stationary, only of very big objects). The data is so abstract, it's not easy to even visualize it. You need to do a lot of pre-filtering to have anything usable at all. But here's an older video of how Tesla sees the world in visual and radar. The circles are radar detections: kzbin.info/www/bejne/gKqVY5Wglseceac

[@ferkeap]

Distance measurements. And you could detect the difference of hard and soft radar objects. And acceleration of objects

[@jamesjross]

Teledyne was cited as an inspiration for Terminator's Cyberdyne... the fact that this 60 year old company is now an "AI" company will have everyone thrilled.

[@eugeneputin1858]

Those bounding boxes are horrible..?

[@rhamph]

Jumping around, disappearing/reappearing, missing "obvious" objects.. it's like watching a Tesla video!

[@DerKatzeSonne]

And not to forget: They only show 2D bounding boxes here, Tesla has 3D bounding boxes. That's a HUGE difference if you really want to enter the FSD game. With 2D you can just use it as driving assistance, e.g. for AEB, I think.

[@PaulVanchugov]

"When I was at Ford"... :)

[@ChaimLoecher]

Great interview ... i would have had so many 2-word questions - fantom breaking? Tesla vision ? Total cost ? I wonder who has the compute power and neural net to perfect this "errand" 🤔

[@MunroLive]

Glad you enjoyed it!

[@torchlord11]

So I would think this might work really well in very cold snow-covered roads? Yes or no? Does the Tesla camera approach have difficulty still in winter situations?

[@billdale1]

John Eggert said they moved from Oregon to Minnesota "to be closer to Detroit"--- sorry, that's absurd! I might be supportive of his work with FLIR, but Minnesota is still HUNDREDS of miles, and a Great Lake away from Detroit--- if you wanted to get closer to Detroit, why not just MOVE to Detroit? I bet commercial real estate is cheaper in Michigan than in Minnesota! I cannot understand the half-way move, or the half-way explanation of that move. I do hope the FLIR products are what we all need for FSD.

[@theoldman8877]

A large percentage of the driving public doesn't want to accept the fact that all vehicles have equal rights on the roads. Be they fast, slow , large or small . A woman pushing a stroller has equal rights as a strach lomo. A smart car has the same rights as a 18 wheel trucks. Bottom line is no one is entitled to an open unobstructed road way. Their will always be hazards in the road. If you hit anything it's your fault one way or another. If you didn't see it you wear driving at an unsafe speed for the conditions.

[@Martin.Brennwald]

24:46 Is it just me or did he say that Tesla might be i costumer?? " I can't talk too specifically about any costumers..." Sandy asks and he just "can't answer".

[@Will-be-free]

I think that you read to much into it. If he for example had said that Tesla was not a customer and later received a question "what about Ford", how could he possible answer that without outing that relationship?

[@blackmartini7684]

He said OEM aka Ford

[@ztechrepairs]

I wish them the best of luck but I think Vision will win out. But we shall see

[@slidewaze]

I see what you did there. :^)

[@ztechrepairs]

@@slidewaze Thankyou somebody noticed ahahahahaha!

[@dewiz9596]

Yeah, I saw what you did there too. Nicely done.

[@bobwallace9753]

This is (small v) vision. It's just using wavelengths that most humans can't see very well.

[@GroovyVideo2]

i know someone doing RD this area - he Likes

[@DishNetworkDealerNEO]

I like this guy! The first thing he does is quote Issac Asimov’s First Law of Robotics!

[@ligius3]

I'm not up-to-date on the thermal imaging stuff in automotive, but there's at least one huge European supplier that's been putting FLIR on cars of all sizes. The problem is that this is a heavily restricted technology and that should be clarified. I doubt that you can get 60fps of video from that camera, even though technically possible, it's probably feeding back 60fps of "data". The sensors are also paired to the cars using cryptography, so forget about "hacking" one for DIY use. Reading between the lines, the "new" FLIR company wants to get into the space of AI/NN instead of just providing the sensor like it did before. That's not a bad thing, it just takes a HUGE amount of worldwide data to train the networks, data that other companies already have spent >10 years gathering. It's unfortunate that such a lifesaving technology is heavily crippled by ITAR restrictions. Hopefully it will eventually get to the consumer space, like GPS did. But GPS can be remotely shut down, while FLIR cannot (to my knowledge) so it makes a bit of sense. For those wondering why ITAR: mostly to deter building heat-seeking missiles.

[@shazam6274]

Notice how the motorcycle at 33:10 is not detected. Maybe if the muffler was still hot?? How does this work in cold, i.e. snow / sleet / hail etc.? Human vision can detect a person bundled up for minimal thermal signature in falling snow, but I would be surprised if any IR camera can do that. If "VGA" resolution is 640x480 and this thing operates at 60 frames per second than the implied

[@bobwallace9753]

Professor Sandy brings us important knowledge. Thanks, Sandy. I do love learning me something new.

[@sjwright2]

Important knowledge? Please. This is a joke. FLIR is great as a specialist tool, but for automotive it's a joke. A sensor that is expensive, can't spot a tree fallen on the road, can't spot a human on the road when it's very hot outside, might not see cars that aren't emitting heat (EVs?), cannot be protected from damage, gets blinded if a drop of water hits its silicon "lens", and has severe export restrictions. Yep, this is going nowhere. Guaranteed.

[@lltv8142]

Great technology. But it seems a bit overkill for the purpose considering the price. Since the selfdriving range is more limited compared to a fighteryet. I think the same vision can be achieved via good headlights with some additional powerful IR spotlights. And those are much cheaper. But great to see new solutions to the existing problems for self driving

[@bobwallace9753]

How expensive would thermal imaging be at scale? If I'm not mistaken it's a butt simple digital camera with no IR filter. Digital sensors are already able to pick up IR, there's a filter in front of the sensor to block IR for 'picture camera'. Once the hotter stuff triggers more pixels then from there on it's just how the data gets processed.

[@lltv8142]

@@bobwallace9753 I could be wrong. But if I remember correctly there is a difference between flir cameras (heat) and ir cameras (respond to a light of 850nm and 940nm). Ir can indeed be viewed with most cameras (like security cams). But for flir (heat responsive) you need another sensor. And those are usually lower res and expensive

[@bobwallace9753]

@@lltv8142 I did the google. My memory did not fail (this time). There are multiple companies that converts normal digital cameras to IR capable cameras. They remove the IR/UV blocking filter and replace it with a visible/UV filter which allow longer wavelengths to get to the sensor. Digital cameras are now dirt cheap. A filter swap during production should add almost nothing to the cost of the camera. "Heat" emits infrared wavelength energy. IR light.

[@blueredbrick]

@@lltv8142 You are correct. Heat infrared is quite different from near infrared.

[@reindrescher9092]

What is Sandy’s reaction to Andreij Karprthy’s recent CVPR’21 discussion?

[@RedlioNxTR]

Stop whining about marketing, it's not a shampoo commercial.

[@capslock9031]

Ok, Teledyne, you’re an AI company… Cool. How large is your data set? What is its rate of growth and how clean is it? How is it annotated? How big is the fleet? What’s the hardware you’re running it on in the car in terms of compute power and energy consumption? Need I go on?

[@DerKatzeSonne]

He even admitted they are missing the data collection from a fleet in the context of deer. Sure they want to use simulations for training but that only is half of the work needed...

[@PhotoHoag]

This is great. Thanks Munro and FLIR! Glad you got on not seeing through glass and options to still have sensor.

[@damianweiss217]

Is FLIR also night vision ? What about heavy rain or snow ? Parked cars covered in snow ❄️ snow ? Arizona sandstorms ? Can FLIR see all these ?

[@Clearphish]

I seem to recall Sandy saying that it works well in rainy conditions. Might get a few of your questions answered if you watch the video a second time.

[@sjwright2]

@@Clearphish FLIR is no better than visible light for heavy rain, in fact it's potentially worse because the sensor cannot work at all if there's water on it. There's nothing useful that FLIR can do which visible light can't so long as the vehicle has decent headlights.

[@Clearphish]

@@sjwright2 Thanks. That may be why Sandy said the system needs to be installed in the windshield with wipers to keep the water off.

[@sjwright2]

@@Clearphish Good luck getting a silicon lens to align perfectly with a hole in the windshield glass so that it doesn't leave any water residue and doesn't scratch away the coatings. Oh and congratulations, you've also increased the cost of replacement windshields by another $300+. Awesome.

[@Clearphish]

@@sjwright2Thanks, I'll remember that If someone actually succeeds in incorporating the tech into vehicles. If they don't, I guess it won't matter.

[@frankhage1734]

I suggest using some goat, lama, dog, and horse herds to generate the data. You could mount static cameras at several locations to simulate approaches at various speeds. Deer, Elk etc tend to cross in small groups. If you see one, there's more close by. Even for humans, it's a hard lesson to learn.

[@MarkXHolland]

Have you ever tried to herd goats, llamas, dogs and horses?? What could possibly go wrong?? :D

[@frankhage1734]

@@MarkXHolland No, no, yes and yes. Pretty much everything could go wrong, but won't because it's staged. Drive around with some ranch dogs running outside and you might understand.

[@user-in4ex3tr4d]

This feels like marketing verry unmunro like ... sad if this is where the channel is going

[@sjwright2]

Agreed. It was embarrassing to see Sandy being wowed by such a rudimentary demonstration of machine learning vision systems. And it was hilarious to see him impressed by the quality of the non-FLIR, visible light only image! Eggert must have been too embarrassed to point out that Sandy was more impressed with the visible light sensor than the FLIR sensor.

[@spelare2]

No discussion on Teslas vision system?

[@davidbeppler3032]

Not the same. In 10 years they build a million sensors. In 10 years Tesla built 1 million cars.

[@Urgelt]

If Tesla is right, then purely optical cameras, with no added sensors, *if* the software is good enough, will lead to L5 autonomy. Software is the key. Human eyes, with a narrower spectra band than cameras, already do this with wetware. The problem with human drivers tends to be suboptimal decisions, slow reaction times and inattention, not spectra adequacy. (If conditions are so terrible that eyes aren't good enough, it's not safe to be driving anyway. Get off the road!) Adding FLIR to Tesla's approach would increase computation needs (more spectra, more data, more power draw, reduced range). So the core question about whether to use FLIR, or radar, or lidar is, is Tesla correct? I think the answer is 'yes.' I don't expect Tesla to be interested in FLIR. So what about everyone else? Without solving vision, these companies have no developmental path to L5. They can implement crash-avoidance systems with these sensors. But they can't recognize objects with them, and if you don't know what objects are, you can't predict them. There is an upper limit to the safety that can be obtained without solving vision. Teledyne FLIR appears to be using neural net software to draw little boxes, but this level of software is a far cry from solving vision. And to their credit, they don't pretend they've done that. 'The best part is no part.' The real challenge isn't the sensors, it's the software.

[@Tom-ku8bu]

I think the idea is that the flir would help with the decision if that just an empty plastic bag there on the road or if there's something like a baby in it.

[@TheAegisClaw]

Actually, as a photographer, I can tell you camera sensors have less dynamic range than the human eye. That's why HDR software exists. For example when I photograph a wedding, my high end DSLR can either expose the groom's black suit or the bride's white dress, but not both (we always pick the bride, or at least mostly the bride). We recover detail on the one we didn't pick in post processing.

[@TheFunfighter]

I think what special sensors do for autonomous driving atm, is that they filter information as a side effect of their mechanisms. That reduces the need for processing, which is currently the big hurdle. If object recognition were easy, we would just stick on a bunch of cameras and get rid of like 50% of a car's environment sensors.

[@slopedarmor]

I'd still count thermal cameras as vision.

[@fredbloggs5902]

@@Tom-ku8bu I’ve been driving for over 40 years, I’ve encountered plastic bags fewer than 10 times. When it’s been an option, I’ve simply stopped and waited. FLIR would not have changed anything.

[@MarkXHolland]

What if it's a very sunny summer day and the road is hot, the trees are hot, everything is hot?

[@daniellyons3970]

That’s where the visual camera comes in … a blend of both thermal and visual is great way to view things

[@samg5543]

The view adjusts based on the hottest object

[@craigduncan4826]

@Sam G - yes it is the contrast between hot and cold. So even if it’s 50 degrees outside, you will still be able to see people- the temperature gradient is just different, vulva

[@markplott4820]

Francis - sensor can be RECALIBRATED , modern FLIR dont even need LIQUID NITROGEN to function.

[@MarkXHolland]

@@craigduncan4826 OK but don't call him vulva. 😆

[@AWESEM0]

Interesting.. but now they going to need to find someone to interpret the data and at the current rate they'll take another 20 years .... Tesla has this in the bag

[@DerKatzeSonne]

So... "we now can do VGA resolution" (saying it like it's a good thing and was much lower before; still very low resolution) Continues a bit later with "the advantage of our computing is that we are very efficient because of the lower resolution" (sure; I bet you guys are working on upping the resolution) And later "the problem with deer and such is that it's difficult to collect data to annotate because you don't come across deer very often"... Well - that's what Elon Musk is always prasing about Tesla: They have the unique advantage to collect data from the fleet. Even in high numbers for rarer cases such as animals crossing the street.

[@christophermcgrath8847]

Correct but remember this is a thermal camera so VGA is considered higher resolution, other characteristics are more critical to performance in thermal than in optical/CMOS cameras, for example NETD/pixel pitch are something you may not be aware of. Also the lower resolution and the fact that thermal is black/white would lend itself to image processing analytics as there is a lot less data to process and it is easier to train the algorithm as the data is alot more homogenous. FLIR is a sensor company similar to Sony so isnt in the same position to gain data that Tesla is, also btw the CMOS system on a Tesla has killed people due to inability to discern a truck from a cloud so maybe they could do with the data, as Musk would say "You have to break a few eggs to make an omlette".... I just dont want to be an egg..haha

[@kathleenclendennen6744]

Come to Texas and you can get all the examples of deer that you can use! We have a neighborhood full!

[@aeroskiff]

Hi Sandy, were they using the newly developed "Akida" chip from Brainchip to process this tech?

[@dclpgh]

IIHS has completed tests of the 2021 Tesla Model 3's new camera-based front crash prevention system, which rates superior for vehicle-to-vehicle interactions and advanced for pedestrian interactions. And that's with no radar and no lidar.

[@DerKatzeSonne]

But tbh it's done in daytime and they claim FLIR works better at night time. There should be different light conditions for these tests at some point and I bet we'll see them as soon as all manufacturers score quite well in the IIHS and other tests.

[@markedwards4879]

24:56: “Are Tesla moving toward this?” “I can’t talk about customers”….

[@sjwright2]

Don't read anything into that. Even if Tesla has purchased one camera on a lark, then they're a customer.

[@jonathanmelhuish4530]

I'm sure Tesla is experimenting with FLIR cameras, they use many types of sensors for testing and benchmarking.. even LIDAR!

[@DerKatzeSonne]

That means nothing. If they really were customers, he would have straight away said "can't talk about it". But he thought a bit too long for this. So most likely, the answer is a straight up "no" but he also knows that doesn't really sell it to other OEMs and that he is not supposed to talk about customers or even non-customers. Oh and Andrej Karpathy was asked about thermal imaging in the last talk and while he didn't deny it straight away like Lidar, he basically said "no". It sounded like maybe in the future at some point but at the moment, it's still too low resolution and we still have a lot of potential to solve just with cameras.

[@DerKatzeSonne]

@@jonathanmelhuish4530 no, their lidar is basically most likely only for calibration and ground truthing.

[@sjwright2]

@@DerKatzeSonne Why did you say "no" to Jonathan? He said "testing and benchmarking". You said "calibration and ground truthing". He might have been using more casual terminology and you're using more accurate terminology, but I don't think you disagree with each other.

[@yeaokay10011]

Are we ever going to get to a point where vehicles will summarize and share their data through a standardized network? Letting vehicles inform each other of their position, status and visible obstacles, etc?

[@fredbloggs5902]

Maybe, while an attractive idea it’s not really clear how useful that is in terms of making a difference.

[@danubiosalas4231]

@@fredbloggs5902 I think it would make a difference in obstacle avoidance, at high speed, in FSD. Basically same concept as brake lights (informing next driver that my speed is slowing, before perception kicks in). It would allow for more compact blocks of self driving vehicles, and smaller roads needed.

[@polarbearigloo]

So a FLIR camera is great in this. Would I say other things like Radar no. But can it work on a pure vision application as a running change yes. FLIR cameras are slightly more expensive with black out lights. I would not be surprised if tesla uses this in tandem and (maybe black out lights)

[@katiegreene3960]

Minnesota in the house.......nice.

[@m_sedziwoj]

sells man... most blabling, but technology looks interesting

[@40watt_club]

I wonder if Tesla isn't already using that kind of tec :-)

[@davidbeppler3032]

Not exactly, they use cameras with a larger vision bandwidth than humans which helps.

[@Matzes]

Tesla is all in on vision

[@davidwilkie9551]

Video plus comments, all very interesting. I like the extended spectrum, and a Machine Operator has always monitored "self driving" machinery, it's the training of learning by doing Intuition, to relax into the job without straining over the details. No harm in letting a computer help..

[@lobotomite3248]

Frsx bundles this for OEMs? Or Teledyne does?

[@Miata822]

How embarrassing. Sandy doesn't get that the video he really likes and any color on the screen is coming from an optical camera, not an infrared one.