PyTorch at Tesla - Andrej Karpathy, Tesla

  Рет қаралды 507,183

PyTorch

PyTorch

Күн бұрын

Hear from Andrej Karpathy on how Tesla is using PyTorch to develop full self-driving capabilities for its vehicles, including AutoPilot and Smart Summon.

Пікірлер: 358
@la-civetta
@la-civetta 4 жыл бұрын
Some people have great empathy, Andrej has great carpathy.
@lonnybulldozer8426
@lonnybulldozer8426 2 жыл бұрын
*self-driving carpathy. (Some people are self-driven by empathy. Andrej is a self-driving carpathy.
@AgentOffice
@AgentOffice 4 жыл бұрын
Karpathy, very fitting name
@ishunyu
@ishunyu 4 жыл бұрын
Agent Office LOL true. Never realized. 🤣🤣🤣
@gridcoregilry666
@gridcoregilry666 4 жыл бұрын
might explain pls?
@borisdemelo
@borisdemelo 4 жыл бұрын
Julio Chao he has ‘kar’ in his name and works for a ‘car’ company. :)
@AllanSustainabilityFan
@AllanSustainabilityFan 4 жыл бұрын
​@@gridcoregilry666 Kar-Pathy -> Car + Path -> self driving cars.
@AgentOffice
@AgentOffice 4 жыл бұрын
@@gridcoregilry666 car path
@buzz8545
@buzz8545 4 жыл бұрын
How many of you checked the playback speed?
@carvalhoribeiro
@carvalhoribeiro 4 жыл бұрын
looks like Abigail Doolittle from bloomberg 1.5x speed
@Wingman77tws
@Wingman77tws 4 жыл бұрын
came here to say that. might be the only video ever worth watching at .75
@sebbecht
@sebbecht 4 жыл бұрын
Dead serious, many videos I watch on youtube is on 1.25 or often 1.5 speed. but this guy, everything is 0.75, just to make sure I dont miss anything
@akarmdit2267
@akarmdit2267 3 жыл бұрын
usually 1.5 but his 1.25
@pretamane9646
@pretamane9646 3 жыл бұрын
yessss
@cappuccinopapi3038
@cappuccinopapi3038 4 жыл бұрын
*Me who’s struggling even with my basic calculus class: Fascinating
@alekseysoldatenkov5675
@alekseysoldatenkov5675 4 жыл бұрын
All you need is curiosity and resilience! You got this dude!
@GRMREAP3R97
@GRMREAP3R97 4 жыл бұрын
That's the mark of a true teacher, they can make the most difficult of concepts seem easy and fascinating
@Wulfcry
@Wulfcry 4 жыл бұрын
Google Children wooden educational toy's Montessori math , Did you know we all played with those didn't sink in then, Well these structure are practically the basic shape which also to find in calculus. If you can't rotate them in you're head get them and visualize the image space. These shapes also hold structures done with programming. It might seem simplistic after all these years but breaking you're head to grasp something that has these shape why not. Circle -> loop , iteration, segmentation etc. Try fill in the blanks with Square, Triangle and Pentagon shape. and cube some. We all are well endowed with knowledge use it.
@WandererOfWorlds0
@WandererOfWorlds0 4 жыл бұрын
There's very little calculus in neural networks besides differentials (gradients).
@kalp2586
@kalp2586 3 жыл бұрын
You don't need to know calculus for deep learning. Like you don't need to learn assembly language for building web apps.
@thischannelhasaclevername5481
@thischannelhasaclevername5481 4 жыл бұрын
11:02: "Thank you" 11:03: Exit to work
@ashh3051
@ashh3051 4 жыл бұрын
That was awesome. Lots of new insights beyond what was presented at Autonomy Day. I wish he had 1 hour to talk.
@TheNyatzAnger
@TheNyatzAnger 4 жыл бұрын
Always great to hear Andrej's talks. He's left an indelible impression on my research career in deep learning through CS231N
@UmuroElema
@UmuroElema 4 жыл бұрын
🙏
@sreekarnim163
@sreekarnim163 4 жыл бұрын
CS231n is IMO the BEST online CS course on the internet.
@aishahsofea3128
@aishahsofea3128 3 жыл бұрын
can i take the course if im not a stanford student?
@sreekarnim163
@sreekarnim163 3 жыл бұрын
@@aishahsofea3128 yes
@MdSheraj
@MdSheraj 3 жыл бұрын
@@aishahsofea3128 kzbin.info/aero/PL3FW7Lu3i5JvHM8ljYj-zLfQRF3EO8sYv&feature=share There you go.
@JuanLupion
@JuanLupion 4 жыл бұрын
Wow Karpathy's a fast talker, its like the video was sped up.
@Findalfen
@Findalfen 4 жыл бұрын
I slowed the video down to 0.75 and it was finally intelligible. Speaking as fast as he is doing is not a good trait. If you want people to understand and remember what you said; you need to let them time to process it. Quite the irony for someone working in data processing. Does it believe we are all machines?
@cleberz8072
@cleberz8072 4 жыл бұрын
It's interesting to hear your opinions - I've been living in the US for 14 years and have no problem understanding him, as he's still relatively slower than how people generally speak (perhaps not in presentations). He has an accent which makes slight harder to understand than a native but it's still not as fast as it appears. But yeah I definitely would not have kept up 14 years ago, the 0.75 is a nice trick you guys got there, I wish I had that back then
@m3po22
@m3po22 4 жыл бұрын
Yeah I was like, "Did I already speed this up? Oh, it's just him."
@joelodlund6979
@joelodlund6979 4 жыл бұрын
talking fast makes him able to make complex points while still keeping the listeners attention. I have no trouble following along, and I'm not a native speaker.
@cleberz8072
@cleberz8072 4 жыл бұрын
@@joelodlund6979 I'm thinking the issue with the native speakers complaining here must be because most of the tech vocabulary he uses are unfamiliar terms to them. Complainers, please enlighten me. I'm used to most if not all of those terms given my professional background. How about you? My brain processing will certainly slowdown when I encounter a completely new subject regardless of the languages I'm fluent at
@pks.
@pks. 3 жыл бұрын
no LIDAR? just wowww As always, Andrej is the best in explaining Computer Vision :)
@kintaro182
@kintaro182 4 жыл бұрын
there should be a "game" in model 3 where people can tag things like traffic lights and other unsolved obstacles manually so the AI is learning from as many humans as possible. Maybe you reward them with free supercharging or something. 🤘😜
@cleberz8072
@cleberz8072 4 жыл бұрын
I think we all do that manually every time we take over and correct the autopilot when driving a Tesla. I bet Tesla uploads the data and uses the action taking as the training feedback
@mattz2729
@mattz2729 4 жыл бұрын
This is a great idea, you should somehow share this
@dougdstecklein
@dougdstecklein 4 жыл бұрын
Cleber Zarate Disengaging autopilot does not label objects. If a deer runs across the road and the driver swerves to avoid hitting it, autopilot will disengage and the driver’s actions can be used to teach the neural net how to react in that situation. But driver input(steering, braking) does not label the object. You can teach a neural net how to properly recognize objects but it requires labeled data. So the OP is correct that this could be helpful. Whether Tesla needs help labeling objects 🤷‍♂️ .
@cleberz8072
@cleberz8072 4 жыл бұрын
@@dougdstecklein you're right, it won't label it but it will vastly reduce the amount of data to go through, as you could, for instance, know all those pictures had a red light since the car had to step on the brake coming to an intersection when no objects were in front of it. See what I'm talking about?
@JordanPriede
@JordanPriede 4 жыл бұрын
Many captchas on websites allow humans to label such data.
@JoseDiaz12
@JoseDiaz12 4 жыл бұрын
High levels of intelligence and passion always make for an astounding presentation. Andrej is the f*ing man.
@DirtyTesla
@DirtyTesla 4 жыл бұрын
Amazing stuff. I love seeing this stuff improve every few weeks first hand.
@RubenKelevra
@RubenKelevra 4 жыл бұрын
I still think they currently hold back a huge step forward due to inconsistency. Probably navigation of roundabouts and automatic traversing of intersections with yield, stop signs and traffic lights straight.
@DirtyTesla
@DirtyTesla 4 жыл бұрын
@@RubenKelevra I totally agree. We've seen Green use stop sign and traffic recognition on his S months ago and it was pretty damn good. Not flawless tho, and you can't mess around with a red light. I wish we could opt into some extreme beta program :)
@RubenKelevra
@RubenKelevra 4 жыл бұрын
@@DirtyTesla Mapillary shows a bit more in depth what they are capable of in terms of detection. kzbin.info/www/bejne/aXqsnZZmfd94iZI You can even go on their website and look at pictures other people has provided, so completely different angles, cameras, countries and climates and their detections are pretty much spot on when it comes to "where are cars? where is the road? Where are obstacles?" And the sign detection is able to identify the most important signs as far as a human would be able to, but with an average smartphone as camera.
@DogaOzgon
@DogaOzgon 3 жыл бұрын
This was very insightful, would love to get a follow up!
@MuscleTeamOfficial
@MuscleTeamOfficial 4 жыл бұрын
I saw the name and I had to click this, Andrej Karpathy is one of the greats.
@georgemavimbela
@georgemavimbela 4 жыл бұрын
I did..lol
@alexissuazo3122
@alexissuazo3122 3 жыл бұрын
Brilliant session, thanks for the info.
@handsonlabssoftwareacademy594
@handsonlabssoftwareacademy594 4 жыл бұрын
Pytorch configuration in Linux environment can take up to 3hrs esp if ur building it freshly from source. However, it's one of my favorite deep learning framework's besides keras an Tensorflow. Great presentation pls keep it up n coming.
@saminchowdhury7995
@saminchowdhury7995 4 жыл бұрын
great talk. try to give the photo of the speaker on the thumbnail. thanks
@PyTorch
@PyTorch 4 жыл бұрын
Hi Samin. Thanks for the feedback, we'll be sure to pass it along to our team!
@1989arrvind
@1989arrvind 9 ай бұрын
Andrej Karpathy exquisite technical explanation on Tesla Autopilot 👍👍👍
@maverick3069
@maverick3069 4 жыл бұрын
Brings back 231N memories!
@reportingavenues1975
@reportingavenues1975 Жыл бұрын
Anytime I think I have got a hang of how to use deep learning, get to see a video like this.
@Peter8831
@Peter8831 4 жыл бұрын
I've used Andrej's RNN techniques to do significant work in medicinal chemistry - great stuff!!!
@rangv733
@rangv733 4 жыл бұрын
Hey there. For what have you used it ?
@safekidda46
@safekidda46 4 жыл бұрын
Humble brag
@Peter8831
@Peter8831 4 жыл бұрын
@@rangv733 So, far I've used RNNs for De Novo Drug generation, ie. create new potential drugs from scratch. You can read more here - www.wildcardconsulting.dk/teaching-computers-molecular-creativity/ Another team, which uses a very similar technique provided a good visualization- github.com/MarcusOlivecrona/REINVENT/blob/master/images/celecoxib_analogues.gif
@lukelukelukeluke
@lukelukelukeluke 4 жыл бұрын
No need to set the speed to 1.25x when Andrej is doing a presentation
@sagarmeena0210
@sagarmeena0210 4 жыл бұрын
Great Presentation
@ABC2007YT
@ABC2007YT 4 жыл бұрын
Amazing design and engineering!
@kuaranir2440
@kuaranir2440 2 жыл бұрын
I love PyTorch and Tesla
@LarryPanozzo
@LarryPanozzo 4 жыл бұрын
I learned about machine learning from Andrej in a video 4 years ago whoa
@0xNameless
@0xNameless 4 жыл бұрын
He's breaking my neural net with his speech speed...
@akarmdit2267
@akarmdit2267 3 жыл бұрын
indeed the right kind of business path and beyond kudos to ALL
@gitc13
@gitc13 4 жыл бұрын
6 people who have disliked the video are from Waymo :)
@nicop6750
@nicop6750 4 жыл бұрын
Tesla is winning autonomy folks. Watch the stock price over next 2 years. Should look like a falcon heavy launch accelerating into atmosphere.
@cestlavieeee
@cestlavieeee 4 жыл бұрын
Yup. Bought in june at 199USD. Thought I was late. Now in november its 350USD.
@CreativeBuilds
@CreativeBuilds 4 жыл бұрын
Falcon heavy? I think it'll look more like Starship 😎
@nicop6750
@nicop6750 4 жыл бұрын
@@CreativeBuilds Damnit you're right! It's gonna be epic bro
@__ihexx__5654
@__ihexx__5654 4 жыл бұрын
I dunno, waymo isn't sleeping either
@SweatySockGaming
@SweatySockGaming 4 жыл бұрын
Bump
@CabrioDriving
@CabrioDriving 4 жыл бұрын
Is it speed up 3 times ? ;)
@Saad-mh8rb
@Saad-mh8rb 3 жыл бұрын
i am proud to be a python machine learner prodigy after seeing this video
@SaurabhGuptacurious
@SaurabhGuptacurious 2 жыл бұрын
the number of knowledgeable folks in comments section is just overwhelming
@MrNightLifeLover
@MrNightLifeLover 4 жыл бұрын
Are the slides available anywhere?
@matteovalenza
@matteovalenza 4 жыл бұрын
wow well done !
@eaglesofmai
@eaglesofmai 4 жыл бұрын
This guy should really be appretiated...using State-of-the-art algorithms directly into Production, its a big risk but also a big achievement...plus Tesla's approach is safer to Human eyes as certain LiDars can cause blindness.
@runvnc208
@runvnc208 4 жыл бұрын
In my opinion, this roll-out without having injuries or fatalities was one of the greatest engineering accomplishments of the last decade. However, there have been some dangerous near-misses with recent versions, and I left a comment on my other account under this video about one of them. The comment was removed. I think suppressing critical comments is dangerous and is an abuse of KZbin's moderation system.
@cleberz8072
@cleberz8072 4 жыл бұрын
The current generation of autopilot relies on the owner paying attention and be ready to intervene so I would like to know more about how these near-misses were so terrible.
@runvnc208
@runvnc208 4 жыл бұрын
@@cleberz8072 kzbin.info/www/bejne/nHzchqSQeb6qqLc Its not that they are "so terrible". Its that its a bad idea to pretend they don't happen and hide comments about them. What I was asking was for Karpathy to address that particular near miss. One big thing is actually that the owner in the video says that he believes that Tesla will automatically receive a bug report. But I have a feeling there is actually not an automatic way for Tesla to know that this disengagement was a bug rather than a normal disengagement. So at the very least, there needs to be an easy way to report these "life on the line" bugs and all Tesla owners need to be properly informed about it if/when that exists.
@RichOrElse
@RichOrElse 4 жыл бұрын
@@runvnc208 according to Elon during autopilot all driver input are considered errors, which is an automatic bug report.
@joythought
@joythought 4 жыл бұрын
Losin KZbin comments on random videos is not a conspiracy to silence you...
@runvnc208
@runvnc208 4 жыл бұрын
@@RichOrElse There is nothing to distinguish driver input in a minor situation or from a driver that likes to give unnecessary input from a life-threatening situation.
@PiduguSundeep
@PiduguSundeep 4 жыл бұрын
Fascinating
@RishabhGKoenigseggRegera
@RishabhGKoenigseggRegera 4 жыл бұрын
Is that Hwy 403 on the way to Hamilton?
@adriancabrera4870
@adriancabrera4870 4 жыл бұрын
Where can I find a written version of this?
@vishwanath-ts
@vishwanath-ts 4 жыл бұрын
PyTorch is really cool. 😎 😎 😎
@Mrwiseguy101690
@Mrwiseguy101690 4 жыл бұрын
It's crazy how the human brain can perform the task so easily, yet state of the art computers and algorithms find it very difficult.
@HimanshuAroraa
@HimanshuAroraa 4 жыл бұрын
That is kind of obvious. Technology is nothing in front of nature. This is why AI is so hyped up right now even though it is good only in a few very specific tasks.
@leonardselksnis4326
@leonardselksnis4326 4 жыл бұрын
Just remember that vision had 543 million years to evolve. Computer vision algorithms are here only for 55 years. Also extremely impressive how far we have come in such a short time.
@TheZeeray
@TheZeeray 4 жыл бұрын
This statement is true yet misleading, we can perform driving, but we don't do it well. We screw up often in many ways every time we drive a car, we all get some form of road rage and some point, we all have attention issues when it comes to driving, we're terrible at staying centered in our lanes and following the rules of the road. Driving is easy, but we're relatively bad at it. Autopilot however has none of these issues, it never gets tired, or needs a break, nor has road rage nor attention issues. It is trained and drives the car while having vastly better vision and situational awareness of the cars environment. All it takes is improving the software and convolutional neural networks as described in this video as well as a few others from Andrej
@JohnDoe-xo2yf
@JohnDoe-xo2yf 4 жыл бұрын
@@TheZeeray and it can use the turning signal! Way ahead of my fellow drivers
@stevethompson210
@stevethompson210 4 жыл бұрын
They used to say the same about adding big numbers. That day has long passed. Soon it will be the same with driving a car.
@Anonymous-nj2ow
@Anonymous-nj2ow 4 жыл бұрын
this would be a dream job, damn working on AI at Tesla..
@gregh5061
@gregh5061 3 жыл бұрын
The pay and work hours are crap though
@gametony947
@gametony947 2 жыл бұрын
@@gregh5061 how?
@gregh5061
@gregh5061 2 жыл бұрын
@@gametony947 ive had some developers talk about it to me. mechanical engineers and a few computer science professionals.
@pauljnellissery7096
@pauljnellissery7096 2 жыл бұрын
@@gregh5061 what did they say
@gregh5061
@gregh5061 2 жыл бұрын
@@pauljnellissery7096 that they're terrified of musk showing up at the office because he fires people on a whim, the work hours are way too long and the pay is way lesser than their counterparts in other companies like google, apple, microsoft etc, which have more flexible work hours and better pay ( and pretty much have the same hiring standards). Although working at tesla would look really good on my resume so i'd probably take it up if i had a chance lol
@SourceCodeProjects
@SourceCodeProjects 3 жыл бұрын
Nice tutorial, *Car pathy*
@bidhanmajhi
@bidhanmajhi 4 жыл бұрын
Is he talking really fast or there is a playback speed bug in KZbin?
@Suro_One
@Suro_One 4 жыл бұрын
Does Tesla train using their own hardware or in the cloud? And if so, how long does training take with these methods?
@DevonHensley211
@DevonHensley211 4 жыл бұрын
They use GPU's (it was in presentation). And it takes a lot of time. 70.000+ GPU hours for full stack (1 nod with 8 GPU would take more then a year. My guess is they have many nods with lots of GPU's but not sure how many. If they would have 70.000 GPU's that means they can train full stack in 1 hour (70.000 GPU's x 1 hours = 70.000 GPU hours), but that would be huge super computer. You can put around 20-ish nods in one server rack (42U) so that means one rack would have around 160 GPU cards. In order to train this network in relative fast time, lets say you have 20 rack servers that would give you 20 racks x 160 GPU's = 3200 GPU's x 24 hours per day = 76.000 GPU hours. So every 24 hours they can train network again. Network = each time they want to train / upgrade network, they would need to wait 24 hours to see if new network is better then older one. In short, they use a lot of resources to make this work, and he also talked about Dojo project. Tesla Dojo is super powerful training computer that would replace GPU's (this is my best guess). It's dedicated hardware and Dojo can possible improve performance bt factor x10-20-ish so that would means that if they need now 24 hours to train full network, it would only take 2.4 hours. This will speed up things and they can test more variations and what not.
@ipconfigrenew
@ipconfigrenew 4 жыл бұрын
In the past Tesla has used AWS to host a lot of their backend services. I know at one point it was reported that some AWS instances were mentioned being used for Autopilot training, but that was a few years ago and I haven't heard anything new since then. I suspect they are still using cloud services for now - with the plan being to move things to their new Dojo hardware once it's up and ready. They showed the development timeline for the FSD computer in their Investor Autonomy event, and if Dojo is following a similar timeline it will probably be up and ready in the next year which will line up nicely with their plans for ramping up autopilot capabilities (like the Taxi network).
@rkan2
@rkan2 4 жыл бұрын
It seems they need about one nuclear plant's hourly production to train the network once... That is quite the electricity cost. (250W*70000 hours)
@DevonHensley211
@DevonHensley211 4 жыл бұрын
@@rkan2 3200 GPU cards would be around 1 MW (give or take few, since you need to power servers as well, network hardware and everything in between, not just GPU's). 1 MW peak usage is a lot, but nuclear plants can do anywhere from 550-ish megawatts (MW) up to 4 GW-ish (that's 4000 MW). Even so 1 MW is huge amount of power usage and it makes sense for them to try to find better way to do the processing, ergo Dojo project.
@DevonHensley211
@DevonHensley211 4 жыл бұрын
@@ipconfigrenew I honestly dont know if they using AWS or dedicated clusters, I was just doing math based on some simple numbers. I dont have any inside Tesla information :) I work in industry (servers, cloud etc) so I just run the basic numbers for fun! But for sure Dojo project could make a lot of difference for them if they can build it cost effective.
@anubhavanand6573
@anubhavanand6573 4 жыл бұрын
At 8:26, when he says predictions can't regress, what does he mean ?, Any explanations/links ?
@ryman1
@ryman1 4 жыл бұрын
Basically, when adding new functionality to the autopilot, it should be tested to ensure that the existing functionality doesn't break/get worse.
@Eminosrrr
@Eminosrrr 4 жыл бұрын
He is speaking like x1.5. So I reduced the playback speed to 0.75 and it makes more sense now.
@badrekb5175
@badrekb5175 4 жыл бұрын
to understand this guy, i had to put the vid on 0.5 speed :P
@dzerres
@dzerres 4 жыл бұрын
I wish my Tesla had a "learn/train" button when I'm driving around and I KNOW that the car won't be able to handle the upcoming traffic circle, for example. I would hit "train" in advance and gather data for the next 5 minutes to be sent to Tesla for their database to watch and learn how I drove the car around the circle, dodged the incoming cars from the left and then from the right, and maneuvered the car over to take the correct exit. I was wondering if Tesla remembers or builds up a view from my car and other Teslas driving around that particular traffic circle? If not, why not?
@TheZeeray
@TheZeeray 4 жыл бұрын
The cars are "learning" whether autopilot is engaged or not, every mile you drive is being recorded by the cameras for the company to fetch thousands of different specific events and study how the car behaves in those
@jumpingblue1623
@jumpingblue1623 3 жыл бұрын
I thought there was a button exactly for this. You press it when your car didn't drive ideally.
@ivankuljismusic8895
@ivankuljismusic8895 4 жыл бұрын
MAGICAL!!!!! Elon, Andrej, Pete Bannon, etc. etc. etc., OUR ONE WORLD LOVES YOU! See you on Mars___'One World 2.0'
@SAINIVEDH
@SAINIVEDH 3 жыл бұрын
Whoa, epic !!!!
@CHAITHANYAkitta
@CHAITHANYAkitta 4 жыл бұрын
144 TOPS is 144 trillion operations per second! it is an astronomical figure that even nvidia doesnt have at that watt hours! it deserves title "insane". Imagine when you get a 300TOPS chip on a phone, laptop, watch, ipads! that is godly power..
@myRed8Rain
@myRed8Rain 4 жыл бұрын
to check out instagram :D
@tiro0oO5
@tiro0oO5 4 жыл бұрын
Jap, that is impressive
@TheLastCrankers
@TheLastCrankers 4 жыл бұрын
I would like to see how they achieved that given NVidia's best accelerators right now are 47 TOPS while consuming 2,5 times more power. It's either a breakthrough or a lie. Edit: ah, nevermind, I was looking at 5 year old gpu accelerators. NVidia doesn't say how do modern cards do in int8 TOPS, but they have around 130 TFLOPS Tensor-wise
@DanFrederiksen
@DanFrederiksen 4 жыл бұрын
Is int8 good enough?
@Lord2225
@Lord2225 4 жыл бұрын
Yes. While high accuracy is required during training, on prediction you can round the calculation to 16 bits or even 8.
@DanFrederiksen
@DanFrederiksen 4 жыл бұрын
@@Lord2225 I wonder if there is a point in that if a classifier net relies on finer precision than 8bit that it's too fragile. Maybe sigmoid invites fine balances and some things need threshold.
@Lord2225
@Lord2225 4 жыл бұрын
​@@DanFrederiksenIt makes sense. The average activation of (neurons or layers) is on close to zero and there is no large standard deviation. even using better functions than sigmoid (elu, relu). In general, you can do 8 bit multiplication and 16 bit sum if someone is worried about a problem. heartbeat.fritz.ai/8-bit-quantization-and-tensorflow-lite-speeding-up-mobile-inference-with-low-precision-a882dfcafbbd ~ you can get better results by comparing time petewarden.com/2016/05/03/how-to-quantize-neural-networks-with-tensorflow/ ~ tricks removing problems with low precision.
@Lord2225
@Lord2225 4 жыл бұрын
Only in bad models weights explode to huge numbers.
@eugenedsky3264
@eugenedsky3264 4 жыл бұрын
@@Lord2225 Also there are other ways to compress learned data: twitter _ com/NENENENENE10/status/1151530562844332033
@vancekang
@vancekang 2 жыл бұрын
Tell me you are smart without telling me you are smart: I watch Andrej at 1.25x speed
@mosesindecks
@mosesindecks 4 жыл бұрын
Wow.
@BlackHermit
@BlackHermit 3 жыл бұрын
PyTorch is quite good.
@abdoulayediallo3777
@abdoulayediallo3777 4 жыл бұрын
And what about if you use tensorflow.
@jumpingblue1623
@jumpingblue1623 3 жыл бұрын
It would probably run too hot on nvidia.
@michaellidster1389
@michaellidster1389 4 жыл бұрын
Runs away after finishing his talk
@youtubehelge5049
@youtubehelge5049 4 жыл бұрын
These are probably lightning talks.
@Allumik
@Allumik 4 жыл бұрын
It is just to keep up with the talking speed.
@AnyFactor
@AnyFactor 3 жыл бұрын
Doesn't want to answer questions
@mmanuel6874
@mmanuel6874 4 жыл бұрын
So pytorch >> tensorflow?
@kuaranir2440
@kuaranir2440 2 жыл бұрын
I guess yes
@Dr-Asim
@Dr-Asim 4 жыл бұрын
There must something wrong with the editing of this video. Is the speed set higher? Had to set the video speed to 0.75 to watch and understand.
@JohnDoe-xo2yf
@JohnDoe-xo2yf 4 жыл бұрын
I heard him in other videos, this is how he talks
@starshipcaptain4753
@starshipcaptain4753 3 жыл бұрын
Another reason why Tesla will completely dominate, glad he is on our side.
@tanyouliang
@tanyouliang 4 жыл бұрын
The end game: Operation Vacation
@MrKeilstrup
@MrKeilstrup 3 жыл бұрын
The nerdiest presentation ever! Love it
@josy26
@josy26 4 жыл бұрын
8:27 what does he mean by "make sure that none of this 1000 predictions that we make, can regress"?
@jacobholloway7653
@jacobholloway7653 4 жыл бұрын
There are 1000 things the full network is trying to do (such as label curbs, lights, other cars, is the car going to cut me off, etc.) Each of these 1000 things will have their own accurcy across their test data (labeling a light a light and not labeling a stop sign a traffic light, etc). When you regress, you are losing accuracy. So they might be at 99% accuracy in labeling stoplights, but as they train to recognize curbs for the Smart Summon feature, the network might forget something about recognizing stoplights, and that 1 prediction now has regressed to 98.5% accuracy. They want to make sure they gather more information into the network without losing anything they previously learned.
@josy26
@josy26 4 жыл бұрын
@@jacobholloway7653 Thanks Jacob! That makes sense. In this context, regress = loosing accuracy. I looked up the definition just to add: return to a former or less developed state.
@leixun
@leixun 4 жыл бұрын
*My takeaways:* 1. They use shared backbone network because if each task has its own neural network, the computation is not affordable 3:00 2. Their inference hardware 9:00
@nycandre
@nycandre 4 жыл бұрын
Any thoughts about using extra data, like from V2X /V2M sources? It would be like cheating, I know, BUT why not use what is available to train the NN even faster? I would imagine even adding V2X /V2M hardware in large cities like New York, LA, San Francisco might be cost effective.
@alanzom1503
@alanzom1503 3 жыл бұрын
I usually watch at x1.25 or even x1.5 but I had to watch x0.75 for this
@fabiolalatisnere3767
@fabiolalatisnere3767 3 жыл бұрын
I thought it was just me hehehe
@DougGrinbergs
@DougGrinbergs 4 жыл бұрын
9:01 FSD computer discussion
@Level6
@Level6 3 жыл бұрын
1. 브스스 + 브오스 + 뎁스 구현 2. OTA 구현 3. 운전자 정보로 검증 shadow mode 구현
@AmCanTech
@AmCanTech 4 жыл бұрын
Amazing technology, apple also uses python a lot for their ml projects.
@nanosmart2553
@nanosmart2553 4 жыл бұрын
@schräg schau dir das mal an als Autopilot-Tester ;)
@TaylorAlexander
@TaylorAlexander 4 жыл бұрын
Does anyone have any papers or examples of “hydra nets” like this? I want to implement a system with a few hydra heads.
@akhilkatpally4188
@akhilkatpally4188 4 жыл бұрын
I guess they have came up with that term. Look for Feature Pyramid Networks, concept seems to be same.
@zoemayne
@zoemayne 3 жыл бұрын
This is probably the first time i've had to slow a video down to 0.75 instead of speed it up
@lauriekane772
@lauriekane772 4 жыл бұрын
Tesla paygrade based on the number of times you can drop "order of magnitude" into your presentations. Paygrade escalation rate is of course an order of magnitude greater than order-of-magnitude-isms / hour * base-pay-rate
@user-tr9yx2qk5m
@user-tr9yx2qk5m 3 жыл бұрын
FUCKING AWESOME
@rkan2
@rkan2 4 жыл бұрын
It seems they need about one nuclear plant's hourly production to train the network once... That is quite the electricity cost. (250W*70000 hours)
@cleberz8072
@cleberz8072 4 жыл бұрын
if that was done in a day, the math has it would require 17MWh. According to the EIA website the smallest American nuclear plant (R.E. Ginna) with a 584MW capacity would generate over 13GWh in 24hours so seems like you're off by a few zeros. 17MWh is the equivalent to 170 Model S P100D batteries and the nuclear power plant would be able to supercharge them all in 4h using only 4.5MW, which is less than 1% of such power plant capacity. That said, they probably use way less than that thanks to the fact the multitasking he refers to in the 48GPU system likely parallelizes tasks heavily so the max power they'll need at a given time is 250W*48=12kW which is the equivalent of a basic Tesla Solar Roof.
@user-hs5qf2ij1m
@user-hs5qf2ij1m 4 жыл бұрын
Visualization about Recurrent Network can be referred here: vision.stanford.edu/pdf/KarpathyICLR2016.pdf
@prashantgupta6885
@prashantgupta6885 3 жыл бұрын
When your brain is faster than normal...you speak faster than normal
@kesavae9552
@kesavae9552 3 жыл бұрын
Watch it at 0.75x
@rashmiabbigeri3268
@rashmiabbigeri3268 4 жыл бұрын
The fact that Elon Musk companies manufacture their own components is how they can price their products a little bit lower and also have control and provide quality. Ex : SpaceX
@vikaskrishnan4018
@vikaskrishnan4018 4 жыл бұрын
He speaks so fast, I thought my video speed was 1.5x
@georgemavimbela
@georgemavimbela 4 жыл бұрын
me too..
@user-ke2ug1dk2s
@user-ke2ug1dk2s 4 жыл бұрын
Look where our badmephisto is now
@petko4733
@petko4733 4 жыл бұрын
Wait... This is badmephisto?
@mirguitarist6656
@mirguitarist6656 3 жыл бұрын
@@petko4733 yaa, he is the one that teach us on tubing, can watch him at his old tube 'badmephisto'
@dakshbhatnagar
@dakshbhatnagar 8 ай бұрын
Wondering how many cars got smashed in the process of actually getting this to work initially.
@murtazanazir9997
@murtazanazir9997 4 жыл бұрын
Who's here after lecun roasted Elon?
@hole62
@hole62 Жыл бұрын
I am humbled and beyond thankful to the Andrej Karpathy, the Ai team, and Elon Musk for providing the service of uploading human consciousness into electronics such as Teslas. #ForeverGrateful
@akzsh
@akzsh 5 ай бұрын
???
@harryandriyan6430
@harryandriyan6430 3 жыл бұрын
playback_speed = 0.75
@ahh8895
@ahh8895 2 жыл бұрын
I'm from twitter
@MarushDenchev
@MarushDenchev 4 жыл бұрын
Here is a job I can never get ... head of AI at Tesla
@blessedspear2642
@blessedspear2642 4 жыл бұрын
WOW. And people still think Waymo is ahead lol
@Dan-xl8jv
@Dan-xl8jv 4 жыл бұрын
How do you arrive to this conclusion? Does Waymo have tech talk about the stack they use?
@dansken610
@dansken610 4 жыл бұрын
Why would this video suggest that waymo is not ahead?
@BosonCollider
@BosonCollider 4 жыл бұрын
Waymo uses a completely different stack. Not comparable. But I would guess that Tesla is significantly ahead on SLAM with vision-only, while Waymo circumvents that with LIDAR for the SLAM and uses vision to complement the data from their lidar
@Dan-xl8jv
@Dan-xl8jv 4 жыл бұрын
@@BosonCollider hmm is there a video or page that gives an intro on how Waymo did their autonomous driving systems? I would hold on the thoughts that 'Tesla is much better'. Tesla's autopilot was officially ranked 2 levels below Waymo's. Using LIDAR is better than not using it. The only obvious advantage of Tesla's autopilot is that they have much much more real world data and they dared to put not finished product into test/production. (the smart summon)
@AllanSustainabilityFan
@AllanSustainabilityFan 4 жыл бұрын
@@Dan-xl8jv Waymo's implementation is not general purpose, but rather based on defining geo-fenced areas and then training their system for each individual pocket they want to use. It will do a good job for its intended niche, for example serving a campus perhaps. But Waymo's approach is not scalable on its own, it's limited to those geofenced areas and the training tailored for each of them they add. It's not meant to tackle the larger more "general purpose" challenge of a car being able to drive itself across multiple states, for example. That requires a slightly different kind of skillset on behalf of the AI in order to handle, much more complex problem to tackle, so in that regard Tesla is way ahead in the field, but the question is going to be "when" they'll be able to be able to achieve that holy grail of automation. Nobody else has all those miles of training data needed to accomplish the task, by many orders of magnitude, so if they can't do it, I don't know who else can really.
@tacitlytesla7912
@tacitlytesla7912 4 жыл бұрын
Andrej sounds so nervous
@HariKishanAlluri
@HariKishanAlluri 4 жыл бұрын
Hail Hydra!! 😂
@ondrejtkadlec6807
@ondrejtkadlec6807 4 жыл бұрын
this guy has been watching too much 0.75 x speed videos on KZbin
@k.chriscaldwell4141
@k.chriscaldwell4141 3 жыл бұрын
Now do one called _Rent-Seeking at Tesla._
@NRV44
@NRV44 4 жыл бұрын
Please make Sentry Mode on CLOUD. Tesla can make money and owners can subscribe
@alluprasad5976
@alluprasad5976 3 жыл бұрын
I thought Andrej Karpathy is 70+ years old person !
@Takeonm
@Takeonm 3 жыл бұрын
Super Interesting. If you follow his speech by mimicking it in your own thought speech mind it’s actually easy to keep up with him while taking all of this information in. Great technique
@StEvUgnIn
@StEvUgnIn 2 жыл бұрын
*Switch speed from normal to 0.75*
@ka9dgx
@ka9dgx 4 жыл бұрын
I'm interested in this subject, but something is wrong with the compression... the audio sounds like it's been fed through a time compression algorithm.... makes it unwatchable.
@Findalfen
@Findalfen 4 жыл бұрын
Unfortunately no. It looks like he just speaks that fast. But possibly the video editing made it worse, I don't know.
@MoeSalih
@MoeSalih 4 жыл бұрын
Try watching at 0.75x speed. Might help
@bidhanmajhi
@bidhanmajhi 4 жыл бұрын
0.75 x
@aimatters5600
@aimatters5600 Жыл бұрын
never knew pytorch is used in tesla
Tesla Full Self Driving explained by Andrej Karpathy
25:55
Tesla Owners Online
Рет қаралды 25 М.
Building the Software 2 0 Stack (Andrej Karpathy)
17:55
Databricks
Рет қаралды 65 М.
白天使和小丑帮助黑天使。#天使 #超人不会飞 #超人夫妇
00:42
M3GAN’s ARMY got my finger! 😱🦾 #shorts
00:10
Adam B
Рет қаралды 18 МЛН
النودلز هي دائماً الخيار الأفضل! #شورتس
00:10
Heroes of Deep Learning: Andrew Ng interviews Andrej Karpathy
15:11
Preserve Knowledge
Рет қаралды 175 М.
PyTorch vs TensorFlow | Ishan Misra and Lex Fridman
3:47
Lex Clips
Рет қаралды 101 М.
I Tried a Disney Secret Project!
11:33
Marques Brownlee
Рет қаралды 3,8 МЛН
Let's build GPT: from scratch, in code, spelled out.
1:56:20
Andrej Karpathy
Рет қаралды 4,2 МЛН
Day in the life of Andrej Karpathy | Lex Fridman Podcast Clips
12:45
Why Some Designs Are Impossible to Improve: Quintessence
33:03
Design Theory
Рет қаралды 48 М.
Tesla FULL self driving explained by an engineer (with Elon Musk)
25:55
CNET Highlights
Рет қаралды 120 М.
[1hr Talk] Intro to Large Language Models
59:48
Andrej Karpathy
Рет қаралды 1,7 МЛН
Making AI accessible with Andrej Karpathy and Stephanie Zhan
36:59
Sequoia Capital
Рет қаралды 199 М.
Робот зарядка на 65W🤖
0:36
serg1us
Рет қаралды 1,6 МЛН
Why spend $10.000 on a flashlight when these are $200🗿
0:12
NIGHTOPERATOR
Рет қаралды 16 МЛН
План хакера 🤯 #shorts #фильмы
0:59
BruuHub
Рет қаралды 931 М.