HARDCORE TECH! Tesla's DOJO Will Change the AI World--Transparently! AI Day 2

At AI Day 2022, Pete Bannon, Tesla's Chip Architect, said that Tesla is in fact a "Hardcore Technology Company." He then went on to introduce Dojo, Tesla's AI Supercomputer, and between the insane hardware engineering and the amazing software and compiler, Dojo is set to be a major foundational element of Tesla's hardcore tech culture. Sacrificing memory size for bandwidth, overcoming crazy materials issues, and making a finely tuned compiler that allows users to simply use PyTorch, the Dojo team will change not only Tesla but possibly the AI as a Service world!
**To become part of our Patreon team, help support the channel, and get awesome perks, check out our Patreon site here: / drknowitallknows . Thanks for your support!
**Want some awesome Dr. Know-it-all merch, including the AI STUDENT DRIVER Bumper Sticker? Check out our awesome Merch store: drknowitall.itemorder.com/sale
For a limited time, use the code "Knows2021" to get 20% off your entire order!
**Lift pucks for your Tesla: amzn.to/3vJF3iB
**Emergency tire fill and repair kit: amzn.to/3vMkL8d
**If you're interested in a CO2 Monitor, you can check it out here: amzn.to/3PsQRh2
**For 30% off the amazing OneCarStereo instrument display for Tesla Model 3/Y, use this link: bit.ly/3vd7NRv
**Get 20% off an awesome Ekster wallet and/or wallet locator(!), check out my affiliate link here: shop.ekster.com/DrKnowitall
**Want to get in on WeBull's awesome Crypto and stock fun? Check out this link, and get started trading stock and Crypto!
a.webull.com/i/DrKnow-it-allK...
**If you are looking to purchase a new Tesla solar roof, solar tiles or PowerWall, just click this link to get up to $500 off! www.tesla.com/referral/john11286. Thank you!
**You can help support this channel with one click! We have an Amazon Affiliate link in several countries. If you click the link for your country, anything you buy from Amazon in the next several hours gives us a small commission, and costs you nothing. Thank you!
* USA: amzn.to/39n5mPH
* Germany: amzn.to/2XbdxJi
* United Kingdom: amzn.to/3hGlzTR
* France: amzn.to/2KRAwXh
* Spain: amzn.to/3hJYYFV
**What do we use to shoot our videos?
-Sony alpha a7 III: amzn.to/3czV2XJ
--and lens: amzn.to/3aujOqE
-Feelworld portable field monitor: amzn.to/38yf2ah
-Neewer compact desk tripod: amzn.to/3l8yrUk
-Glidegear teleprompter: amzn.to/3rJeFkP
-Neewer dimmable LED lights: amzn.to/3qAg3oF
-Rode Wireless Go II Lavalier microphones: amzn.to/3eC9jUZ
-Rode NTK Studio Microphone: amzn.to/3vjMomW
-Focusrite Scarlette 2i2 audio interface: amzn.to/3l8vqDu
-Studio soundproofing tiles: amzn.to/3rFUtQU
-Sony MDR-7506 Professional Headphones: amzn.to/2OoDdBd
-Apple M1 MacBook Pro: amzn.to/3wPYV1D
-Docking Station for MacBook: amzn.to/3yIhc1S
-Philips Brilliance 4K Docking Monitor: amzn.to/3xwSKAb
-Sabrent 8TB SSD drive: amzn.to/3rhSxQM
-DJI Mavic Mini Drone: amzn.to/2OnHCEw
-GoPro Hero 9 Black action camera: amzn.to/3vgVMrH
-Suction car mount for camera: amzn.to/3tcUfRK
-Extender Rod for car mount camera: amzn.to/3wHQXsw
**Here are a few products we've found really fun and/or useful:
* NeoCharge Dryer/EV charger splitter: amzn.to/39UcKWx
* A totally fun Tesla coil toy--awesome for the whole family: amzn.to/30rvzbI
* The Humane Interface, by Jef Raskin: amzn.to/3ajM7YT
* AI Superpowers, by Kai-Fu Lee: amzn.to/3dgRSs3
* A camping mattress for your Tesla model S/3/X/Y: amzn.to/3chSRrQ
*And of course the amazing "Elon Musk: Tesla, SpaceX and the Quest for a Fantastic Future": amzn.to/36hqdmS
**Music by Zenlee. Check out his amazing music on instagram -@zenlee_music
or KZbin - / @zenlee_music
Tesla Stock: TSLA
**JOWUA:
*For an awesome wireless charging phone holder mount, check out: bit.ly/3tRMSQR
use the Promotion Code: DRKNOW5
*Tesla USB Hub: bit.ly/2PXHPit
use the Promotion Code: DRKNOW5
*Powerful wireless iPhone/Android charger for your Tesla Model 3 or Model Y, check this link: bit.ly/3b5fHSW
use the Promotion Code: DRKNOW5
**EVANNEX
Check out the Evannex web site: evannex.com/
If you use my discount code, KnowsEVs, you get $10 off any order over $100!
**For business inquiries, please email me here: [e-mail]
Twitter: / drknowitall16
Also on Twitter: @Tesla_UnPR: / tesla_un
Instagram: @drknowitallknows
Contents:
Intro: 0:00
Dojo Intro: 0:28
Hardware: 12:15
Software: 29:29
**Want some outdoorsy videos? Check out Whole Nuts and Donuts: / @wholenutsanddonuts5741

[@rickkay9548]

Companies are defined by what they do, and by what they sell. Hardcore tech company for sure.

[@fteoOpty64]

The aim is to be an "alien dreadnought" factory. Raw materials go in one end, finished products exit the other end. Inside, AI controlled robots do all the build work including cleaning up their mess. They will have rooms full of human engineers supervising the work, plus evolving designs/testing designs.

Dr. KiaKia - you are really great explainer. I should be sleeping already, but I can't stop watching!

[@AndrisVizulis]

Thanks! You do a fantastic job of ramping up those of us whose computer sci knowledge is at intermediate or even basic levels to one that at least makes us aware of the cutting edge. You also give us tools to explain what Tesla is doing to non-techies, which Elon and Tesla hate is making increasingly more difficult these days. Keep up the great work.

[@formytots0128]

Many, many thanks Dr. Gibbs for this informative video. Quite sure that Tesla employees are enjoying your content as much as we do. Good job. :)

[@coachkevinyoung]

Awesome distillation of this incredible event! 😊👍👍👍

[@rackoone]

Another way to think about it, if they proccess something that would normally take them a month and do it in a day that would be like turning 10 years into 4 months. So a lot of people are saying that human capable driving is 10 years away...

[@NickJPeros]

Thank you for making the message from AI Day much more understandable.

[@andrewradford3953]

Well explained. I follow a lot of AI developments and Tesla seems way ahead with the best suited hardware for training new models efficiently. Auto labelling, and petabytes of training data. They hold all the cards.

[@jbarvideo12]

Amazing, simple explanations you gave so that non-nerds can understand, to some degree, the complexities that Tesla has overcome in developing and producing one of the world's fastest computers to accomplish analysis and training.

[@ChrisBrengel]

Great video, thanks.

[@fractalelf7760]

What is bizarre is how the possible impact and future earnings by Dojo AI applications are just mostly ignored by the press…

[@macrumpton]

The press is kind of like a cat. Anything that does not fit into its worldview is treated as invisible. The changes coming in the next decade are too big and fast for the press to fit into their narrative. They never predict the coming disruptions, in fact they deny the changes far longer than is reasonable.

[@elangovee]

You are exceptional is translating complex topics into plain English!!!💥 I can understand what Tesla engineers are saying most of the time reasonably well ...and seeing you translate it in real time into plain English without a script is truly impressive! 👍

[@Haroldus0]

Really well done ingesting a huge topic. Im reeling! Love your work.

[@de-kat]

Once Dojo is up and running, they should bring that to market as well, as Elon said best as buyable cloud computing power.

[@billgibson2520]

Very Helpful.

[@Chainyanker007]

I was blown away a few months ago when an article said Tesla software people had defined a new variable length data type for whatever languages (Python and C++ ) they are using. This should speed up and/or optimize storage depending on what’s being done, incredible, it will become a new standard data type one of these days. This means they developed there own custom compilers for those languages, definitely not just a car company, this kind of development puts Tesla at a much higher level than legacy auto. A Tech company for sure, they make cars too, good luck trying to ‘catch up’. And this video says it all, Tesla is at a totally different level of tech ability. Don’t hold your breath for GM and Ford to catch up.

[@Accur8Bill]

Thanks for the translation, John...techspeak to peoplespeak!

[@ericchild3363]

Very good complement to the james douma videos on dojo, thank you

[@nareshshah9302]

Superb 👌👌👍👍

[@BlondynCh]

awesome !!!

[@TheWood1965]

I like your idea of changing the mission statement 👌

[@denero66]

Thanks for this video. I am in finance but very interested in this. The AI day DOJO presentation was certainly not easy to follow as a non-expert but your explanations are really help. You explain very clearly without losing people in detail. I was wondering what, if any, the remaining bottlenecks are? I didn’t hear any on AI day but I may have missed it.

[@terryeasterday580]

Scaling, right now they can't make them fast enough. I'm talking about Dojo in particular. They will prove out the design and then upgrade where needed. But they can only make so many chips a month right now.

[@kazedcat]

The obvious bottleneck is data acquisition. As FSD gets better and better it will require exponential amount of miles driven to collect enough ultra rare events to train and improve the neural network. Tesla may need to evolve or solve one shot learning for ultra rare cases where not enough data can be collected.

[@Chainyanker007]

Actually the very early days of supercomputing was even before Cray machines came on the scene. Back in the ‘60s it was IBM with its 360/90 and Control Data with its 7000 series computers. Those led to Seymour Cray’s supercomputers. My iPad has more compute power than those old supercomputers, amazing. I’ve followed supercomputers since those days, been around computers since ‘61 so have seen and experienced tremendous change, retired from IT in ‘03.

[@mayetesla]

Maye the force

[@rickkay9548]

I would love to see the final version go against a Cerberas cluster or IBM

[@randreas69]

To get the Dojo to run the Lc0 chess engine against Carlsen?

[@luism5514]

In order for it to become apparent that Tesla is a hard-core technology company and not a car company they will need to release dojo services like AWS, gaming out side the car, and obviously the Tesla robot. In addition, they’d likely have to release a phone or something else for people to not consider them a car company.

[@daveblack8831]

If Tesla can position itself as the leader in AI compute speed, it won’t matter what other companies are working on unless they develop models that are optimized to compute faster with lower bandwidth-but would also take them extra time to optimize which Tesla wouldn’t need to spend so….

[@alexforget]

Again Tesla is breaking an assumption that everyone had: we have to use slow dram around a fast chip and live within the limitations of a cluster of single accelerators. Building a fabric of low latency accelerator operating close to the clock frequency is breaking a huge barrier that’s been the standard for 20+ years.

[@AlphaCrucis]

Your Y and Z axes are swapped in your mind John? I see your 3D modeling experience overpowering your engineering preference! Haha

[@toddsmith4280]

Well Elon tamped down expectations for Dojo on the earnings report.

[@amiman23]

When studying the brain scientists found the main difference between man and animal is we have one extra layer in our brains

[@Chainyanker007]

Tesla engineers seem to have an extra layer more above us semi-nerds.

[@Chainyanker007]

Ford and GM, are you watching? Any comments?

[@williamal91]

dont you ever take a rest

[@MacEwanRobert]

Linux. Yum. Old memories 😏🦄

[@wreckinball11]

Will DOJO play Call of Duty Modern Warfare 2 at 8K resolution?

[@dwr46]

I assume that some day, you can present a neural network training task to your Optimus bot. The bot will negotiate a cost for b you and you can then agree or decline. If you agree, the bot will deliver your nn learning task to Tesla and return the result to you.

[@rickkay9548]

That would be so dope! Nn model trained and emailed

[@Johnny2Feathers]

I don’t think Pat is saying that Tesla isnt a car company but instead is just saying Tesla is also a tech company.

[@Chainyanker007]

Tesla is a tech company who also makes cars and other stuff.

[@deal2live]

It like Uber is a tech company not a taxi company!!!

[@nguyep4]

What is your case for that?

[@flutieflambert]

Question: (and forgive me for being so naive) I understand that Tesla needs a supercomputer to solve autonomous driving, but how can the supercomputer transfer that massive knowledge to a relatively tiny chip in a car? Why doesn’t the car also need a supercomputer? (Please and thanks in advance.)

[@lmcclymont]

The actual neural nets are pretty small but take a lot of compute when training with a very large amount of data. Once it is trained the weights are set and the small net is put into the car. There is also a whole host of optimisation as well in the planning to reduce the compute in the car.

[@chrismuir8403]

Because it takes a lot more processing to develop and improve the neural net software than it does to run the software. The processor on the car just has to drive the car. Dojo has to see how thousands of cars are driven over millions of miles, analyze all that drive data, and figure out how to improve it.

[@budgetaudiophilelife-long5461]

🤗👍😎💚💚💚

[@hodders9834]

My Albanian friend pronounces Elon Musk as......."Alien Mask" and he thinks "Alien Mask" is not a real person 😂

[@randreas69]

He's waiting for his people to pick him back to home. It's outside our realm nothing like we could take a look at.

[@Asterra2]

Those people taking you to task for misspeaking in your earlier video are just folks closely monitoring this situation but either anti-Tesla, anti-Musk, or working for a competitor, and are running scared over what they're seeing and hearing coming from Tesla.

[@FrunkensteinVonZipperneck]

I need more hits on my channel, so I'm changing the name to, "CLICKBAIT."

[@mb345]

Weird thumbnail!

[@michelcote]

Hard core tech company, but like 90 % of revenues come from selling cars. This is why it's misundertood

[@lmcclymont]

For now… the hard core tech started first with cars, now the focus is shifting more to batteries, AI and Optimus. Looking forward when they focus on solar production, It has been left behind forTesla standards for a while now.

[@Xx0GsaburzxX]

What’s going on with the weird looking thumbnail?

[@sofiabaldwin]

Despite the economic downturn,I'm so happy☺️. I have been earning $ 60,200 returns from my $7,000 investment every 13days.

[@noahmadlaina1412]

Trading with Benedict wong strategy is nothing new to me, I opened a trading at $5K, and I've been able to scale to $18K in two weeks

[@helenboyles886]

That's amazing, I have been hearing about Benedict wong and their genius mind in the crypto market, please how did you earn such good amount?..

[@travis4178]

I started with just 4,000 dollars now earning upto 17,000 dollars WEEKLY

[@commerceusa]

Write him on ω н α т ѕ α ρ ρ ✍️👇

[@commerceusa]

He’s active with this ＋𝟭𝟮𝟭𝟱𝟱𝟬𝟵𝟬𝟭𝟴𝟵 currently

[@abadplanner1]

Its over. The bloom is off. Still over valued by 3x. See you at 70

[@niederrheiner8468]

No, it wont. Everyone can build a big computer. But this wont bring an breakthrou in AI.

[@rickkay9548]

Will it help you spell better? If so, its a success

[@niederrheiner8468]

@@rickkay9548 No arguments so you attack the person 😀

[@andrewsaint6581]

Has Ford, VW, gm or Toyota held an AI day or even spoken about the tremendous advances in computer science and software programming they've made? Just asking for clarification on your "anyone can do what Tesla has done" comment. Honda came closest with Asimo. But even that just did what was programmed like Boston Dynamics do.

[@niederrheiner8468]

@@andrewsaint6581 "Has Ford, VW, gm or Toyota held an AI day or even spoken about the tremendous advances in computer science and software programming they've made?" No, why should they? They just present ther working products. Also there are no "tremendous advances in computer science" at Tesla.

[@rickkay9548]

@@niederrheiner8468 No it was a real question. But to the point, the world works on whats is done, not on what could be done - and ZERO car companies have done what Tesla is doing - NOT ONE. Also, people assume that just because one company can do it, they call can. That's literally NOT how engineering works. It takes tons of tech and the right people in place to make things happen. ZERO car companies are doing anything remotely close to what tesla can (which is why they all outsource almost everything). Toyota took one look at tesla's hw3 and said they can't do it. Good move on their part

[@noleftturns]

I know nothing about how AI currently works - yet as a retired engineer, I keep seeing fundamental mistakes from Tesla. I don't need to fully understand every minute detail to realize things just don't seem "right". So here is my concern with the AI approach Tesla is taking, at least for FSD: The AI path Tesla is taking just doesn't seem close to how our brains work - which I know little about, to be frank. However, sleep is very important to our intelligence - I'm guessing that our brain takes short-term memory from the day and converts it into long-term memory, which is vastly different. Current AI seems to be focused on short-term memory and ignores long-term memory. This will lead to the failure of AI, at least the way Tesla seems to be tackling it. Our long-term memory is holographic - we can try to remember something that we have little data for, and our long-term memory eventually finds it - it might take seconds to minutes to days. Still, it will find it eventually if it is in our brain. This holographic memory is the key to our intelligence - for FSD, every inch of every trip should be stored in long-term memory. That memory is "fuzzy" in that parallel searches of billions of inquiries occur in a fraction of a second. Holographic memory is extremely dense since tremendous processing power converts it from short-term to long-term holographic memory. Anyway, that's my theory of why FSD will fail, and Tesla will get nowhere with AI until they create holographic long-term memory that can be shared in a hive mind at night while the Tesla is recharging at home. There is no reason why every tesla can't have every route driven by every Tesla on board the car, which will make Level 5 autonomy feasible. Take this for what you paid for it...

[@replica1052]

(what can foresee movement is intelligence -as in from where brains origin )

[@zerix01]

My understanding is that they use multiple neural networks each trained on specific objects and scenarios. Depending on what is going on in the scene, certain NN will be brought in to handle what it sees right now and react based on how it was trained. Think of it as modular long term memory.

[@Elias_Ainsworth92]

Dojo (and the rest of tesla's supercomputers) = sleep. They are literally taking the 'short term memory' data that the cars are giving them, transforming that into many similar but different situations using 'holographic' memory, and then re-encoding that into the decision making memory that is uploaded back to the cars. Also, you seem to have this idea that FSD will need some memory of every road it can drive on. This is an incorrect assumption, it only needs similar roads. I have never been to the east cost of the US but I could manage to drive there safely. I wouldn't even need a map, just directions if I wanted to get to a specific place. Tesla is attempting to go for the same thing. Some info on how to do 'holographic' memory from an Intel paper: kzbin.info/www/bejne/aGO2oJ2qq5mamMk Tesla has public demonstrated using this and other papers like it, mostly at AI day 1

[@andrasbiro3007]

What Tesla does usually seem strange at first, but 90% of the time they are proven right, and their way becomes the industry standard. This is because they don't follow conventions and traditions, they sit down with a blank sheet of paper, and look for the optimal solution given current possibilities and limitations, while traditions are building on possibilities and limitations from decades ago. Cars don't need long term memory. What little long-term memory is involved in driving is almost entirely covered by navigation data. If anything more turns out to be necessary, Tesla will do it. No, memory is not holographic, that was a fringe theory only. It was proposed before digital computers existed, so its missing enormous amounts of knowledge we accumulated since. The huge success of artificial neural nets is especially bad for the theory. You should read some books about the human mind that are based on modern science. Actually, FSD does "sleep" and even "dream". It just doesn't happen in the car, but on supercomputers at Tesla's lab. The processing that you miss is the retraining of the neural network, we can call it sleeping. And dreaming is when they generate training samples in simulation.

[@noleftturns]

@@andrasbiro3007 I guess what I'm saying is that Tesla is using the wrong tools to solve this problem. Holographic memory is analog and not digital - our brains are not digital but 100% analog. Tesla is trying to solve an analog problem with digital tools - it won't ever work. Analog computers are needed, and I can't buy one on Amazon right now. That's what the geeks at Tesla need to be focused on - use the right tools for this job.

[@daffydaffy7673]

First 4 minutes is just waffling..got bored and stopped watching

[@andrewsaint6581]

He does it deliberately, sorts the wheat from the chaff. 😉

[@daffydaffy7673]

@@andrewsaint6581 yea...i agree..the chaf keep listening