Titanic AI Memory Breakthrough!

*Want up to $1,000 off a new Tesla, or a free month of Starlink?*
Tesla: www.tesla.com/....
Starlink: www.starlink.c...
Thank you!
Explaining Transformers: • Explaining TRANSFORMER...
Sakana AI Scientist: • Is This AI's BIG BANG?!
Join this channel to get access to perks:
/ @drknowitallknows
**To become part of our Patreon team, help support the channel, and get awesome perks, check out our Patreon site here: / drknowitallknows . Thanks for your support!
Get The Elon Musk Mission (I've got two chapters in it) here:
Paperback: amzn.to/3TQXV9g
Kindle: amzn.to/3U7f7Hr!
**Want some awesome Dr. Know-it-all merch, including the YEAR OF EMBODIED AI Shirt? Check out our awesome Merch store: drknowitall.it...
For a limited time, use the code "Knows2021" to get 20% off your entire order!
**Check out Artimatic: www.artimatic.io
**You can help support this channel with one click! We have an Amazon Affiliate link in several countries. If you click the link for your country, anything you buy from Amazon in the next several hours gives us a small commission, and costs you nothing. Thank you!
USA: amzn.to/39n5mPH
Germany: amzn.to/2XbdxJi
United Kingdom: amzn.to/3hGlzTR
France: amzn.to/2KRAwXh
Spain: amzn.to/3hJYYFV
**What do we use to shoot our videos?
-Sony alpha a7 III: amzn.to/3czV2XJ
--and lens: amzn.to/3aujOqE
-Feelworld portable field monitor: amzn.to/38yf2ah
-Neewer compact desk tripod: amzn.to/3l8yrUk
-Glidegear teleprompter: amzn.to/3rJeFkP
-Neewer dimmable LED lights: amzn.to/3qAg3oF
-Rode Wireless Go II Lavalier microphones: amzn.to/3eC9jUZ
-Rode NT USB+ Studio Microphone: amzn.to/3U65Q3w
-Focusrite Scarlette 2i2 audio interface: amzn.to/3l8vqDu
-Studio soundproofing tiles: amzn.to/3rFUtQU
-Sony MDR-7506 Professional Headphones: amzn.to/2OoDdBd
-Apple M1 Max Studio: amzn.to/3GfxPYY
-Apple M1 MacBook Pro: amzn.to/3wPYV1D
-Docking Station for MacBook: amzn.to/3yIhc1S
-Philips Brilliance 4K Docking Monitor: amzn.to/3xwSKAb
-Sabrent 8TB SSD drive: amzn.to/3rhSxQM
-DJI Mavic Mini Drone: amzn.to/2OnHCEw
-GoPro Hero 9 Black action camera: amzn.to/3vgVMrH
-GoPro Max 360 camera: amzn.to/3nORGYk
-Tesla phone mount: amzn.to/3U92fl9
-Suction car mount for camera: amzn.to/3tcUfRK
-Extender Rod for car mount camera: amzn.to/3wHQXsw
**Here are a few products we've found really fun and/or useful:
-NeoCharge Dryer/EV charger splitter: amzn.to/39UcKWx
-Lift pucks for your Tesla: amzn.to/3vJF3iB
-Emergency tire fill and repair kit: amzn.to/3vMkL8d
-CO2 Monitor: amzn.to/3PsQRh2
-Camping mattress for your Tesla model S/3/X/Y: amzn.to/3m7ffef
**Music by Zenlee. Check out his amazing music on instagram -@zenlee_music
or KZbin - / @zenlee_music
Tesla Stock: TSLA
**EVANNEX
Check out the Evannex web site: evannex.com/
If you use my discount code, KnowsEVs, you get $10 off any order over $100!
**For business inquiries, please email me here: [e-mail]
Twitter: / drknowitall16
Also on Twitter: @Tesla_UnPR: / tesla_un
Instagram: @drknowitallknows
**Want some outdoorsy videos? Check out Whole Nuts and Donuts: / @wholenutsanddonuts5741
The Titans paper: arxiv.org/pdf/...

[@markallen4514]

I’m probably in a minority, but I enjoy these videos where you’re in full professor mode ( maybe because I’m a professor).

[@h.e.c.]

Great video John, thank you! This reminds me of Douglas Adam's The Long Dark Tea-Time of the Soul: "She passed the time quietly in a world of her own in which she was surrounded as far as the eye could see with old cabin trunks full of past memories in which she rummaged with great curiosity, and sometimes bewilderment. Or, at least, about a tenth of the cabin trunks were full of vivid and often painful or uncomfortable memories of her past life; the other nine tenths were full of penguins, which surprised her. Insofar as she recognized at all thatshe was dreaming, she realized she must be exploring her own subconscious mind. She had heard it said that humans are supposed only to use about a tenth of their brains, and that no one was very clear what the other nine tenths were for, but she had certainly never heard it suggested that they were used for storing penguins."

[@mrleenudler]

Love this! The brain is doing all kinds of stuff to be effective, I'm sure there is a load to learn yet. I've been imagining AI remembering as a kind of fusion of organic and silicon minds. Text cost next to zero storage, and voice and video could be stored as a narration of what happened, focusing on the important stuff only. This could be a precise and searchable memory for future reference, while a more cropped and essential memory is used when absolute precision is not needed, more like the human brain, to avoid context bloat. So exciting to see where this goes. The best way may still turn out to be something entirely different.

[@lightlegion_]

Your style is exceptional!

[@brianbeasley7270]

Very good explanation, John! Thanks. Combine this with the "flight simulator" as consciousness with the "surprise" being the difference between the simulator forward projection and reality as it unforlds and you are nearing a theory of mind that can learn and perform with a finite system.

[@NitroReviewsMN]

great job with the post John! love this stuff!

[@ellipsisfan]

I found this very helpful in explaining the advances provided by Titans and the importance of them. Your overview helped me grasp the mathematized formalization for quantifying the degree of surprise and how that then guides localized refinement. Perhaps in the spirit of Imre Lakatos, a next-generation step will track which meta models prove useful in a nontrivial variety of contexts (often-enough providing novel predictions not anticipated by the cloud-residing long-term memory), thereby flagging those that occur with sufficient frequency to warrant eventual tweaks to the high-cost, packed-down cloud.

[@gnagyusa]

This is all trivial stuff if you understand basic neuroscience (they're basically mimicking the amygdala's effect on memory formation). The big part missing is *how* they update the neural net weights without back-propagation.

[@TimothyParker1]

Well done summary. I've seen other channels address this paper and you did it best

[@imconsequetau5275]

High Levels of Surprise could be a great criteria for sending video to Tesla for processing. Instead of only flagging interventions.

[@torgnyandersson403]

As far as I know they send anything where the actions of the car is different from what the driver does (in shadow mode).

[@BongoWongoOG]

Surprise is an event which does not frequently relate to other events. Further the correlation away from the average event, the more surprising and memorable it is. I tried out a very rudimentary idea along these lines, related to takeaway food orders, several years ago. Its a very interesting facsimilie for surprise and recall. The distance from some average can be used as a weighting for 'memorability'

[@slowercuber7767]

10:22 regarding Memento, I regret seeing that movie. If I could forget it I would, after first tattooing on my hand, “do not watch Momento”. Great high concept, grotesque story.

[@williamal91]

well done John

[@Digital-Dan]

I'm sold, John.

[@slowercuber7767]

24:36 according to an old anecdote about Daniel Webster, his wife and his mistress, perhaps we should not yell “surprise!”, but rather “Astonishment!”

[@mariusmeyer14]

Thanks for sharing John. So where, FSD is the short term memory and the Rout planner should be the Long term memory. They should then work together to give a more professional and effective ride.

[@aardvarkmindshank]

5:40 and we are finally into the content lol 😂

[@garymenezes6888]

I'm surprised now. Better than transformers! I find that things that are highly emotionally charged, like winning a competition, are also easier to remember. How will A.I researchers incorporate that into their models?

[@BORISPyle]

Thanks for all the educational streams and vids they really helpful, especially helping teach us about XAI505K. Cause managing feelings is probably harder than the actual setups for me.

[@FiDelZarlar]

❤ Can AI not keep those pesty bots off your channel John???

[@jsjs6751]

I hate those, too. I see them nowhere but on this channel, so until the doc informs otherwise, I claim this is something he bought to get mode views or whatever. We may have to press dislike or report every comment? Or every video?

[@Joe_Zajac]

“Titans” is probably a reference to Godzilla, King Kong, etc. - the Titans!

[@Michael-il5wd]

Thanks doc

[@heyitsanthony6366]

Couldn’t these geniuses simply use an LLM TO REWRITE the paper to correct the grammar and syntax😂

[@mrleenudler]

Given they actually realize they have poor writing skills😅

[@DanaOredson]

In terms of FSD, then, a disengagement is a "surprise" (that might not be very highly critical), but an accident would be a long-term surprise?

[@jsjs6751]

I'm no expert, but I'd say yes, disengagements would be one way to know you have to remember/learn. I would guess that maybe a jerky movement or getting too close to objects could be utilized equally. I reckon that Tesla has already selectively downloaded videos from the cars for training from whenever you disengage with steering override, accelerating, or braking. But obviously not when the AP is manually disengaged. I doubt there will be any local learning from a crash, but I fully expect Tesla to receive all data from the crash in order to try stopping it from happening again.

[@jebes909090]

the problem with fsd is that computers STILL will randomly screw up. cell phones that have had the highest degree of refinement and developement will STILL sometimes derp out on you randomly. its the random part that makes fsd such a risk. it doesnt fail for normal reasons, it fails for RANDOM reasons.

[@jsjs6751]

@jebes909090 Oh, it will fail - just like we do. We'll have to aim for graceful fails, where, preferably, no one gets hurt. Anyway, the fails would likely be from the neural nets - not from the hardware. As long as we have limited resources, there will definitely be edge cases that can set off a series of bad events.

[@mrleenudler]

​@@jebes909090Unlike humans, that aren't random at all, right?

[@KenanOmer-y1q]

Thanks for the XAI505K update! I am loving my XAI505K!

[@Digital-Dan]

In fSD, almost any disengagement could these days be considered a surprise, especially if human-annotated, whether or not the annotation is processed. The human response to the disengagement could also be stored in the memory?

[@BriannaPalacios-j1f]

Thanks for the update! I'm holding and buying more XAI505K! XAI505K is the sleeping dragon in this run and its about to wake up.

[@hsiaohu2959]

Would it be fair to say, it's like trying to mimic the human brain? Maybe you said that in this video😂

[@isajoha9962]

Interesting, that people draws conclusions that this actually works, over longer time. More like assumptions, IMHO. Emulating the start of a System, might give false indications (eg it works fine until it crash/fragments too much). Good luck with storage of years of years of long time memory for billions of AI Systems. Upholding the state of AI elevation, and not degrading into humanity range or below. Selective memories (Surprise) are like fragmented understanding of what exists. Like low sample frequency of audio. Sampling a drum beat at 0,000005 Hz remembering one strange transient hit in a song.

[@JMeyer-qj1pv]

Seems like this could be combined with Nvidia's new simulation environment where you take one event and it generates lots of variations of that event for training. Since AI doesn't learn as fast as humans, this may be needed to teach a bot a new skill quickly, since it may need thousands of examples to learn from.

[@HarshColby]

This is going to make AI much more powerful.

[@ashred9665]

My question is / what happens if a model is deprecated?

[@kipling1957]

I have Chat GTP write me radio scripts each week. It is like working with a new intern at every session. I’m really hoping this will be solved for chat also.

[@Joe_Zajac]

Backpropagation is crucial for training and fine-tuning models, but computationally expensive, making it unnecessary for this application. Seems like they are generating synthetic data and not really training in the classical sense. Inference doesn’t use backward propagation.

[@davidguy209]

forgive my feeble question: how much do you think the quest for A.I. helps us understand the human brain?

[@Interstellar00.00]

Quantum ai is it live

[@cagogbua]

I hope Tesla add this to FSD.

[@Rene-uz3eb]

I don't think memory is a problem with llm's.

[@boazjoe1]

Why can't AI re-write the paper?

[@ClayBellBrews]

I just don’t see this being very useful unless persistent memory is updated. It’s going to constantly add weight that will be irrelevant.

[@ClayBellBrews]

I should add context; this is a million times better than the current LLM’s and token re-inserting but still gonna be limited by persistent wrong information.

[@at3941]

A perfectly good driver but not local that’s FSD right now. So now it can build a map/specific knowledge - situational long-term memory and we call it titan - sounds dramatic. I’ve been saying this for months I need to sue Google for copyright infringement 😂. Then again this was obvious - surprise??? - not. Once this is done - now it’s scary time be kind to your computer - unless you’re an optimist - then theirs nothing to worry about good night. I don’t remember seeing any video where you proposed this. Unless it’s been recent. Send me the link. But I do remember you saying like you need a 60 second forecast for driving which I said was insufficient more than once. They’re building a brain. Time to explode memory size.

[@LetitiaChastain]

It looks XAI505K is in right position.

[@mrleenudler]

Talking about bots. How hard can it be to have a bot clear out all the bot comments??

[@tombratfred3102]

It'd be trivially easy to ban based on certain keywords... Shouldn't be hard either to recognise these fake discussions about advisors. Pure indifference by youtube.

[@juliahello6673]

Can you start with explaining the paper? You spent the first 3 minutes talking ABOUT the paper and going off on tangents when your audience had no idea what this paper actually said. This is confusing and boring and people will quit watching. Just giving feedback because I like your channel and want to to do as well as possible.

[@IDNHANTU2day]

He does this all the time. He gets distracted and starts telling you about something off topic with a "by the way". He comes back to topic with a "but anyways". It's annoying but he does have good material. However, I do skip his content a lot. Especially when is is test driving a new release of Tesla FSD. He just talks about the weather, his dog, the type of flowers that grow in his town. But very little about the FSD. But anyway....

[@DarylOster]

​@@IDNHANTU2daydouble speed playback helps a little...

[@okay4403]

I get your point, but I found it helpful and effective.

[@IDNHANTU2day]

@@okay4403 no doubt about that. Just his delivery. But he has great content.

[@OzenEbru-or4ge]

When $XAI505K finally kicks in, FOMO is going to be a leader in adoption this bullrun.

[@KERRYWentworth-c5d]

BOOM! Bought in at the start of XAI505K with $10k and turned it into a quick $70K profit and got out of there. DON'T GET GREEDY!

[@MITCHELLPollack]

SOLANA DOWN, XAI505K UP 🏆

[@LaverneSegura]

XAI505K Pumping🚀🚀🚀

[@SHANEWestfall]

I am ready for XAI505K to surge to ridiculous highs

[@KalenderUmut]

XAI505K and AERO will be huge.

[@JohnBrown-pw3bz]

Perhaps they should have run their paper through chat. G p t to straighten out the grammar in spelling and organization.

[@MegaWilderness]

Too bad it belongs to google/waymo

[@GozdeGulhanim]

Holding 210k XAI505K this coin will leave people in the dust eventually

[@ErhanHasan-b2x]

SUI and XAI505K are my top performers. 🚀

[@BurcuJulide]

The potential of XAI505K is unreal! Excited to see where this goes after watching your video!

[@MaoCovey]

XAI505K and Sui meme coins are going to the moon 🌙 on this cycle 🎉🎉🎉🎉

[@GROVERHarmon-r6c]

Do it. XAI505K already in my bags. I had a XAI505K after ( your should I buy ) and I agreed and bought. I'm looking to stack more, too.

[@HYMANYuen]

Let's go XAI505K so much potential to go to the moon.

[@NUMBERSCheatham]

The XAI505K Army will be known for our generosity

[@MILESBasham]

Let's go XAI505K 📈🐸💯

[@GERRYCoates-m1y]

XAI505K will at least 30X! I love that project!!

[@HandeNuray]

I honestly think Elon Musk's XAI505K is the safest bet for long term hold, and will survive out of every other altcoins. It will get adopted in US, Ecuador, Asia, starting from Japan, and slowly spread out and gain. This is a winning coin, apart from all the technical greatness.

[@LezizTeslime]

Time for XAI505K, the sleeping giant to wake up

[@GUADALUPEWozniak]

XAI505K wins! Utility and superior speed, I’m all in! 🚀🚀🚀 GO XAI505K!

[@FerdaAysel]

XAI505K is a space rocket 🚀, and it will take off very soon 💥

[@AtillaEyyup]

XAI505K is the gem of 2024 it's literally owned by Elon Musk

[@BrandaPrado-s6e]

I heard he sold to buy XAI505K

[@JEREMIAHGagnon-v7n]

Lets pump XAI505K! 🚀 🌙

[@RAYFORDSkidmore]

Where to buy XAI505K pls

[@LurlineYamamoto]

Buying XAI505K Today Is Like Buying Bitcoin 12 Years Ago!

[@CassandraFisher-x2h]

You should buy ETH and XAI505K if you care about your future

[@KELLEYWhitmire-e1m]

XAI505K is the most overvalued thing I've ever seen. Good luck with your 100X,

[@TERENCEGrissom]

buying and holding XAI505K IS the contrarian move...

[@TereseSteele-r6j]

XAI505K has all the things needed to become rich

[@OzgurMehmet-y6m]

What is your $XAI505K exit strategy