How AI/ML memorization happens: Overparameterized models

In this video, you'll learn about how model size growth and overparameterization created the ability to both generalize and memorize well. You'll investigate model size and what parameters are, while analyzing how this created the ability to memorize data and still generalize well.
Related Article to learn more or reference research: blog.kjamistan...
Series Homepage: blog.kjamistan...
What do you think about memorization in AI systems? Feel free to comment below!

[@60pluscrazy]

Good 🎉🎉🎉

[@Matlockization]

Nicely explained. I eventually believe that the whole node array of the AI will organically set up specialised areas (if allowed to ??) I didn't know that the nodes memorise stuff. But it makes sense that they would reuse their experiences for future tasks. How is this similar or different to memGPT ?