Self-classifying MNIST Digits (Paper Explained)

[@CristianGarcia]

Thanks Yannic! I found it odd that they used L2 instead of Crossentropy + "label smoothing", it should have the same regularization effect of not pushing the logits to infinity.

[@YannicKilcher]

Good point!

[@edoardoguerriero2464]

It seems to be really sensitive to the image size. If you draw a number big as the size of the window the cells start getting a hard time finding an equilibrium.

[@albertwang5974]

Great, it's a mini implement of thousand brains theory.

[@JuanColonna]

Great video and explanations. Thanks

[@etiennetiennetienne]

isn't this just a convolutional rnn? if so can't they pass a hidden state bounded between -1,1 and not the decoded logits ?

[@YannicKilcher]

yes there's certainly a lot in common

[@etiennetiennetienne]

if it is a rnn, i don't think this L2 stuff is needed, othewise people would do the same for standard rnn training. either detach the prediction from the state or feedback the probabilities after a softmax ?

[@patf9770]

There's something about this very reminiscent to Geoffrey Hinton's GLOM

[@shairuno]

Wow, I realize that a transformer in NLP is also similar to the message passing mechanism. But for a transformer, the number of passing messages is limited.

[@potatooflife8603]

It's so cute.

[@rickandelon9374]

I like your name :~)

[@RickeyBowers]

It definitely needs a "this is not a number" option.

[@sayakpaul3152]

I see many grounds to connect this to implicit neural representations.

[@herp_derpingson]

9:30 Its like the children's game called Chinese whispers? 10:30 In non-trivial problems, we might run into some bandwidth issues. 17:30 Cells deteriorating over time. Is this analogous to ageing? 26:40 is that what I think it is? :)

[@YannicKilcher]

It's like chinese whispers with backprop, whatever that translates to :D The analogy to ageing might be pretty weak, but maybe :) something like being more and more set in your ways over time ;)

[@qw4316]

Hi your explanation is good ,is there any paper about CNN

[@АлексейТучак-м4ч]

that recurrent residual convolution resembles neural ode paper

[@004307ec]

what if the signal processing part use localized convolution (with some constraint to prevent kernels from being too different from each other)

[@1998sini]

How and where do you find these interesting papers?

[@TheMazyProduction]

Follow a bunch of the researchers on twitter.

[@herp_derpingson]

Also the discord group

[@mohamedosman6740]

@@herp_derpingson group name?

[@priyamdey3298]

also keeping an eye on arxiv-sanity I guess

[@herp_derpingson]

@@mohamedosman6740 Its in the description

[@Marcos10PT]

19:44 Freudian slip

[@default632]

urinal instead or neural haha

[@DanFrederiksen]

Romanticizing without benefit?

[@spsharan2000]

Which tablet and app are you using?

[@YannicKilcher]

OneNote on an old surface

[@not_a_human_being]

now do this with brain-cells and we're done! :D

[@zhangcx93]

looks like gcn in some sense...

[@default632]

Isn't that AMD graphics architecture?