Catherine Olsson - Induction Heads

[@undisclosedmusic4969]

So how does "meta-learning suggest that the model is learning something new in the context" while "in-context learning" suggests something else?

Induction looks like a "static" CRF method created in the fly during the inference...and based on memorization.

[@yzz9833]

Would love to hear a philosophical take on this as well. This paper isn’t getting the recognition it deserves outside of people who understand.

[@ryz177]

People of AI should watch this!