GPT-2: Language Models are Unsupervised Multitask Learners

[@bobsalita3417]

We need more paper talkers such as Yannic. Yes, Two Minute Papers is great but there's many papers worthy of discussion, many opinions needed, and many worthy methods of analysis.

[@jcorey333]

It's such a shame that the field stagnated after this. Nothing bigger or better than GPT2. Maybe someday.

[@michaelcarlon1831]

These paper-talks are great!

[@xaxfixho]

We need more of this

[@eab4984]

Would be awesome if Yannic made the video on Byte Pair Encoding mentioned 18:30

[@ben2258]

Did you ever end up making a video that discusses byte pair encoding?

[@YannicKilcher]

Not yet :)

[@ambujmittal6824]

kzbin.info/www/bejne/Y2Gsm3ljbLR1adU&ab_channel=Rasa Here you go. :)

[@dongilseo5727]

Thanks for sharing this video. I just found that GPT2 models will be available soon at Ainize Teachable NLP for free fine-tuning.

[@dongilseo5727]

@Web Front-end You can just search with 'teachable nlp'! (it seems links are auto-deleted on youtube)

[@kumarsubham2078]

Great video! Btw, is the model released now and do we have weights available?

[@YannicKilcher]

Yes, I think so

[@harmitchhabra989]

I think a neural network is essentialy a function that we can't express explicitly... The function is fine tuned and generated uaing the training data and then the said function is passed an input that we want to know the output of and since the function was fine tuned to the dataset that we gave it we can expect a prediction of output similar to the dataset. Essentialy Nn can be used to rougly map huge pieces of data to each other and then use the mapping to obtain similar outputs for inputs whose outputs are otherwise unknown to us. Also to check wether a given input is similar to the other inouts of our dataset we can input the input on a trained neural network and then see accuracy of neural network to compare similarity of this input to training inputs. Thus can be used for a recommendation system like youtubes.

[@neuron8186]

Open Ai is more like Close Ai

[@ambujmittal6824]

How can we say that GPT is simply not overfitting since it literally has seen so much data that now any down-stream task would already have been covered in the training dataset?

[@YannicKilcher]

Not necessarily. They do deduplication of the downstream tasks

[@user-or7ji5hv8y]

Is there a good video that explains how transformers work?

[@YannicKilcher]

kzbin.info/www/bejne/n3XYnZulhpejqNE

[@Xnaarkhoo]

First ten minutes no substance - don’t have more time to waste here

[@mannacharya4088]

7:10 Got me rolling on the floor laughing

[@meditationMakesMeCranky]

Hilarious!