BERT for pretraining Transformers

Next Video: • Vision Transformer for...
Bidirectional Encoder Representations from Transformers (BERT) is for pretraining the Transformer models. BERT does not need manually labeled data. BERT can use any books and web documents to automatically generate training data.
Slides: github.com/wangshusen/DeepLea...
Reference:
Devlin, Chang, Lee, and Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In ACL, 2019.

[@erdi749]

One of the most underrated Transformers tutorials on KZbin, please keep up the great work!

[@sachavanweeren9578]

This was exactly the missing detail I was looking for. Thanks for the very clear explenation!

[@darkingdarlingguy]

Very nice and clear knowledege. Thank you for your contribution.

[@md.mushfiqurrahman4925]

This is the best tutorial video on transformers. Thank you so much

[@szymonpogodzinach2495]

This is the best video on Bert.

[@johnsmith-mp4pr]

great explanation!

[@user-wr4yl7tx3w]

This is an excellent presentation

[@asifpervezpolok2243]

thank you for a great video

[@iiirannn1]

great and simple explanation

[@unicarn4475]

Learn a lot.

[@zhaoxiao2002]

excellent!

[@maj46978]

❤❤❤ One simple Query if every word has embeding, what would be embeding for CLS token and SEP token..are they just random vectors Initialized