L19.5.2.6 BART: Combining Bidirectional and Auto-Regressive Transformers

Slides: sebastianraschka.com/pdf/lect...
0:00 Introduction
0:33 BART. Combining Bidirectional and Auto-Regressive Transformers
2:14 BART. BERT Encoder + GPT Decoder - Noise Transformations
4:39 Noise Transformations in BART for Pre-Training on Unlabeled Data
6:19 BART Performance Under Different Noise Transformations
7:04 Fine-Tuning on Labeled Data
8:21 BART Performance for Discriminative Tasks
9:26 BART Performance for Generative Tasks
-------
This video is part of my Introduction of Deep Learning course.
Next video: • L19.5.2.7: Closing Wor...
The complete playlist: • Intro to Deep Learning...
A handy overview page with links to the materials: sebastianraschka.com/blog/202...
-------
If you want to be notified about future videos, please consider subscribing to my channel: / sebastianraschka

[@siruili7924]

The best tutorial for BART

[@kartikrustagi]

Is there a video where you explains what is exactly an encoder and what is a decoder? Sorry I am moving between videos and have not gone through the complete playlist

[@erenjaeger5907]

You mentioned that BART is not the state of the art anymore, do you have ideas as to what you would consider now state of the art for text summarization?

[@anshumansinha5874]

Hmm, yah