Stanford CS224N NLP with Deep Learning | 2023 | Lecture 11 - Natural Language Generation

For more information about Stanford's Artificial Intelligence professional and graduate programs visit: stanford.io/ai
This lecture covers:
1. What is NLG?
2. A review: neural NLG model and training algorithm
3. Decoding from NLG models
4. Training NLG models
5. Evaluating NLG Systems
6. Ethical Considerations
What is natural language generation?
Natural language generation is one side of natural
language processing. NLP =
Natural Language Understanding (NLU) +
Natural Language Generation (NLG)
NLG focuses on systems that produce fluent, coherent
and useful language output for human consumption
Deep Learning is powering next-gen NLG systems!
To learn more about this course visit: online.stanford.edu/courses/c...
To follow along with the course schedule and syllabus visit: web.stanford.edu/class/cs224n/
Xiang Lisa Li
xiangli1999.github.io/
Professor Christopher Manning
Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science
Director, Stanford Artificial Intelligence Laboratory (SAIL)
#naturallanguageprocessing #deeplearning

[@420_gunna]

Great lecture! :)

[@uraskarg710]

Great lecture! Thanks!

[@mshonle]

Ah, interesting… I had wondered about the distinction between NLU and NLP and now it makes sense! Cheers!

[@robertokalinovskyy7347]

Great lecture!

[@l501l501l]

Hi there, based on the schedule on your official course website, maybe this course should be lecture 10 and Prompting, Reinforcement Learning from Human Feedback by Jesse Mu) should be lecture 11?

[@mshonle]

Can using dropout during inference be another way to set the temperature and perform sampling? E.g., if training had a 10% dropout rate, why not apply a similar random dropout during inference? The neurons which get zeroed out could depend on some distribution, such as selecting neurons evenly or favoring the earlier layers or targeting attention heads at specific layers. One might expect the token distributions would be more varied than what beam search alone could find.

[@JQ0004]

The TA seems attends Ng`s class a lot. Seems to imitate "ok cool" a lot. 😀