How Gen AI Models Trained

In this video, we'll explore the fascinating process of training generative AI models like GPT. There are three main stages:
1. Generative Pre-Training:
Goal: Knowledge Creation
Process: Training on vast amounts of internet text to learn language patterns.
Resources: 12 days, ~10 TB data, ~6000 GPUs.
Outcome: Generates text, summarizes, and performs sentiment analysis.
2. Supervised Fine-Tuning (SFT):
Goal: Becoming a Helpful Assistant
Process: Fine-tuning with curated human conversation data.
Resources: 1 day, ~100K request-response pairs.
Outcome: Generates appropriate and socially acceptable responses.
3. Reinforcement Learning through Human Feedback (RLHF):
Goal: Aligning with Human Preferences
Process: Learning from real user interactions and feedback.
Resources: Weeks to months, ~100K to 1M comparison labels, 10K to 100K prompts.
Outcome: Aligns responses closely with human preferences.
Join us as we dive deep into how these stages transform a basic language model into a sophisticated conversational agent!
Newsletter:
Subscribe now to my newsletter "𝗧𝗵𝗲 𝗔𝗶 𝗗𝗶𝘀𝗰𝗼𝘃𝗲𝗿𝘆" (www.newsletter...) and get 𝗙𝗥𝗘𝗘 eBook "Ultimate Resources for Generative AI".