BERT, Knowledge Distillation, GPT and LLMs

In 2024, LauzHack organized its first bootcamp on deep learning.
Syllabus, slides, and Jupyter notebooks can be found on GitHub: github.com/Lau...
Recording links:
Day1-Part1: Introduction to Python, Training, Optimizers, Learning Rate • Introduction to PyTorc...
Day1-Part2: Improving Python and Git Workflow: • Improving Python and G...
Day2-Part1: Fully-Connected and Convolutional Neural Networks, ResNet • Fully-Connected and Co...
Day2-Part2: Recurrent Neural Networks, Normalization Techniques • Recurrent Neural Netwo...
Day3-Part1: Transformer and Attention Mechanism • Transformer and Attent...
Day3-Part2: Improved Logging with WandB, R&D Coding Techniques • Improved Logging with ...
Day4-Part1: Introduction to Deep Learning in Audio and Tasks Overview • Introduction to Deep L...
Day4-Part2: Voice Deepfake Detection and Graph Neural Networks • Voice Deepfake Detecti...
Day5-Part1: Classic CV methods, Image Recognition, and Object Detection • Classic CV methods, Im...
Day5-Part2: Object Detection and Image Segmentation • Object Detection and I...
Day6-Part1: Efficient Single-GPU Training and Distributed Deep Learning • Efficient Single-GPU T...
Day6-Part2: On-Device Learning, Domain Adaptation, and Continuous Learning • On-Device Learning, Do...
Day7-Part1: Introduction to NLP: Tokenization, Embeddings, CBOW, BERT • Introduction to NLP: T...
Day7-Part2: BERT, Knowledge Distillation, GPT and LLMs • BERT, Knowledge Distil...
Day8-Part2: Explainable AI (Intrinsic, In-Hoc, Post-Hoc Methods), Multi-Modal • Explainable AI (Intrin...
Day8-Part3: Multi-Modal Networks, Flamingo, CLIP, 3D CV • Multi-Modal Networks, ...
Day9: Fine-Tuning, Transfer Learning, Hugging Face • Fine-Tuning, Transfer ...