LLM inference optimization: Model Quantization and Distillation

  Рет қаралды 470

YanAITalk

YanAITalk

Күн бұрын

Пікірлер
LLM inference optimization: Architecture, KV cache and Flash attention
44:06
LLM inference optimization: Architecture, KV cache and Flash attention
YanAITalk
Рет қаралды 3,2 М.
LoRA explained (and a bit about precision and quantization)
17:07
LoRA explained (and a bit about precision and quantization)
DeepFindr
Рет қаралды 65 М.
Walking on LEGO Be Like... #shorts #mingweirocks
00:41
Walking on LEGO Be Like... #shorts #mingweirocks
mingweirocks
Рет қаралды 7 МЛН
А я думаю что за звук такой знакомый? 😂😂😂
00:15
А я думаю что за звук такой знакомый? 😂😂😂
Денис Кукояка
Рет қаралды 1,5 МЛН
When u fight over the armrest
00:41
When u fight over the armrest
Adam W
Рет қаралды 29 МЛН
Уральские пельмени в Камеди 🥰 #ComedyClub #КамедиКлаб #овршоу #уральскиепельмени #тнт #харламов
00:58
Уральские пельмени в Камеди 🥰 #ComedyClub #КамедиКлаб #овршоу #уральскиепельмени #тнт #харламов
Comedy Club
Рет қаралды 2,5 МЛН
Mixture of Experts: Mixtral 8x7B
39:42
Mixture of Experts: Mixtral 8x7B
YanAITalk
Рет қаралды 240
Scaling Laws for Neural Language Models
55:12
Scaling Laws for Neural Language Models
YanAITalk
Рет қаралды 697
vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024
56:09
vLLM Office Hours - FP8 Quantization Deep Dive - July 9, 2024
Neural Magic
Рет қаралды 1,4 М.
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mistral
30:25
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mistral
MLOps.community
Рет қаралды 16 М.
Quantization vs Pruning vs Distillation: Optimizing NNs for Inference
19:46
Quantization vs Pruning vs Distillation: Optimizing NNs for Inference
Efficient NLP
Рет қаралды 23 М.
Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training
50:55
Quantization explained with PyTorch - Post-Training Quantization, Quantization-Aware Training
Umar Jamil
Рет қаралды 23 М.
Parameter-efficient Fine-tuning of LLMs with LoRA
48:25
Parameter-efficient Fine-tuning of LLMs with LoRA
YanAITalk
Рет қаралды 136
Inference Optimization Tutorial (KDD) - Making models run faster - Part 1
1:21:53
Inference Optimization Tutorial (KDD) - Making models run faster - Part 1
West Coast Machine Learning
Рет қаралды 175
GraphRAG: The Marriage of Knowledge Graphs and RAG: Emil Eifrem
19:15
GraphRAG: The Marriage of Knowledge Graphs and RAG: Emil Eifrem
AI Engineer
Рет қаралды 63 М.
Coding tutorial: LLM fine-tuning with LORA
50:58
Coding tutorial: LLM fine-tuning with LORA
YanAITalk
Рет қаралды 369
Walking on LEGO Be Like... #shorts #mingweirocks
00:41
Walking on LEGO Be Like... #shorts #mingweirocks
mingweirocks
Рет қаралды 7 МЛН