DistServe: disaggregating prefill and decoding for goodput-optimized LLM inference

  Рет қаралды 979

PyTorch

PyTorch

Күн бұрын

Пікірлер
@PyTorch
@PyTorch 2 ай бұрын
Slides available at: drive.google.com/file/d/1MDw6zBzQFc2mkgUCy09ORwFRZYb-UuyU/view
Efficient Streaming Language Models with Attention Sinks
35:50
Efficient Streaming Language Models with Attention Sinks
PyTorch
Рет қаралды 1,1 М.
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA
34:14
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA
PyTorch
Рет қаралды 4,4 М.
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
00:32
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
Family Games Media
Рет қаралды 55 МЛН
乔的审判，精灵应该上天堂还是下地狱？#shorts #Fairy#fairytales
00:58
乔的审判，精灵应该上天堂还是下地狱？#shorts #Fairy#fairytales
精灵少女
Рет қаралды 9 МЛН
“Don’t stop the chances.”
00:44
“Don’t stop the chances.”
ISSEI / いっせい
Рет қаралды 62 МЛН
家庭版踩气球挑战，妈妈竟然什么也没得到#funny #宝宝 #萌娃 #comedy
01:04
家庭版踩气球挑战，妈妈竟然什么也没得到#funny #宝宝 #萌娃 #comedy
搞笑爸爸带俩娃
Рет қаралды 10 МЛН
Ilya Sutskever: "Sequence to sequence learning with neural networks: what a decade"
24:37
Ilya Sutskever: "Sequence to sequence learning with neural networks: what a decade"
seremot
Рет қаралды 148 М.
AI Trends for 2025
7:32
AI Trends for 2025
IBM Technology
Рет қаралды 121 М.
AWS re:Invent 2024 - Deep dive into Amazon Aurora DSQL and its architecture (DAT427-NEW)
59:40
AWS re:Invent 2024 - Deep dive into Amazon Aurora DSQL and its architecture (DAT427-NEW)
AWS Events
Рет қаралды 10 М.
2024's Biggest Breakthroughs in Math
15:13
2024's Biggest Breakthroughs in Math
Quanta Magazine
Рет қаралды 546 М.
Demandbase Ditches Denormalization By Switching off ClickHouse
47:19
Demandbase Ditches Denormalization By Switching off ClickHouse
CelerData
Рет қаралды 257
Introduction to Wait-free Algorithms in C++ Programming - Daniel Anderson - CppCon 2024
1:04:42
Introduction to Wait-free Algorithms in C++ Programming - Daniel Anderson - CppCon 2024
CppCon
Рет қаралды 18 М.
Stanford Webinar - Large Language Models Get the Hype, but Compound Systems Are the Future of AI
58:06
Stanford Webinar - Large Language Models Get the Hype, but Compound Systems Are the Future of AI
Stanford Online
Рет қаралды 77 М.
Keynote: PyTorch 2.1 Technical Deep Dive - Mario, Mark, Mergen, Joe, Peng, Will, Yanan
53:46
Keynote: PyTorch 2.1 Technical Deep Dive - Mario, Mark, Mergen, Joe, Peng, Will, Yanan
PyTorch
Рет қаралды 5 М.
Anthropic MCP with Ollama, No Claude? Watch This!
29:55
Anthropic MCP with Ollama, No Claude? Watch This!
Chris Hay
Рет қаралды 15 М.
How does batching work on modern GPUs?
33:29
How does batching work on modern GPUs?
PyTorch
Рет қаралды 1,4 М.
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
00:32
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
Family Games Media
Рет қаралды 55 МЛН