Running a High Throughput OpenAI-Compatible vLLM Inference Server on Modal

  Рет қаралды 1,121

Modal

Modal

Күн бұрын

Пікірлер: 9
@connor-shorten
@connor-shorten 2 ай бұрын
Incredible session!
@ModalLabs
@ModalLabs 2 ай бұрын
thanks @connorshorten6311!
@Jay-wx6jt
@Jay-wx6jt 3 ай бұрын
Keep it up charles
@ibbbyscode
@ibbbyscode 3 ай бұрын
Finally, a YT channel. 👌👏
@charles_irl
@charles_irl 3 ай бұрын
I hope not to disappoint!
@RandyRanderson404
@RandyRanderson404 3 ай бұрын
This guy LLMs.
@charles_irl
@charles_irl 3 ай бұрын
like my status if you remember the sesame street era
vLLM: Easy, Fast, and Cheap LLM Serving for Everyone - Woosuk Kwon & Xiaoxuan Liu, UC Berkeley
23:33
vLLM: Easy, Fast, and Cheap LLM Serving for Everyone - Woosuk Kwon & Xiaoxuan Liu, UC Berkeley
PyTorch
Рет қаралды 865
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mistral
30:25
Exploring the Latency/Throughput & Cost Space for LLM Inference // Timothée Lacroix // CTO Mistral
MLOps.community
Рет қаралды 16 М.
龟兔赛跑：好可爱的小乌龟#short #angel #clown
01:00
龟兔赛跑：好可爱的小乌龟#short #angel #clown
Super Beauty team
Рет қаралды 112 МЛН
Sigma baby, you've conquered soap! 😲😮‍💨 LeoNata family #shorts
00:37
Sigma baby, you've conquered soap! 😲😮‍💨 LeoNata family #shorts
LeoNata Family
Рет қаралды 36 МЛН
UFC 308 : Уиттакер VS Чимаев
01:54
UFC 308 : Уиттакер VS Чимаев
Setanta Sports UFC
Рет қаралды 899 М.
Human vs Jet Engine
00:19
Human vs Jet Engine
MrBeast
Рет қаралды 178 МЛН
MLOps on Modal
36:22
MLOps on Modal
Modal
Рет қаралды 62
host ALL your AI locally
24:20
host ALL your AI locally
NetworkChuck
Рет қаралды 1,2 МЛН
Building End to End ML Applications on Modal
51:09
Building End to End ML Applications on Modal
Modal
Рет қаралды 449
Full stack web applications in pure Python with Modal & FastHTML
43:55
Full stack web applications in pure Python with Modal & FastHTML
Modal
Рет қаралды 529
Enabling Cost-Efficient LLM Serving with Ray Serve
30:28
Enabling Cost-Efficient LLM Serving with Ray Serve
Anyscale
Рет қаралды 6 М.
vLLM on Kubernetes in Production
27:31
vLLM on Kubernetes in Production
Kubesimplify
Рет қаралды 3,5 М.
Accelerating LLM Inference with vLLM
35:53
Accelerating LLM Inference with vLLM
Databricks
Рет қаралды 6 М.
Australian UN expert lashes out at Netanyahu, shuts up Israeli reporter | Janta Ka Reporter
30:23
Australian UN expert lashes out at Netanyahu, shuts up Israeli reporter | Janta Ka Reporter
Janta Ka Reporter
Рет қаралды 403 М.
The Evolution of Multi-GPU Inference in vLLM | Ray Summit 2024
30:52
The Evolution of Multi-GPU Inference in vLLM | Ray Summit 2024
Anyscale
Рет қаралды 285
Deploy LLMs using Serverless vLLM on RunPod in 5 Minutes
14:13
Deploy LLMs using Serverless vLLM on RunPod in 5 Minutes
AI Anytime
Рет қаралды 6 М.
龟兔赛跑：好可爱的小乌龟#short #angel #clown
01:00
龟兔赛跑：好可爱的小乌龟#short #angel #clown
Super Beauty team
Рет қаралды 112 МЛН