NVIDIA Triton Inference Server and its use in Netflix's Model Scoring Service

  Рет қаралды 4,465

Outerbounds

Outerbounds

Күн бұрын

This spring at Netflix HQ in Los Gatos, we hosted an ML and AI mixer that brought together talks, food, drinks, and engaging discussions on the latest in machine learning, infrastructure, LLMs, and foundation models.
This talk was by Amr Elmeleegy, NVIDIA, Fan Yang and Liping Peng, Netflix.

Пікірлер
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA
34:14
Understanding the LLM Inference Workload - Mark Moyou, NVIDIA
PyTorch
Рет қаралды 7 М.
vLLM on Kubernetes in Production
27:31
vLLM on Kubernetes in Production
Kubesimplify
Рет қаралды 4,7 М.
24 Часа в БОУЛИНГЕ !
27:03
24 Часа в БОУЛИНГЕ !
A4
Рет қаралды 7 МЛН
요즘유행 찍는법
0:34
요즘유행 찍는법
오마이비키 OMV
Рет қаралды 12 МЛН
Air Sigma Girl #sigma
0:32
Air Sigma Girl #sigma
Jin and Hattie
Рет қаралды 45 МЛН
Their Boat Engine Fell Off
0:13
Their Boat Engine Fell Off
Newsflare
Рет қаралды 15 МЛН
How Trump is remaking US public health, with NY Times reporter Apoorva Mandavilli
28:49
How Trump is remaking US public health, with NY Times reporter Apoorva Mandavilli
GZERO Media
Рет қаралды 2,6 М.
Fast (and Furious) Data with Metaflow
15:32
Fast (and Furious) Data with Metaflow
Outerbounds
Рет қаралды 479
Шестопалов Егор - Как мы сервинг на Triton переводили
10:19
Шестопалов Егор - Как мы сервинг на Triton переводили
ODS AI Ru
Рет қаралды 1,4 М.
Official PyTorch Documentary: Powering the AI Revolution
35:53
Official PyTorch Documentary: Powering the AI Revolution
PyTorch
Рет қаралды 203 М.
Transformers (how LLMs work) explained visually | DL5
27:14
Transformers (how LLMs work) explained visually | DL5
3Blue1Brown
Рет қаралды 4,7 МЛН
Enabling Cost-Efficient LLM Serving with Ray Serve
30:28
Enabling Cost-Efficient LLM Serving with Ray Serve
Anyscale
Рет қаралды 7 М.
3090 vs 4090 Local AI Server LLM Inference Speed Comparison on Ollama
10:07
3090 vs 4090 Local AI Server LLM Inference Speed Comparison on Ollama
Digital Spaceport
Рет қаралды 17 М.
Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM
12:21
Demo: Optimizing Gemma inference on NVIDIA GPUs with TensorRT-LLM
Google for Developers
Рет қаралды 3,3 М.
Fast LLM Serving with vLLM and PagedAttention
32:07
Fast LLM Serving with vLLM and PagedAttention
Anyscale
Рет қаралды 30 М.
A Deep Dive into NVIDIA NIM with Outerbounds
50:26
A Deep Dive into NVIDIA NIM with Outerbounds
Outerbounds
Рет қаралды 270
24 Часа в БОУЛИНГЕ !
27:03
24 Часа в БОУЛИНГЕ !
A4
Рет қаралды 7 МЛН