Creating our Own Kubernetes & Docker to Run Our Data Infrastructure | Modal

ABOUT THE TALK:
In this talk, Erik Bernhardsson will share how Modal starts 1000s of large containers in seconds, and what they had to do under the surface to build this. This includes a custom file system written in Rust, their own container runtime, and their own container image builder. This talk will give you an idea of how containers work along with some of the low-level Linux details underneath. We'll also talk about many infrastructure tools hold data teams back, and why they deserve faster and better tools.
ABOUT THE SPEAKER:
Erik Bernhardsson is the founder and CEO of Modal, which is an infrastructure provider for data teams. Before Modal, Erik was the CTO at Better for six years, and previously spent seven years at Spotify, building the music recommendation system and running data teams.
ABOUT DATA COUNCIL:
Data Council (www.datacounci...) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers.
Make sure to subscribe to our channel for the most up-to-date talks from technical professionals on data related topics including data infrastructure, data engineering, ML systems, analytics and AI from top startups and tech companies.
FOLLOW DATA COUNCIL:
Twitter: / datacouncilai
LinkedIn: / datacouncil-ai

[@Anhar001]

all this jank just to solve the issue which is basically Python. Just write a fully statically compiled binary and shove that on a NFS, then just use rsync between dev machines and NFS. Have a shell script watch binary file changes and relaunch when file is changed. Look ma, I just replaced entire solid with a few bash scripts 😂

[@jimshtepa5423]

10:55 what's wrong with uzbekistan?))))

[@sohansingh2022]

Legendary talk! Loved it.

[@irshviralvideo]

This guy is insanely smart. He and Akshat. Can anyone comment on which AI customers would use modal vs. pytorch lightning ?

[@chrismcgrath7610]

2nd Legendary talk, I can't remember how many years it's been since I last actually watched a tech video at 1x speed, and had my attention completely captured / enjoyed it, this was fascinating. This guy is in the Venn diagram of smart person, who knows how to properly present/communicate, and was willing to do the prep work. VS many other smart people suck at communication/presentation or aren't willing to do the prep work.

[@InquilineKea]

Is content addressing how we store data as something other than files?

[@bluebabboon]

Chad engineer