Data Engineering with the Open Source Modern Data Stack (From MDS Fest '23)

[@maxisqt]

This is the coolest tech demo I've ever seen. I have wanted for so long to see an end-to-end analytics stack demo, or tutorial, and never found it. You just did it in 15 minutes, using free, open source tools I can run locally on my laptop. Absolutely incredible!

[@dagsterio]

Thanks. The Dagster capabilities are expanding with each new release.

[@VaibhavPatil-rx7pc]

Excellent information thanks

[@gauravlotekar660]

Excellent demo

[@allthatyouare]

Trying this out! This was a great overview. Thank you!!

[@simondelorean]

Awesome, you just saved me a ton of work. This tool set looks great. One thing I noticed: I had to turn up the volume quite a bit.

[@Robay146]

Great presentation.

[@pnavid]

Great job❤

[@alexzir]

Awesome!

[@IIIxwaveIII]

from your experience. assuming I have my source data in a proprietry format in a proproetry data source. what injestion tool is the easiest to use with regardg to custom connector implementation? (as there are no built in connectors to my proprietry data source) thanks

[@colton-dagster]

When it comes to proprietary ingestion, I feel like dlt is a good option. It allows you to implement your own custom logic, while keeping things relatively standard as they provide a framework for implementing sources, resources, and destinations.