What's Wrong with the Medallion Architecture?

  Рет қаралды 2,652

Databricks

Databricks

Күн бұрын

In recent years, companies have seen an explosion in adopting lakehouses and reaping the rewards, but time and time again, we hear from people that they regret the layering of their lake. The zones don't quite fit what they were trying to achieve, and no one in the company understands what ""silver"" vs. ""gold"" actually means. We need to tackle a key question: Is the Medallion Architecture right for most businesses - and how should you interpret the advice? In this session, we'll break down the different stages of data curation and talk about how it works in reality, calling on practical examples from many, many real-world implementations. We'll talk about schema evolution, data cleansing, record validation, and traditional data modeling techniques, layering them on top of our lakehouse zones so we truly understand what happens where. This session is ideal for data architects, engineers, and analysts looking to design the best platform possible!
Talk By: Simon Whiteley, CTO, Advancing Analytics
Here's more to explore:
Rise of the Data Lakehouse: dbricks.co/3NH...
Lakehouse Fundamentals Training: dbricks.co/44a...
Connect with us: Website: databricks.com
Twitter: / databricks
LinkedIn: / data…
Instagram: / databricksinc
Facebook: / databricksinc

Пікірлер: 2
@indy.bhattacharya
@indy.bhattacharya Ай бұрын
Amazing explanation and SPOT ON!! This is exactly what I have gone through establishing enterprise data environment that brings data from 80 LOB to 14 data domains with each domain having a '3 layered medallion' architecture to unify domain data ... internally it had 6 layers because that's the nature of that business
@RafaelCavalcanti84
@RafaelCavalcanti84 2 ай бұрын
Yeah, if i could summarize this video, for me, would be, always think of what is the best for the business in cost benefit. Also need to put in that equation the people involved into the development and maintenance. Every lake has your context to be in that reality, but we need to keep always looking for better approaches.
Evolution of Data Architectures and How to Build a Lakehouse
22:34
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
00:32
Family Games Media
Рет қаралды 33 МЛН
Noodles Eating Challenge, So Magical! So Much Fun#Funnyfamily #Partygames #Funny
00:33
The IMPOSSIBLE Puzzle..
00:55
Stokes Twins
Рет қаралды 194 МЛН
Event-Driven Architecture (EDA) vs Request/Response (RR)
12:00
Confluent
Рет қаралды 173 М.
Medallion Architectures...Do They Actually Work??? (Simon Whiteley)
1:40:43
Havens Consulting
Рет қаралды 1,2 М.
ML Drift: Identifying Issues Before You Have a Problem
15:25
Fiddler AI
Рет қаралды 18 М.
Building Production RAG Over Complex Documents
1:22:18
Databricks
Рет қаралды 12 М.
Databricks Cost Management: Tips and Tools to Stay Under Budget
39:09
What is Medallion Architecture? Scalable Data Lakes | 2023
20:57
Make With Data
Рет қаралды 18 М.
Power BI on Databricks Best Practices
50:51
Databricks
Рет қаралды 17 М.
Dive Deeper into Data Engineering on Databricks
34:36
Databricks
Рет қаралды 10 М.
Quilt Challenge, No Skills, Just Luck#Funnyfamily #Partygames #Funny
00:32
Family Games Media
Рет қаралды 33 МЛН