Data Mesh in Practice: How Europe's Leading Online Platform for Fashion Goes Beyond the Data Lake

[@SilverDog-zl9wm]

Interesting that Arif states that Zhamak was the first person to coin the term Data Mesh in her whitepaper. Gartner published an article on Data Mesh in 2016. The term has been around a while.

[@αλήθεια-σ4κ]

New paradigm, very well explained ! Summary 30:29

[@Amivit]

Going on the same journey! How do you handle number of workspaces and external hives? Is it feasible having as few workspaces as possible and using table access control, or do you want many different workspaces and then use syncing mechanisms between external hives to share the metadata? I'm a bit torn on how to do the split.

[@hrishabhg]

Do we require centralised infrastructure despite having central metastore (table meta & access)? Every derived data product can use its own infra by getting relevant accesses via metastore controls. Centralised governance on all metastores and workflow orchestration (or minimum CDC triggers) can bring the best isolation and best scaling model.

[@Gus4r4po]

Great presentation!!! I wonder if at the metadata layer you are using among other many things lake formation.

[@anuragmishra5526]

I did not understand the compulsion of keeping still the centralised service and taking your own bucket concept. A data mesh should everything be decentralised where everything is domain bounded. The interoperability and discoverability should be provided by each service rather than keeping this in centralised storage.

[@Michiel012]

I think it is because then you can still do central governance. Easier to manage access rights when the buckets are centralised.

[@hermannjoel7208]

cristal clear!!!

[@vio4jesus]

Looks good on paper ~ HOWEVER ~ most of the consumers need data ACROSS domains (or products). This is the VERY REASON we have data warehouses. Integrated data, or data across domains or products. Looks good on the back of a napkin, but uh, it's NOT practical in the real world where integrated data is needed.

[@ShravanKumar-yv4be]

You just explained what data mesh is. There are no insights as to how this was implemented.

[@vio4jesus]

Build data products QUICKLY..... uh, like we just 'add water' and the data grows on its own? LOL. My friend, you be smokin' something. Operational data ALWAYS coexists with function. It's a form or an app that creates data, generally either by selection or data entry. SOMEBODY has to write code SOMEWHERE if you want new data (for a product) that doesn't already exist.