Data Mesh 101: What is Data Mesh?

[@Confluent]

To see the entire course and the immersive hands-on experience, check out 👉cnfl.io/data-mesh-101-module-1

[@vio4jesus]

PROBLEM: When the business has questions about data ACROSS domains, which is one of (not the ONLY, but one of) the reasons data warehouses exist - to answer questions, to gain information across areas or domains; THEN - data ownership by domain becomes problematic. Even in the operational world; there are use cases where data consumption across domains is needed, and accessing data by domain and aggregating it real time is not sensible, or feasible - so you persist data from multiple domains in a single repository (hub, ODS, etc.) ~ which is very common; but this breaks your architecture.

[@shevchenkojjj]

couldn't agree more. the thing only exists when data marts are isolated. when you are querying for across domain data, then it becomes a DW again, each domain becomes a database.

[@romulus1981]

Data mesh is a reincarnation of pre DWH Data landscape. It’s a great way to create and reinforce silos under the guise of “micro” architectures driven by modern tech influences. Solution providers and engineers love this because it constrains the problem space and allows them to leverage new tech to create cool factor - in effect creating a Data terrarium. The case for it starts with “everything is a mess” and we can clean up by creating islands. And perhaps they have a market because DWH are expensive, take a long time to build and require organisational alignment to get the best value - and many orgs have a mess by default because of the inertia. I think this will run its course and then we will anyways end up with a hybrid (like we always have).

[@michaeldebellis4202]

@@romulus1981 That is something I don’t understand about Data Mesh. Most of the stuff I read regarding data these days starts with the dictum: “No more data silos” but Data Mesh almost seems to encourage data silos. I don’t think it really does, just saying that’s the impression I get sometimes.

[@trejohnson7677]

just use NATS.

[@AhmedDada]

The presentation is great but the cherry on the top is the Rocinante email!

[@bradk7462]

Warning. Last time I signed up for a Confluent course, I was unemployed and used my personal details just so I could learn about confluent and kafka. After all the trial credit ran out I actually started running up a bill, and was sent many email reminders to pay up. I sorted it all out, eventually, but a warning to everyone signing up here to make sure you understand how the charging works (particularly if you're unemployed!). If you're a newbie (which you probably are given this is a 101 course) then it can get confusing real quick. It would be great if Confluent could make this material a bit more sandboxy for the noobs trying to self learn :-)

[@masonhooten2978]

Interested in learning more about how to build a data mesh and understand the difference between a data mesh and a data fabric?

[@jamesbenedict6480]

You explained it well. Data products are connect across a network for each access. Great! How does governing those locally curated nodes?

[@DerrickGray-k5d]

Love the ref to the Expanse :D

[@trejohnson7677]

lol concepts? we don't need a description of a horse, we need the horse and the buggy.