You are covering the topic comprehensively within 5-10 mins .... That's awesome.....Thank you so much.....Appreciate it
@BigDataThoughts3 жыл бұрын
Thanks Chanakya
@ambar7523 ай бұрын
To summarize, what the Datamarts are for a DataWarehouse, same are the DataMesh for a DataLake
@Watson22j Жыл бұрын
very nicely explained, Thank you! I have one doubt though, since we are storing data domains separately managed by different data owners/teams, do we still have one pipeline? or every domain works on their own pipeline?
@SQLwithManoj Жыл бұрын
Nice introductory video about #DataMesh...You explained about how the data is managed here, but can you provide some idea on how the data will be served or retrieved from diff domains?
@AmitGupta-ok2wq Жыл бұрын
Shreya what Cloud Data Mesh Platforms available today for enterprise grade today to manage TB's of data and scalable upto petabytes
@kannanlg49793 жыл бұрын
But once these data domains and it's corresponding ownership is created, how will it serve data scientists who need collective raw data to work with.. I see it's similar to have different schemas per domain in a data warehouse.. I kind of see a lot of overlap.. probably the only difference is the ETL can be handled by the domain team or can still be managed by a centralised team based on needs.. also how can it complement with some thing like a data lakehouse?
@chandrashekharkotekar84533 жыл бұрын
Hi Shreya, thanks for the video. How do we join data from multiple data domains? Usual ETL process or something else?
@BigDataThoughts3 жыл бұрын
In data mesh we can get data from different domains and then we can join and utilizes as per need
@Flavoursandfrequencies3 жыл бұрын
hey chandra , we creat GUID to join different domain data .if a consumer would want data from different domain for specific then based on guid the data is extractes from different domains
@sahil0094 Жыл бұрын
Can you pls make a video of implementation of data mesh, as per comments I see people still have a lot of doubts like how will data inter domain will be accessed, will there be duplication etc
@pushkarchavan2 жыл бұрын
Are we going to the old way again having separate data (lake or function tables) connected by primary or foriegn keys relationships?
@luisgutierrez924811 ай бұрын
thank you Ms.
@HariharasudanP3 жыл бұрын
Hi Shreya, what about having different schema and storing data as different domain(sql DB) compared to having data domains that you are talking about ? Any example pls to correlate and understand more ? (Thanks again for video on Monday, I thought no video for today. But you haven’t missed it. Thanks once again)
@BigDataThoughts3 жыл бұрын
We can have data in different schema or different storage. But the essence is that we are storing data domains seperately managed by different data owners/teams rather than centrally with one ELT pipeline. The concept is to not have a central storage that may not be flexible in terms of adding new data source.
@srirambalasubramanian16273 жыл бұрын
Thanks Shreya for the knowledge shared. Truly helpful. Had one question though. Is that only with the help of global governance team, one domain data can be accessed/understood by a different domain in case required. Or they have to create/duplicate a separate metadata on their own domain despite the presence of the same in a different domain instead of reusing that
@BigDataThoughts3 жыл бұрын
Thanks sriram. Data duplication may occur as there may be common datasets across domains. Metadata can be managed centrally. Here we are moving away from centralized storage to domain driven disparate data storages.
@srirambalasubramanian16273 жыл бұрын
@@BigDataThoughts , Thanks for the quick response. If metadata is central, will that Entity hold all the associated attribute from each of the domains? Or only the common ones, leaving the domain specific attributes to be defined individually
@sandiproy12792 жыл бұрын
Nicely expliained Shreya!
@BigDataThoughts2 жыл бұрын
Thanks sandip
@autodiddy2 жыл бұрын
This was crazy helpful
@BigDataThoughts2 жыл бұрын
Thanks Rahkeem
@ramdasnarayanan33453 жыл бұрын
Aren't data domains similar to data marts, where each of them is owned by a set of people?
@srirambalasubramanian16273 жыл бұрын
I suppose data marts are built on top of a CDS which is a common database, extracting only the required information specific to the domain. Wheares, here in mesh, there is no such CDS from which individual data gets pulled, rather the data is sourced directly to the domain (Only the required data specific to that domain)
@artemgri918811 ай бұрын
this is DDD concept which says to have own copy of data in each domains suitable for this domain, since 2012 by Evans. It's not new concept "data mesh".
@muralichiyan4 ай бұрын
Data mesh and snowflake same..? Data mesh and microsoft fabric same?
@jamesbenedict64802 жыл бұрын
I am more confused - in the past, various "islands" of data was causing all kinds of headache for the business to consolidate reports....with the consolidated data domains, things were much easier to rely on as they were catagorized as "single version of the truth"... well....now comes Data Mesh which is encouraging (so to speak) to seggregate the single verson of the truth and distribute is across the enterprise?
@KunalKumar-yr8bm Жыл бұрын
I am a recruiter I need a data mesh candidate for the location at connecticut if you have please let me know
@maryammoslemi2 жыл бұрын
Good comparison!
@BigDataThoughts2 жыл бұрын
Thanks Maryam
@amitgtalks6 ай бұрын
found it helpful. You may go slower though. I had to stop and rewind few times.
@mithunnambiar14332 жыл бұрын
This technology is still at it's evolving stage right? Isn't this the same that Zhamak Dehgani advocates upon?