Frameworks of Data Governance

[@MyMibo]

Late reply here, but thought I’d do a knowledge share. I’ve seen data governance handed both “well” and horribly in large and small tech and manufacturing industries. How fast the industry moves seems to play a major role in how much of the governance actually applies by means of quality control. For example, tech companies that foster the “fail fast” attitude will operate at a pace will inherently have poor controls and visibility into who has (but shouldn’t) or does not have (but should). Manufacturing seems to do this better by “process” oriented nature, but can also become overly bureaucratic and hard to make changes or additions with it having any “strategic initiative” go through a number of different gates and approvals to get anything done. These types of organizations seem to foster the “if they care enough, they’ll push it through” types attitude. Of course these are over-generalizations and there a pile of contractors crammed in to make it all possible in every in between scenario (who should honestly be full time employees data architects and data engineers - different issue with the industry). The absence of awareness among business leaders (i.e. anybody who approves purchase orders), and frankly competency, is what I’m observing as one of the major influencing causes for most organizations having poor or virtually no governance rather than too much. Mostly too much focus on “optimizing” costs. Don’t get me wrong, there is a place for optimizing, but there’s a difference between doing it knowing that there are inefficiencies in a pipeline that could save some costs to reinvest for inevitable changes that will arise and doing it because there a perception of high spend and costs need to be cut to hit some arbitrary market standard. Source: I’m in “the industry” trying to make the lives of people like us, easier.

[@SamuelLees-jv8ji]

Super interesting video. I am particularly interested in concrete examples of data product interoperability and federated computational governance within a data mesh organisation i.e. the computational enforcement of data product design principles so that engineers can understand a data source and quickly incorporate it into their end product simply based on the documentation and know that it'll work without having to explore the data set, speak to producers, etc.

[@TheR0yalBeast]

Best Data Governance I ever saw was what we built from the ground up for a client. the entire BI department. We built full internal website, data catalogue solution, satellite inmmon approach model to bring in lots of different erp systems they had. Otherwise at small scale just documenting and coding your script is usually sufficient for the more small siloed projects.

[@dudettethechicklet2828]

Brilliant

[@kaleohao2426]

I’m going to sound ignorant: what’s the difference between data and architecture and data modeling?

[@nullQueries]

There's some overlap. But I'd say architecture includes the bigger picture. What's data storage to use, pipeline strategy, do you need "big data" tools. Streaming vs batch. What platform is all this living on. And so on. While data modeling is focusing on optimizing how the data storage is structured for your specific needs.

[@kaleohao2426]

@@nullQueries That helps me a lot! Thank you for explaining that!!