Your company may not have a documented or formally defined data governance framework. But if data is created and used in your organization, you have a governance framework.
Whether it’s effective or not… that’s another question.
The challenge with data governance frameworks, and data governance in general, is tying together the elements of how your organization collects, manages, and archives data – then ensuring you implement a framework people will abide by.
To that end, in this post, we’ll review what a data governance framework is, why it’s important, what components it’s made up of, and how they come together.
What is a Data Governance Framework?
A data governance framework is a combination of policies, processes, and delegated responsibilities that ensures data is managed in compliance with regulations and in service of an organization’s goals. Because regulations and goals vary from business to business, data governance frameworks vary greatly even though their overall aim is the same.
For instance, implementing a bank’s data governance framework would look far different from an advertising agency’s.
Why Governance Frameworks are Important
As Daniel Teachey, SAS Insights editor, explains, the essence of data governance is getting the right people together to “Discuss what their data means, and then creating rules to manage that information.” The data governance framework documents and formalizes this discussion.
Without a data governance framework, there is no assurance of shared terminology or normalized data use. And the chances that your organization can use data effectively and legally are essentially zero. Your framework lays out the marching orders for your data governance strategy.
The Components of a Modern Data Governance Framework
The job of the governance framework is to detail a plan for “collecting, managing, securing, and storing data.”
That means that any function that involves collecting, managing, securing, and storing data is governed by the framework.
The good news about data governance plans is that they include components you likely already have in place. As mentioned in the introduction, the challenge is becoming aware of these components in your own organization and tying them together.
Frameworks should include formalized policies for implementing the following functions:
Data architecture
Data modeling and design
Data storage and operations
Data security
Data integration and interoperability
Documents and content
Reference and master data
Data warehousing and business intelligence
Metadata management
Data quality
How to Think About Creating a Data Governance Framework
According to Daniel Teachey, the “essence of data governance” is: “Getting groups to discuss what their data means, and then creating rules to manage that information over time.”
In practice, organizations take two approaches to the data governance discussion: top-down and bottom-up.
The Top-Down Approach to Data Governance
In the top-down approach, your senior team sets the direction of your data governance policies and principles. Here are its advantages and disadvantages.
Advantages
Disadvantages
More conducive to a long-term governance vision.
Implies executive sponsorship, which reduces political conflicts and resource shortages.
Implies a clearly established link between business goals and data governance.
Minimal short-term benefit to data management practitioners.
Conflicts with and disrupts teams’ routine activities.
May lead to a vague governance vision for others to implement.
Higher risk of failure due to impracticality of monitoring the program’s evolution.
No real way to know how people are using data and where it is getting synced
The Bottom-Up Approach
The bottom-up approach occurs with the implementation of a new data management project or tool. Here are its advantages and disadvantages.
Advantages
Disadvantages
Focuses practical governance efforts on existing day-to-day activities.
Helps guide prioritization of transformations to pursue based on existing day-to-day activities.
Easier to identify challenges and constraints of the existing system at the field level.
Focuses effort on smaller, more manageable projects with well-defined scope and potential benefits
May involve less executive buy-in; transformation liable to be influenced by conflicting team visions.
May increase the risk of inconsistencies in the approach since there isn’t a single, shared vision.
May lead to departments pursuing goals for their own ends rather than the company’s as a whole.
StreamSets and Data Governance
Compliance and data quality are fundamental drivers of data governance. And the point of ingestion is one of the most important places to ensure compliance and quality. If you can leverage metadata from multiple sources of data throughout your entire system, the data governance picture can come into focus. StreamSets helps you avoid the data corrosion and data loss that so often occur when data is ingested. Among other things, StreamSets gives you the ability to inspect data in motion and automatically detect and respond to schema changes. StreamSets smart data pipelines produce vital metadata that can be used by governance solutions to monitor the use of data from inception to analytics.
The post Thinking Through the Basics of Your Data Governance Framework appeared first on StreamSets.
Read MoreStreamSets
