TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

As with All Data, Big Data Needs Governance

Big data presents significant business opportunities when leveraged properly. When poorly governed or managed, big data presents significant business and technology risks.

• By Philip Russom
• December 2, 2016

In an increasingly data-driven business world, big data takes operational analytics and a 360-degree view of customers to a new level.

Big data from websites, call center applications, smartphone apps, and social media can reveal how your customers behave in diverse situations, thereby enabling modern multichannel marketing. Big data can provide larger data samples, thereby expanding existing analytics for risk, fraud, and customer base segmentation.

The great promise and relevance of new big data is that it can be leveraged in new ways to develop new insights. These new insights contribute to organizational growth, competitiveness, and operational excellence.

Enterprises face a challenge to capture and use big data within the guidelines of external regulations and internal policies for data usage, privacy, and security. When these guidelines are not honored and followed, a business runs the risk of compliance violations, which can lead to legal issues, fines, customer dissatisfaction, and poor brand loyalty.

Successful organizations manage the technology and business risks of new big data by depending on data governance best practices and on modern data management tools that support these practices. A mature and comprehensive data governance program serves and balances two general goals: business compliance and technical standards. Both are highly relevant to the governance of new big data.

Governance Goal #1: Business compliance for regulations, data privacy, and security

This goal is about the control and use of data; it focuses on reducing liability and risk related to data management. Organizations with an existing data governance program should be able to map existing governance policies to new data; in some cases, new policies or revisions of legacy guidelines may be in order.

For example, when a new customer channel opens and starts generating data, older policies about customer privacy may or may not apply. Ideally, this determination and any ensuing updates to data governance policies should be in place before data from the new channel is captured and used.

Compliance policies created by a data governance program must support business goals, for example by:

• Creating policies that reduce risks. The policies control the access, use, and movement of data deemed sensitive by external regulations (e.g., HIPAA and Basel II) or internal measures (e.g., the corporate privacy policy).


• Certifying new data to assure security, privacy, and compliance before a new application, source, or data platform (such as Hadoop) is put into production.

Governance Goal #2: Technical standards for data and data management solutions

This involves the communal creation of enterprisewide standards for data models, exchange formats, metadata, semantics, data quality metrics, and data-driven development processes.

This goal has long been about standardizing diverse enterprise data sets and making them high quality so data can easily be shared across business units. This still applies to new big data, and technical standards also help quickly assimilate new data into the broader enterprise.

Data standards from a governance committee can assist technology goals by:

• Extending your existing customer views with additional insights drawn from big data and other new data sources


• Enlarging the data samples of existing analytics applications for fraud, risk, and customer segmentation


• Enabling analytics applications that are new to you, the hot ones today being the 360-degree view of the customer, logistics optimization, and industry-specific cases such as patient outcomes in healthcare, predictive maintenance in manufacturing, or precision farming in agriculture

Governance Leads to Advantage

Both business compliance and technical standards must be respected when governing big data and other new sources of data. This includes data on both traditional and modern platforms (including Hadoop) and both operational and analytics use cases.

The goal of governing big data is to put more data into the hands of more employees in ways that result in organizational advantage without putting enterprise information at risk or breaking with compliance.

For a deeper dive into these issues, read TDWI Checklist Report: Governing Big Data and Hadoop.]