Impact of 2013 Data Strategies for Data Governance Programs
Modernizing data platforms will unlock additional business value, but enterprises must ensure that data governance will safeguard their investments.
By John O'Brien, Principal and CEO, Radiant Advisors
[Editor's note: John O'Brien is leading several seminars at the TDWI World Conference in Las Vegas February 17-22, 2013.]
In 2012 we saw a major shift in data strategies that moved away from traditional business intelligence (BI) approaches and towards experimenting with and adopting emerging data technologies such as Hadoop. In 2013, following many projects and proof of concepts (such as Hadoop experiments, implementing analytic databases, and powerful desktop analytic tools), data governance programs -- formalized or not -- will have to deal with issues crossing into new areas or possibly grander in scale.
Modernizing data platforms will unlock additional business value for most organizations, allowing for new forms of data -- unstructured, semi-structured, external, and user -- and therefore will bring new governance questions from the information discovery processes of business analytics. Faster and more scalable analytic database technologies raise questions regarding new policies for safeguarding those investments. Empowering business users, analysts, and data scientists to create analytic models (which operate on mixed data sets) demands the diligent governance required by the data or business rules themselves.
Governing Performance in Strategic Analytic Technology Investments
Although some may not consider performance or data platform resource management part of a data governance program's scope, consider the recent mandate given by one company as it purchased a new high-performance analytic database and infrastructure as part of its enterprise data warehouse environment. In this case, the business sponsor stated that it was a tough to decide where to make its strategic investments, but believed the business analytics business case submitted demonstrated return on investment and long-term value. Therefore, the sponsor was holding the BI program accountable for "governing" the performance capabilities of the system. In other words: don't come back next year (or two) asking for more resources because you consumed all your capacity before the 4-year ROI's term was complete.
Extending data governance (DG) for performance policies that are beyond traditional data quality and consistency policies is required to govern performance for strategic business investments. The creation of new policies begins with the questions: "What are we protecting from happening?" and "What is the risk?"
For example, poorly written data access or transformation routines (such as SQL statements) could be inefficient and consume more computing resources than necessary. Will high-performance databases lead to less rigorous data designs and tuning? Begin with existing policies dealing with data access that could be revised; perhaps new ones are required. Who will evaluate data access routines such as SQL statements? Finally, how much tuning of a data access routine is enough? Knowing that benefits will diminish at some point will help establish standard benchmarks and procedures. New data governance policies should evaluate existing policies, expand roles and responsibilities, and establish processes and standards.
Other performance governance policies take the same approach for addressing business prioritization; for example, should all analytic routines be allowed to operate on the new analytic database platform, and how are those determinations made? Some organizations already have data retention policies for compliance reasons; however, retention is also related to performance aspects for maintaining smaller production data sets. Most important will be the procedures for defining metrics, the ongoing monitoring and reporting on the status of performance resources available for accountability.
Governing Information Discovery with Business Analysts
One of the interesting debates in 2012 was about a tenant in the business analytics (BA) movement: the requirement for "data openness" to enable business analytics. This sparked a slew of commentary recounting the history of business intelligence and data governance as being concerned with controlling and managing data consistency, semantics, and usage. Some claimed that data governance was the enemy of business analytics, which seemed like quite the stretch, though it wasn't entirely incorrect. Today's information discovery (ID) process and powerful, new (and affordable) desktop tools have enabled business analysts to pull data from anywhere inside or outside the enterprise, as well as from new user-generated data, to deliver business intelligence. Business subject matter experts with data analysis skills and a powerful tool equals new agile business value.
Keep in mind that tools and technologies require methodologies, processes, best practices, IT, or data governance. It's a bad sign when a term such as "shadow IT" becomes accepted nomenclature without hesitation. However, data governance programs expanded in 2012, focusing on user access and authorization policies as a form of governance. A strong message -- "it's not about preventing access but about ensuring proper access by individuals" -- was communicated. Much more education about the types of data, its responsibilities, and usage was also developed to expand the role and responsibility definitions for business analysts and data scientists. In some cases, a new enterprise analytics and information discovery development methodology was developed that allowed for the open, agile, and iterative nature of analytics, while coupling it with the data governance program and BI development methodology for enterprise consistency and accountability.
Governing Other Analytic Assets Beyond Data Owners
The definition of business analytics still means different things to different organizations; however, for the sake of data governance programs, we'll define business analytics as the advanced, statistical algorithms or models that operate on a given set of data to infer relationships and/or significance.
In this definition, business analytics projects contain two deliverables if they are to be used regularly in a production environment. First is a refreshed set of related data variables from various data sources -- likely cleansed, modified, or derived. Second is an analytic model -- a statistical method -- varying from simple to complex in design. These analytic models output a prediction through probability, affinity, and confidence. Over time, an analytic model can "weaken" its prediction if the variables used lose their associations or if the model simply no longer represents the changing business environment accurately.
Analytic governance is a relatively new area that focuses on issues arising from the new data strategies that incorporate analytic projects in addition to business intelligence projects. For example, if a customer-scoring model routinely runs and delivers the calculated value for every customer to operational systems and business decisions, shouldn't its statistical model be managed alongside the business rules and definition of other enterprise data?
Our concern for 2013 is that "with great power comes great responsibility" and without governance programs extending to address business analytics, additional risks may be at hand. Once again, defining procedures for development, verification, and ongoing management of analytic models will be necessary.
Data Governance Frameworks are the Path Forward
Whether you have a mature, new, or even no data governance program in place, adopting data strategies that involve new technologies, new data, and business analytics processes will create some degree of risk for your organization. Although analytic centers of excellence (CoE) are great for community building and mentorship, don't rely on that organizational structure for governance responsibilities. Data governance programs are well established and proven to manage risk and data, and a good data governance framework will accommodate the ongoing need to evolve the program in an organization.
John O'Brien, CBIP is principal and CEO of Radiant Advisors and a recognized architecture and analytics visionary. You can contact the author at [email protected].