The Perfect (Data) Storm: Moving from Automation to Leverage
We examine why businesses need to provide a solid data foundation to their data-leveraging initiatives. The good news is that this same information architecture can be used for data leveraging strategies.
[Editor's note: This article was produced by Data Blueprint and published by Enterprise Management 360° under its former name, Global ETM.]
As businesses, we have all benefitted from computer automation. We are now reaching the limits of automation-based benefits and will increasingly look to data leverage as the new source of these benefits. Data, of course, plays a huge role in both of these concepts and advantageous data management correspondingly becomes a necessary but insufficient precondition to success. Information architecture patterns such as Master Data Management (MDM) provide "how’s" that correspond to business objective "wants".
In the beginning using computers for automation seemed to be the answer and for a number of decades it was. Organizations have used combinations of hardware, software, people, processes, and data to achieve amazing accomplishments. Tasks ranging from payroll to configuration management to manufacturing have all benefitted from computer automation. But now it appears that we are reaching the limits of what computer automation can do for our organizations and increasingly we are turning to data to answer the question: what next?
Management's perception of data (as our friend John Ladley is so fond of pointing out) is changing from a lubricant to a fuel. To site just one example, social networking data provides more trusted information to the average medical consumer than their physician. Before we can use data as either a fuel or a lubricant however, we need to know how to manage it. We believe data is the most powerful yet under-utilized, poorly managed resource in business today. At the same time, there are a number of specific data challenges facing technology executives and organizations are making misdirected assumptions.
- There is a quick fix: The cloud, big data, BI tools, data warehouses, etc., alone, will create innovative business opportunities
- These are one-sided issues: Data issues are an IT problem — the problem is multi-faceted and spans the organization
- Downplayed importance: Data can be managed without explicit C-level responsibility
Consider your organizational data landscape. Some data is very useful (such as names and contact information of customers) and some is not so useful (shoe sizes of those same customers -- assuming you are not a shoe store). One of the many possibly useful data management patterns that organizations are adopting is called MDM. Gartner holds that MDM is a discipline or strategy" ... where the business and the IT organization work together to ensure the uniformity, accuracy, semantic persistence, stewardship and accountability of the enterprise's official, shared master data." By consolidating its customer information into a master collection of data, organizations are now able to recognize a more complete and accurate picture of who is purchasing what products.
To illustrate, we worked with an EU-based organization that did business around the world. Its data was optimized to support its finance operations -- suitably accounting for transactions using different currencies and occurring around the globe. When it sold widgets in Spain, Egypt, and Thailand, business ran smoothly, but our client knew that many of these sales were going to other multinational organizations. This arrangement of data required the business to run, expensive, complex, and time-consuming reports to answer the questions such as what are the total sales going to multinational "Customer X" because the sales occurred in different countries using different currencies and utilized a series of contracts.
Our client was unable to tell who were their best and worst customers. By creating a data architecture capable of supporting uniformly managed customer data, our client was able to identify their best customers and renegotiate contacts based on this new customer master data. The architecture also saved them considerable resources due to fact that the MDM pattern solution replaced literally dozens of customer data stores with a single source of the truth about their customers. This is the power of data leverage that will drive our organizations in the future. However, to achieve this level of service, organizations face various specific challenges described below.
Challenge #1: Ever cheaper storage
This year the price of "flash" memory will drop below the price of traditional disk-based storage continuing it astounding drop of 30% annual decreases. The results will be that more people want to store more data and it continues to cost them less to do so. Some are fond of stating that "storage is free" and -- yes -- the costs of storage are dropping rapidly, but the cost of managing stored data continue to increase.
Challenge #2: Vastly increasing data volumes
Publication after publication (including this one) have referenced the "coming data explosion." Everywhere the trends point to growing volumes of data from increasing numbers of sources -- ranging from everyone's smartphone to networks of livestock to increasing our domain of semi-structured user-generated content from the piles of work products produced by knowledge workers. Google's Eric Schmidt is quoted as saying that we create as much information every two days as we did from the dawn of man through 2003. Consider the amount of processing that occurs when you "like" something in a social networking context, "watch" an item on an auction site, or simply message a colleague (the average teen sends 3339 texts each month).
Challenge #3: The way things have always been done
Many processes today are driven by the constraints of yesterday. Computing was first used to automate existing manual processing in an attempt to benefit from automation. The original title "data processing manager" quickly morphed into chief information officer. Unlike accounting, where the profession has had at least 7,000 years to mature, the data management profession can trace its roots back just 100 years or so. The first formal study can be traced back to the British Secret Service's attempts to manage information on resident aliens during World War I. So our job -- managing data -- has been lumped into a variety of other tasks "handled" by the CIO. A short list of these includes:
- Technical/business competencies
- Timely and effective execution
- Being collaborative
- Business knowledge
- Creating a strategic vision
- Inspire/leadership responsibilities
- Politically astute
- Business acumen
- Know of funding flows and critical levers
- Human capital management
- Talent evaluation, development, goal-setting and performance management
- Strategic-value creation
- Creating revenue-generating opportunities
- Leadership
- Influence others through consensus building, storytelling, communications, modeling
- Networking, self-promotion, negotiating, empathy
So the question becomes where does data management fit?
Challenge #4: Who "does" data quality?
In addition to data being just a fraction of the concern of most CIOs, there is another challenge: where does the responsibility for data quality lie? When searching for a root cause of generally poor data quality, it becomes apparent that there is a fundamental misperception about whose problem data quality really is. When asked, survey after survey of business professionals indicate that they believe that this function is performed by IT, who think that data quality is a function managed by the business.
While the natural tendency is to assume that the CIO takes care of data, the belief is also often that the business leader is focused on driving their business and as a result they focus on issues directly impacting the business. The belief of many senior executives is that “data quality” is an IT problem. However, there are many critical business issues that are directly impacted by missing, inaccurate, incomplete, or corrupted business data. Clearly the responsibility for this critical business function is not a primary concern for CIOs and it has fallen into the crack between business and IT.
Challenge #5: The average knowledge worker is not a good data manager
We don't teach data management skills in higher education -- we teach how to build new databases (probably the last skill we should be teaching students). Consequently, most knowledge workers do a poor job managing their own personal data. Compounding this, organizational development backlogs often lead to the creation of shadow IT functions and more data silos.
Effectively freeing up knowledge workers from the data management tasks results in tangible savings and greater than expected increases in individual and workgroup, and organizational productivities. As an additional bonus, knowledge workers produce more new ideas faster if they are able to maintain a closer, more focused concentration on their basic tasks, and this list of benefits does not take into account the reduced risk of introducing errors and acting on erroneous information also at the individual, workgroup and organizational levels.
To summarize, the sustainable success in the current and future marketplaces, across all industries, is and will be directly correlated with an organization’s capability to be agile and innovative with their data. The cold, hard truth about data is that poor data management practices are pervasive across industries and represent a significant, poor investment.
Organizations are in denial and/or lack the awareness to know the truth about their data capabilities. Furthermore, most organizations do not have plans to address the poor data practices and, more importantly, they do not have the skills to create and implement a data-driven strategic plan. So in 2012 we are faced with a number of challenges:
- Decreasing data storage costs
- Increasing data volume
- Fighting for our share of time, attention, resources to improve our organizations abilities to manage
- Resolving who is responsible for maintaining the quality of all that data
- Smart folks are not knowledgeable of data management and related, required practices.
All of this adds up to a perfect data storm. The time is now to get serious about putting time and effort into cultivating the only non-deteriorating asset that organizations possess. Let us restate that last point. All other organizations’ resources deteriorate over time: people age; money is spent; buildings and equipment wear out, but the more time and attention you pay to and the more you properly use organizational data the more valuable it becomes as an organizational resource. More important, the amount of leverage that organizations can obtain from their data increases as a direct result of investments in their data.
This is a call to action. Organizations can no longer dabble at data management and data quality -- it does not work and it is a waste of money. Organizations have to embrace the changes that are inevitable and invest in foundational data management practices or they will face a dwindling market share, greater operational costs, and unattainable opportunities. These investments easily pay for themselves. They are not the same as the multi-million dollar investments of the failed IT projects.
The foundational data management practices are actually quite simple, but that does not mean the transformation task will be easy. For example, from research we know poor data quality is an epidemic. The simple idea is enter the data right the first time -- how would implementing such a simple idea impact your organization? The framework for making this happen is simple as well, but the issue is a matter of organizational commitment and discipline, not some technology solution or vendor tools.
Success starts with organizations that take the time to learn how to think about data from a business perspective. You cannot skip this step! A little education goes a long way and a data management body of knowledge has been established. We have observed that organizations assign, almost randomly, anybody to manage their data. Do organizations ask anybody to be financial controllers -- no! Do organizations ask the IT support staff of a financial management system to manage the projected cash flows -- no!
It’s time to go back to basics and build DM from the ground up. Having a good foundational data management practice in place is by far the easiest way to win in the market place. Data leverage will more easily create the efficiencies it has always promised but more important the business insights -- i.e., 360-degree views of customers, products, and channels -- will not require a multi-year, multi-million dollar data warehouse project that is doomed to failure.
Let's leave you with something practical and return to the MDM solution pattern discussed earlier. Our renewed focus on data puts several necessary but insufficient pre-conditions on the selection of a technology. These include:
- The existence of a business architecture illustrating at a minimum, the processes and their anticipated interaction with the MDM.
- Models of data anticipated to be exchanged between MDM processes and business processes.
- Significant other metadata describing as appropriate -- data stewardship, data periodicity, data residency, access frequency/probability, etc.
These three items (metadata) are required to accurately describe a business need -- a "what." Absent these details, selection of a data architecture pattern such as MDM -- let alone purchase of a specific set of technologies from one of more vendors -- is premature because these address the "how" question and one cannot provide a correct "how" unless there exists a correct "what".
Peter Aiken, founder, Data Blueprint is widely acclaimed as one of the top ten data management authorities in the world. In addition to examining the data management practices of more than 500 organizations, he has spent multi-year immersions with organizations as diverse as the US DoD, Deutsche Bank, Nokia, Wells Fargo, the Commonwealth of Virginia, and Walmart. As President of DAMA International, his expertise in the practice is unquestioned. He has been a member of the Information Systems Department at Virginia Commonwealth University's School of Business since 1993 and jointly owns, with the University, DataBlueprint.com, an award-winning data management/IT consulting firm.
Lewis Broome, chief operating officer, Data Blueprint has over 20 years’ experience successfully managing, implementing, and leading global information technology efforts. With a foundation in data management, data engineering, and application development, Mr. Broome honed expertise in the financial services industry to create innovative solutions to solve complex business problems, and developed organizational policies, procedures and staffing to maintain the operations after implementation. Mr. Broome continues to advise clientele on strategies to maximize efficiency by analyzing organizational goals, business processes, data alignment, and IT architecture as Data Blueprint’s chief operating officer.