Let Your Data Flow: Three Steps to Connect Your Data to Your Decisions
Companies that learn how to join their information and decision processes efficiently will be the winners in the marketplace.
- By Kathy Williams Chiang, Andrew Roman Wells
- August 14, 2017
Many manufacturing organizations are familiar with Lean Management techniques pioneered by Toyota and popularized by James P. Womack and Daniel T. Jones in their book, Lean Thinking: Banish Waste and Create Wealth in Your Corporation. One of the core concepts in Lean methodology is the idea of creating "flow" in material movement and production processes to eliminate costly waste driven by batch processing.
Batch processing in manufacturing evolved based on the idea that producing products in large lots decreases cost per unit, increasing business value. Lean thinking maintains that the counter-intuitive concept of one-piece flow is far superior in delivering bottom-line value because it reduces hidden costs driven by the need for warehouses, transportation, and inventory management to keep track of the excess material and product generated by producing large batches.
In business reporting, analysis, and planning, similar practices of batch processing can be found. BI reporting systems churn through data warehouses each night to generate reports that are distributed by enterprise reporting tools to managers' email inboxes in the morning. Managers download the often-massive data sets to their local desktops, then spend hours carving out the subset they need for their analysis.
After days or even weeks working on their analysis, they package it into a presentation sometimes accompanied by hundreds of detail slides (in an appendix) that are seldom comprehensible and hardly read. Eventually, a management committee will review the recommendations, raise some follow-up questions that are sent back down the chain, and the process begins again. Waste analogous to that found in a manufacturing process is everywhere -- overproduction, overprocessing, waiting, and transportation.
BI, analytics, and planning processes can and should be "leaned" to create a seamless flow of information that leads to high-quality decisions. Following are the three critical steps you can implement to create flow in your reporting, analysis, and planning processes.
Step 1: Transform Your Information Architecture
The best place to start is to develop analytics data marts. Operational information systems designed to support business-critical, operational processes are not well suited to support analytics. It is tempting to try to leverage existing IT investment in transactional processing systems and enterprise data warehouses, but transactional databases are good at loading many records while extracting only a few records quickly.
Analytics typically requires extracting many records from diverse tables quickly, which can be a challenge for transactional databases. You can bolt on a BI reporting system that extracts the information, stitches it together, then aggregates it into batch reports to deliver the information -- but that only pushes the problem further down in the process, triggering hidden costs in analyst time and waiting.
You can reduce total cost of ownership and improve performance by investing in information infrastructure suited to your analytics use case. Advances in computer processing and data storage technology are driving down costs and increasing value by processing data at its granular level, something not possible in legacy systems. Batch processing aggregates and averages away valuable information. Analytics databases and data marts are designed to meet the unique needs of analytics, enabling companies to unlock the power of their transactional data quickly, easily, and cost-effectively.
New data visualization and analytics tools let users connect to analytics data marts and process information on the fly. Deploying a self-service BI system lets you tap into the intellectual power of your whole team, benefitting from diversity of thought and experience. By connecting your employees more directly to the flow of information, they will be able to ask and answer their own questions and be more invested in the solutions.
Step 2: Develop a Decision Architecture
Don't overlook the importance of the hard-to-pin-down qualitative aspects of business decisions such as choice architecture, decision theory, and behavioral economics. Don't stop at quantifying the cost and benefit of a decision. Go further to assess the probability of success or failure and confidence in the results. Managers will be able to make more nuanced, informed decisions that integrate the qualitative with the quantitative, protecting against hidden biases in the decision process such as recency, anchoring, loss aversion, confirmation, and status quo.
Integrate data science into your decision process and guide users through the analytics process. You build more knowledge value in your employee base by giving them visibility into the analytics journey, how you got there, and what assumptions were made along the way rather than simply giving them the final solution.
Step 3: Monetize Your Data
Map how your decisions have impact on your business profit and loss. Take a cold, hard look to determine if they will add value or simply generate activity and motion without impacting your bottom line. Test your assumptions and assess probability and confidence factors to make a decision based on the totality of the solution rather than examining disconnected pieces in isolation.
Achieving Flow
Rethinking your information processes, investing in enabling infrastructure, and connecting your business decisions to information flow seamlessly will unlock trapped value, eliminating costs and lost opportunities due to wasteful processes. The role of big data in the business world may have been overhyped, but it is not going away. Companies that learn how to create flow in their information and decision processes will be the winners in the marketplace.