Q&A: Managing Big Data to Improve Decisions
We look at the issues that big data poses for decision making and the best practices for managing big data to enable enterprises to make more profitable decisions.
- By James E. Powell
- July 9, 2013
How does big data affect how your organization makes decisions, and what best practices for managing big data can enable your enterprise to make more profitable decisions? For answers to these questions and more, we spoke with Madhu Nair, who leads global product marketing for SAS data management solutions. Nair’s areas of expertise include data management, governance, and decision management.
BI This Week: What market forces drive data awareness?
Madhu Nair: We hear from customers that pressure is growing from regulatory compliance, corporate governance, information security, IT modernization, and strategic enablement. CEOs often feel the pain early and know data needs to be managed as a corporate asset. The explanations of the organization’s data challenges are varied and often contradictory. Many can’t link data to business value, so the ROI isn’t clear. Too often, they determine that data management is just an IT issue when it is just as much a business issue. These pervasive views present additional challenges in data strategy development and execution.
What problems are users facing with data integration, big data, and data management?
When data management is effective, organizations can make sense of big data by applying analytics to everyday business decisions on credit, fraud risk, next-best offer, pricing, and claims approval. Data management improves the speed and accuracy of these decisions and makes them consistent and repeatable.
One problem users face is in data becoming siloed if their organization handles each information management function separately. When groups across the enterprise work with many different technologies, it limits an employee’s ability to complete cross-departmental tasks.
For example, if a modeler on the analytics team deploys multiple and complex models to a line-of-business team, most of the models may be rejected if the business team insists on simplicity. In one situation, a customer acquiesced to deploy only a small number of analytic models to the business side because each model was met with so much resistance that it required excessive time to deploy. This company and many others are now moving away from separate tools to a unified platform, and that has both reduced resistance and eased deployment. Now they deploy more and better models that result in more accurate predictions. Their improvements were immediate and remarkable. Other benefits of unified decision management include:
- Repetitive tasks are automated to increase productivity
- Staff retention grows as “exception handling” replaces manual, rote work
- Trustworthy data is useful for additional business processes
- As data powers faster decisions, fewer meetings are needed
- Teams and individuals find it easier to collaborate
An organization’s ability to use information for both strategic and operational decisions is correlated with the degree to which the data is managed. In the year ahead, we expect that many more enterprises will move to a unified decision management platform for similar improvements.
Data management standards have met the quality demands required by the business, and once again data management is front and center in today’s trend. Increasingly, data management is playing a more vital part in business. What are some top features customers are demanding this year?
We listen carefully to customer needs. Detailed data analysis is a high priority for supporting business decisions, along with increased internal reporting and information access. When asked what features, functions, and benefits they want from data management solutions, users told us (and we delivered):
- Big data functions with fast processing times
- Integrated data management tools for easy deployment of services and components within the data management environment that require less time and training.
- Business rules and workflows to enforce proper decision making across the enterprise.
- Data governance to allow tracking of data through the entire decision process. This fulfills auditing and compliance requirements and reducing operating risks. Traceability is useful for impact analysis -- a critical component of change management.
- Big data analytics and visualization to give the power of informed decision making to the people who actually make the decisions.
What are some of the issues that big data poses for decision making?
Big data initiatives can enable companies to analyze diverse data -- structured and unstructured, from internal and external sources -- and yield new insights. The biggest problem is that most organizations don’t know how to begin formulating their big data strategy because, for example, they don’t know enough about their data, they lack business support, or they lack data quality in existing systems.
Because of the time required to analyze data, some organizations either only analyze a sample of the data or they analyze their big data too seldom. Both ways, they miss relevant insights and their decision-making suffers.
Speed is often critical. Reputation management requires near-real-time data management and analysis of relevant Twitter posts. Whether tactical, day-to-day issue resolution or strategic, long-term reputation management, unstructured data represents one of the big data challenges that absolutely must be conquered if organizations are to effectively address customer issues.
If outmoded processes are used to deal with big data, the result is siloed data, inconsistent data quality, and missing metadata, and it’s all on a large scale. Sadly, the “time to decision” will grow longer, preventing organizations from effectively addressing challenges or opportunities.
What are some best practices for managing big data to improve decision making?
Analyzing big data delivers big payoffs by creating new business capabilities and helping organizations conduct current processes cheaper, faster, and more effectively.
Most organizations have yet to develop and implement a big data strategy (just 12 percent have started, according to our recent big data survey). Early adopter organizations are now adding data scientists with IT capabilities who can manipulate big data technologies. Others are looking to combine data scientist skills with those of traditional data managers. Solid knowledge of data architectures, data quality, and master data management hubs are just the beginning for firms pursuing big data as a long-term differentiator.
Big data best practices include:
- Foster strong relationships between business and IT
- Manage data as a corporate asset, owned by the enterprise
- Employ virtualization as a strategy to combat data movement
- Use in-memory to increase parallelization
- Formalize monitoring and measurement of data with data stewardship roles
- Accommodate changing big data used for decision making with agile governance
Whether big data or small data, it is effective data management that will determine which companies thrive and which ones struggle in the years to come. When you have data that’s clean, enriched, governed, and trustworthy, you have a consolidated view of every entity you deal with. Pure data is on hand to feed your applications. Your departments and decision makers eagerly await every report that your installed technologies churn out. That’s when decision makers up and down the organizational chain are equipped with insights that translate into tangible business results.