Big Data is Adding to Data Management Challenges
As the use of big data grows, the need for data management in the new environment will also grow. The combination of new data and new technology requires new data management capabilities and processes in order to realize the promised benefits.
Data quality is vital when managing products, customers, and channels. Lack of quality means that data from different sources can’t be linked, whether it’s interactions in a big data platform or transactions in a data warehouse. Big data offers many new capabilities that extend what can be done with information, but this is possible only with proper attention to the data that is being produced and used.
The diversity of data formats presents new challenges for gaining a complete and accurate view of information across the enterprise. Successful organizations are starting to govern “big data” at the point of creation (within applications, Web analytics, documents, and so on) rather than fixing data quality downstream.
Meeting these needs requires changing from a strict reliance on nightly, batch-style ETL for integration and cleanup. This shift introduces new technology and methods, which means supporting new types of workloads.