Real-Time and Big Data Challenge Data Management Best Practices
The incremental movement toward real-time operation is the most influential trend today in data-driven IT disciplines, such as business intelligence (BI), data warehousing (DW), and data integration (DI). From a technology viewpoint, collecting, processing, and delivering data is hard enough; doing it in real time is downright herculean. To exacerbate the situation, the volume of data to be processed in real time continues to swell, due to the big data phenomenon.
When faced with the challenge of making BI/DW/DI solutions operate in real time, the temptation among technical personnel is to omit or reduce some of the best practices of data management, in hopes that the simplification will speed up processing. Likewise, new practices around analytics with big data are almost devoid of data management best practices, such as data profiling, transformation, quality, and enhancement, plus improvements to metadata and master data. The unfortunate consequence is that most data sets delivered in real time today are of minimal quality, richness, and auditability, as compared to data found in the average data warehouse. And datasets delivered as the output of big data analytics are in an even sorrier state.
In this Webinar, you will learn:
- How real time and big data have evolved into compelling business requirements
- Why satisfying these requirements shouldn’t lead to scrimping on data management practices (with a few exceptions)
- Adverse affects on the business from the poor condition of data
- Technologies and tool types for properly managing real-time data and big data
- A data management strategy based on “doing it all,” instead of omissions and trade-offs
Philip Russom, Ph.D.