To deliver data in support their business intelligence needs, most organizations have developed a classic data warehouse. New groups of business users with data science and data investigative needs have also developed the data lake, which is most often a stand-alone system with almost no relationship to the existing data warehouse. On top of this, some organizations have already started to develop a third data delivery system for delivering data to specific business users—the data marketplace. Again, this third data delivery system is being developed as an analytical island, not to mention the still-newer world of streaming analytics.
Developing all these data delivery systems independently is far from ideal. Development-wise, wheels are reinvented, resulting in low productivity, metadata replication, and inconsistencies across reports and analyses. It’s crucial for organizations to somehow bring these systems together. One solution is by deploying a data delivery architecture based on data virtualization technology. Such an architecture can support a wide range of business users, from those demanding a highly agile environment such as the marketplace to those requiring governable and auditable reports. This session will discuss how these environments can be merged into one unified architecture.
You Will Learn
- The different characteristics are of data warehouse, data lake, data marketplace, data service, and data streaming architectures
- The risks are of nonintegrated data delivery systems—inconsistent reporting, high development and maintenance costs, and inflexibility
- The value is of transformation, integration, cleansing, and aggregation specifications
- The benefits of a logical data warehouse architecture and a logical data lake versus those of a physical data warehouse and a physical data lake
- The challenges of developing data lakes—is copying data centrally practical and feasible?
- What a hyper-distributed data architecture is
- The pros and cons of available data virtualization products and how they can be used to develop one integrated architecture
- The new principles for data architectures—e.g., virtual, abstraction, and technology agnostic
- The role of big data technologies such as Hadoop, Kafka, Spark, and NoSQL in future architectures
- How to integrate multiple data delivery systems to form one unified data delivery platform
- Business intelligence specialists and data warehouse designers who want to know about the new developments
- Data scientists, data analysts, and business analysts who use and work with data every day and who want to know which of these developments may help them
- Technology planners, technical architects, and enterprise architects who need to know how to evaluate all these new developments on their technical merits
- Database developers and administrators who need to know what the impact of Hadoop and Spark is on database aspects
- IT managers who need to be informed about these new developments to see what the potential business benefits are