A BI Architectures Approach to Modeling and Evolving with Analytic Databases
Major shifts converging in today's BI environment bring the opportunity to discover new answers to old questions about what BI architectures are about and how they are designed.
By John O'Brien, Principal, Radiant Advisors
Whether you have been a BI architect managing a production data warehouse for many years or are embarking on building a new data warehouse, the new analytic technologies coming out today have never been so powerful and complex to understand. In fact, with so many analytic technologies available on the market, they are somewhat overwhelming as we struggle to make sense of what to do with them and which ones to use with our existing environments.
This is good for BI architects because it brings us back to BI architecture fundamentals, key data management principles, pattern recognition, and agile processes, along with BI capabilities that challenge classic BI architecture best practices to design what clearly makes sense to meet the demands of business today.
There are major shifts converging in today's BI environment, and these changes bring with them the opportunity to discover new answers to old questions about what a BI architecture is all about and how an architecture is designed. As I explore these questions in this article, I will focus on three main themes: BI architectures are strategic platforms that evolve to their full potential; good architectures are based on recognizing data management principles and patterns (so-called best practices are discovered later); and BI architecture design is purposeful at every stage of development and technology decisions follow this purpose.
Architecture Maturity and Information Capabilities
We have all seen the research that says BI architectures evolve into robust information platforms and value over time. BI teams have focused on increasing value through maturing their data warehouses from operational reporting to data marts to data warehouses and finally to enterprise data warehouses. This evolutionary approach is typical when balancing pressing tactical needs for information delivery with strategic development, and is found in many companies where business demands for information drive towards a data warehouse platform.
The classic data warehouse is the last thing to be built, if ever, because the emphasis remains on quicker delivery first and information consistency later. Unfortunately, this leads to data warehouses that reflect current information needs and doesn't foster the evolution of a mature analytics culture.
Instead, an architecture based on BI capabilities focuses on nurturing the analytic culture of the business community by first educating user communities about the BI capabilities available and then on business subject data that is delivered via BI capabilities. This approach centers the data warehouse architecture on BI capabilities such as information delivery; reporting and parameterized reporting; dimensional analytics for goals achievement; and advanced analytics for gaining insights, to name a few. These discussions recognize that the same consistent data has many usage patterns, behaviors, and roles in the decision process. A BI architecture that is designed in this way ensures that data models and chosen analytic technologies are best suited to their intended purpose.
However, this BI-capabilities approach is contrary to some BI architects' belief that there should be an all-in-one data warehouse platform in the enterprise.
Data Management Principles, Patterns, and Best Practices
There are many great BI architects and thought leaders with a variety of opinions and philosophies. Fortunately, there is more than one way to deliver a truly valuable strategic data warehouse platform. One important component that changes from company to company is its requirements, constraints and culture. BI architects should be able to recognize these characteristics, understand how various BI architectures match them, and determine how the architectures can evolve with the company's analytics culture.
Many key data management principles offer opportunities for BI architects to disagree. The goal is to define these key data management principles in such a way that BI architects have a spectrum from which to recognize their position. Think of the 1 to 10 scale that we see often in surveys. Are you mostly in disagreement (1), neutral (5), or mostly in agreement (10)?
Here is a typical example. The key principle should not be a question regarding data consistency; I think all BI architects would agree that maintaining consistent data is important in information management. However, how do you feel about data duplication and synchronization with regards to maintaining consistent data? Are you extremely uncomfortable with duplicating data and feel having data correctly represented in a single database is the best way to ensure consistency (1-4), or are you comfortable with duplicating data across databases and taking on the additionally responsibility of managing consistency and synchronization (6-10)?
You can see how this one key data management principle alone will influence what type of data warehouse architecture you feel is best suited for your enterprise. Understanding your own beliefs and comparing them with those around you will greatly improve the BI architecture strategy in your company. We will explore many more key principles like this one in our TDWI session.
Marketing Hype and Shiny Objects
BI professionals are fortunate to have an ecosystem of great publications, Webinars, articles, and analysts actively monitoring what's happening in the industry. However, sometimes there is so much noise and hype that it's hard not to feel as though you're being left out of the latest trend or technology in the market. If that describes you, I recommend focusing on the latest analytic databases that offer the most value and opportunity to your company through differentiation, or how technology enables end users in new, innovative ways. Remember that most analytic technologies are too new to have field-proven best practices for others to follow.
Again, this comes back to reinforcing and managing our BI-capabilities architecture and strategy, and delivering capabilities to the business through key technologies or analytic databases. When you observe a new technology buzzing in the market, ask yourself what new capabilities this technology will enable the BI architecture or user to do. Today, many BI teams are asking what the best practices are for implementing big data technologies such as NoSQL and Hadoop in their BI architecture, followed by "Why do I need it?"
Understanding an emerging technology's differentiators, strengths, and weakness will help put this is perspective. Companies are realizing that even though they may not have big data yet, Hadoop and other NoSQL analytic databases offer a solution for situations where the incoming structure of the database varies too frequently for more traditional relational databases. Additionally, they offer a solution for situations where the complexity of analysis with the data exceeds the capabilities or constraints of the SQL language in their existing relational databases.
Although an analytic database should be the last choice made when managing a BI-capabilities architecture approach, that doesn't mean we shouldn't be always evaluating them. We simply have a better and less emotional way to do so.
We'll explore these topics in the more detail in the upcoming TDWI World Conference in Boston full-day course, BI Architectures Approach for Modeling and Evolving with Analytic Databases, with emphasis on architecture modeling, evolution, and how to incorporate the latest analytic databases and skills. Today is a great opportunity for BI architects to challenge the norm, reestablish road maps, and create tomorrow's best practices.
John O'Brien, CBIP is principal and CEO of Radiant Advisors and a recognized architecture and analytics visionary. You can contact the author at [email protected].