Q and A: Business Intelligence Architecture in an Agile Environment
What role does BI architecture play in an agile environment?
- By James E. Powell, Claudia Imhoff, Ph.D.
- June 16, 2010
Agile is all about getting more done in less time -- giving an organization the information it needs quickly to be more nimble in a tough, competitive market. What role does the BI architecture play in an agile environment? For answers, we spoke with Claudia Imhoff, a TDWI faculty member who will be presenting Mastering BI with Best-Practice Architecture and Data Models: From Hub and Spoke to Agile Development with Len Silverston at the August and November TDWI World Conferences.
[Editor's note: "Creating an Agile BI Environment -- Delivering Data at the Speed of Thought" is the theme of the 2010 TDWI World Conference in San Diego.]
BI This Week: How can a BI architecture enable a more agile environment?
Claudia Imhoff: Whenever an initiative is undertaken that requires multiple projects for completion, there is always a risk of overlapping, underlapping, or altogether missed requirements. A BI environment consists of multiple projects that must be coordinated to mitigate redundancies, inefficiencies, or rip-and-replace activities. A BI architecture serves as the blueprint for the whole environment ensuring a comprehensive understanding of the "sum of its parts."
Having the architectural diagram up front sets the stage for the coordination of these related projects. Each project team can understand how their project's components fit into the overall environment, which existing components they can leverage, and how their pieces must interface with the existing or future BI components. This "road map" diagram improves the efficiency of implementation efforts; it also reduces overall costs, effort (in terms of time), and post-implementation maintenance activities.
What are the benefits of an architected environment in terms of a more agile development for BI?
The word "agile" is defined as "having the ability to move quickly and with suppleness, skill, and control." Agility without control is simply a rapid path to chaos. The BI architecture ensures that agility is maximized while chaos is minimized.
Perhaps the biggest benefit of the BI architecture is the removal of unnecessary redundancies in data management activities (ETL, data quality, etc.) and in analytic capabilities (e.g., reports, algorithms, and models). The architecture is the documentation of the entire environment thus giving BI project teams a full understanding of what exists for their current project needs and how they will support future projects with their implemented components.
What are the biggest mistakes IT makes when adopting an agile development methodology as it relates to BI?
The biggest mistake IT can make is to assume that an agile development methodology means no documentation or architectural effort is needed. If BI were a single project with no relationship to other projects or IT components, a lack of documentation or architecture might work.
Unfortunately for BI, this is disastrous and leads to a disorganized environment rife with inconsistencies, redundant efforts, scrap and rework, as well as user dissatisfaction with its performance. Agile methodology means implementing IT components faster but with control. That control comes from the recognition that the projects must integrate with each other and that they must reuse any and all components that they can.
There is a great deal of emphasis on rapid development. What do you recommend for companies embracing this development methodology while trying to maintain an enterprise vision for BI?
Although the creation and maintenance of an enterprise BI architecture does take a bit of time upfront, the payoff from having it in each and every project is enormous. Any BI team using the agile development methodology must balance the need for speed with the costs of reworking non-integrating components. A bit of effort up front goes a long way toward reducing the overall time and effort to create a company's critically needed BI components.
How easy is it to sell upper management on moving to an agile development environment?
Selling an agile development to upper management is a no brainer -- they like the idea of getting usable and useful IT implementations every 2-4 weeks rather than waiting for months or years for something. The difficulty comes in selling the need for an overall architecture before delivering the first component.
My advice is to educate upper management on the benefits and overall productivity boosts you get when you combine agile development with a sound architecture. Explaining the difference between chaos (and scrap-and-remake) and an architected approach is simple and that makes the "sell" to executives so much easier.
How disruptive is the move to agile development? Are there recommendations you can make to ease the transition?
Agile can be very disruptive to IT resources who are not familiar with it, have never been trained in its overall philosophy, or never gone through an agile project. They are however, usually familiar with architecture. My recommendation is to educate these resources on the overall agile development methodology and then demonstrate with examples how this methodology works with the architecture. It should be fairly simple to show how each sprint builds upon a prior one and further develops the overall architecture for BI.