What is Agile?
“Agile” has rapidly become the new buzzword for data warehousing and business intelligence development -- but what does “agile” mean? Better yet, what should “agile” mean?
- By Paul Kautza
- May 26, 2010
[Editor's note: "Creating an Agile BI Environment -- Delivering Data at the Speed of Thought" is the theme of the 2010 TDWI World Conference in San Diego.]
“Agile” has rapidly become the new buzzword for data warehousing and business intelligence development -- but what does “agile” mean? Better yet, what should “agile” mean?
Agile software development has been mainstream for several years and now many people and organizations are trying to make it mainstream in DW/BI. I have been in this business for over 25 years and have seen many terms, fads, and buzzwords come and go and very few have succeeded.
Lack of success occurs when we focus on only one aspect of a complex problem. Agile development for DW/BI is no different in that it focuses on only the development side of the problem. I think we would be better served to focus on the word “agile”. If you are agile, you move quickly, act quickly, are accepting of change and are under control. Those attributes describe a behavior, not a methodology; therefore, all of us need to be the main ingredient to making “agile” work.
Being agile across the various disciplines of BI will help move your businesses in the right direction. This will be critical as the economy begins to recover slowly. You’ll realize that in the world we now live, "business as usual" no longer applies. We face a new environment that we must figure out quickly.
To that end I have created an “Agile BI Framework” to help you understand what I mean. This framework has four key components: Agile DW Design and Development, Agile Analytics, Agile BI Organizations, and Agile DW Infrastructure. We need to stop looking at solutions so narrowly and start integrating them across the people, processes, and technologies in our businesses.
“Agile DW Design and Development” is where the more traditional aspects of agile need to be applied. Agile development promotes a specific set of techniques using iterative development for rapid delivery of systems with a minimum of rework and risk. This involves the agile development principles of scrum, extreme programming (XP), and iterative simulation (among others) and includes all of the formal/prescriptive principles of agile development.
Beyond the formal principles of agile development, I believe there is significant value to be gained simply by being more “agile” in how we approach our work. One example is bringing more agility to our data modeling process for our data warehouse environment. We all know firsthand (or have heard about) industry data models and universal data models. Some of us who have been around too long, roll our eyes when we hear these terms because we remember when they first became available. We recall how monolithic and difficult these models were to deal with. If you are like me, or if you’re new to data warehousing, I challenge you to take a fresh look at these universal data models. You will see how they can make projects come together quickly, and how they increase flexibly and sustainability of BI systems.
“Agile Analytics” fascinates me. We all like to think that what we deliver is timely and meets our organization's needs. However, if we talk to our business users, we may find a different perspective. They receive static updates in a variety of traditional ways, such as dashboards, scorecards, and canned reports. That doesn’t sound very agile, does it?
If we are going to meet our business' demands for agile analytics, we must have an agile data architecture that can be flexible, reduce delivery cycles, and provide short-lived or one-time analytics. We need to better understand the strengths and weaknesses of -- and differences among -- ROLAP, MOLAP, HOLAP, and DOLAP, and we must recognize when and how we should be using them to deliver flexible business value. We need to explore the newest technologies and trends that are starting to revitalizing the BI analytics market, including the resurgence of predictive analytics.
“Agile BI Organizations” will be at the center of all successful, business-value-driven projects in this new economic environment. I've long believed that the Achilles heel of our industry has been the lack of true partnership between business and IT. Both sides have provided lip service to this partnership over the years, but very few have truly committed to it with structural changes that break apart the organizational silos that we all live in. The only way we will succeed in advancing our businesses is by creating a more agile BI organization that has business users and IT professionals working side by side on the same team and under the direction of one executive sponsor. The winners will be those who can lead the change and growth of their organizations to maximize the people, processes, and technology across the enterprise.
“Agile DW Infrastructure” is currently an explosive area for data warehousing and business intelligence. With cloud computing, virtualization, software-as-a-service, open source, and other technologies taking center stage in most industry publications, we must all take a hard look at what these technologies can do for us. They -- and paradigm shifts in solution delivery options -- have the potential to change the way computing resources are delivered. The overall price/ performance that can be delivered should help level the playing field and help the smaller, more agile enterprises more effectively compete with their larger rivals. I also believe that these technologies will play a major role within organizations working to drive out costs.
Agile is learned behavior. None of us is "born agile" and we won't get there without learning and practicing. This holds true across the board for business people, IT people, managers, and staff. We can't become agile by training only our developers. My expanded use of the term “agile” is behind my belief that we need to bring together people, process, and technology on a broader scale.
If you are intrigued by these ideas and would like to learn more, please join us at the TDWI World Conference in San Diego, August 15 - 20, 2010. Our theme is “Developing an Agile BI Environment.” This event will offer 50 full- and half-day vendor-neutral educational courses on a variety of topics related to data warehousing and business intelligence, including 15 courses specifically related to the Agile BI Framework I've outlined. In addition to the conference, we are holding a three-day Executive Summit focusing on “Agility, Alignment, and Analytics.”
Paul A. Kautza
Director of Education
TDWI
[email protected]