Q&A: Extreme Scoping: A Data-Driven Agile Methodology
The traditional Scrum approach is well suited to developers but not enterprise information management professionals. We discuss a new, data-driven agile approach with its creator, Larissa Moss.
- By James E. Powell
- July 16, 2013
If your enterprise is trying to improve its timely delivery of BI functionality, your IT department may be exploring the use of agile methodologies. The most popular agile methodology, called Scrum, is based on the concept of a scrum -- a technique for improving communication and focusing project direction. The problem is that Scrum is best suited to development-driven projects, but not for enterprise-class data-driven projects.
Larissa Moss is founder and president of Method Focus Inc., a company specializing in enterprise data warehousing and business intelligence. She has over 30 years of IT experience, and for over 20 years, her focus has been on data management, project and program management, and methodologies. In this Q&A, we explore with Moss her new agile approach, called Extreme Scoping, that she says addresses this problem.
Moss is a frequent speaker at conferences in the U.S. and Europe on the topics of data warehousing, business intelligence, master data management, agile enterprise data warehouse project management, spiral data warehousing/business intelligence methodologies, enterprise information management, data governance, and data quality. She will be leading a session at the TDWI World Conference in San Diego (August 18-23, 2013) about Extreme Scoping: An Agile Approach to Enterprise-Class Data Warehousing.
BI This Week: Some project teams love using Scrum while others hate it. Why is that?
Larissa Moss: Let me restate your claim this way: Developers love Scrum while enterprise information management or EIM professionals hate it. The answer is pretty simple. Scrum and other popular agile methodologies currently on the market were written by developers for other developers -- specifically developers of operational systems, which are customized silo systems by nature. The main purpose was to replace the old traditional waterfall methodologies with more dynamic, more interactive, and more effective methodologies -- namely agile.
The main goal of developers is to turn functional user requirements into working code as quickly as possible. In other words, everything revolves around development work. Scrum is a development-driven methodology. There’s nothing wrong with that, except that EIM professionals (the data people) don’t work on development-driven projects, and they have no use for development-driven methodologies, agile or otherwise.
EIM professionals are not interested in turning functional user requirements into working code. They are interested in standardizing, rationalizing, and cleansing corporate data and providing a pool of unique entities populated with unique attributes. I’m talking about the proverbial “single version of the truth.” That goes for EDW/BI projects as well as for MDM, CRM, CDI, and other enterprise-class initiatives. They all need data-driven methodologies.
How is your Extreme Scoping agile method different from Scrum?
There are three ways to answer this question. The first and probably the shortest answer is that Extreme Scoping is based on a robust DW-specific soup-to-nuts methodology, which recognizes that 80 percent of any enterprise-class EDW/BI project is data management and only 20 percent is data delivery (BI application, reports, etc.). It is a data-driven approach, not a development-driven approach. With Extreme Scoping, EIM professionals run the project, not developers. Furthermore, I combine proven project management techniques with new agile principles to achieve similar agile project dynamics.
The second way to answer your question is in terms of agile vernacular and buzzwords. In that respect, Extreme Scoping is totally different from Scrum. I don’t use their terminology or their rules or their team structures or their ScrumMaster role or anything else that’s unique to Scrum. I am not interested in adapting Scrum to enterprise-class EDW/BI projects. Scrum was meant to be a development-driven methodology and I believe it’s best to leave it that way.
Answering this question a third way, I can say that Extreme Scoping supports all four affirmations of the agile manifesto:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
You don’t need Scrum to implement these four affirmations. You can implement them with any agile methodology. In addition to these affirmations, there are many agile concepts or principles that Extreme Scoping uses, and then there are some that I don’t believe are applicable to enterprise-class data integration projects. I will give you some examples of each.
Agile principles that are totally applicable to EDW/BI projects are business vision instead of requirements, speculation instead of estimating, quality before quantity (in Scrum this refers to code, in Extreme Scoping this refers to data), exploration (I call it operational prototyping) instead of sequential analysis-design-coding-testing, self-organized project teams (although their makeup in Extreme Scoping is different from Scrum teams), time-boxing, and a few others. There are fewer agile principles that do not apply to EDW/BI projects, such as cadence, a ScrumMaster, freezing the scope, the notion of a product owner, how releases are estimated, burn-down charts, and probably a few more. I explain the reasons for this in my presentations and in my upcoming book, Extreme Scoping: An Agile Approach to Enterprise Data Warehousing and Business Intelligence .
What types of BI projects should use Extreme Scoping?
I am not proposing to replace Scrum on all BI projects. The decision must be based on the type of BI project being developed. I am specifically referring to TDWI’s BI Maturity Model, showing a bell curve across five levels of maturity. Companies on the left side of the bell curve are in the prenatal/infant or child stages, still building customized silo solutions for their users. I usually say that these companies are still climbing the mountain -- the bell curve being the mountain.
When building these customized silos, the project teams focus on delivering functionality as quickly as possible. They don’t spend much time -- if any -- trying to integrate and standardize the data across the enterprise. That makes these project development-driven projects. Since Scrum is a development-driven methodology, it will work just fine for those project.
On the other hand, once the companies are coming down the mountain on the right side of the bell curve, their primary focus is on enterprisewide data integration, and the BI applications and reports just come along for the ride. That changes the makeup of the projects entirely from development-driven to data-driven. Extreme Scoping is a data-driven method and is applicable for companies in the teenager, and especially in the adult and sage stages of BI maturity.
How do you get organizations to convert from traditional methodologies to agile?
With Scrum, that’s not so easy because it is so radically different from a traditional methodology. Companies are expected to switch “cold turkey” from their familiar waterfall methods to Scrum. Developers love Scrum and have no problem switching, but IT management, in particular the project management office (PMO) and auditors, as well as many business people, hesitate. They see too much risk in it.
On the other hand, with Extreme Scoping, making such a transition is actually very easy because people can see the robust methodology components underneath the agile approach, which takes the risk away. Plus, some of the Extreme Scoping planning steps look very familiar to IT management and to the business people. In a way, they can have it both ways -- agility based on a robust DW-specific, soup-to-nuts methodology. They know that they will not miss anything important, especially on the data management side.