View online: tdwi.org/flashpoint
April 3, 2014
ANNOUNCEMENTS
NEW TDWI E-Book:
Forward-Looking Business Intelligence
NEW TDWI E-Book:
Modernizing Your Data Warehouse Architecture with Hadoop
NEW TDWI Checklist Report
Eight Considerations for Utilizing Big Data Analytics with Hadoop
CONTENTS
Why Build Architecturally Complete BI/DW User Stories?
Watson and Siri: The Rise of the BI Smart Machine
Mistake: Making It About the Technology and Not About the Business Pain
See what's
current in TDWI Education, Events, Webinars,
and Marketplace
Why Build Architecturally Complete BI/DW User Stories?
Lynn Winterboer
Winterboer Agile Analytics
Topics:
Business Intelligence, Data Warehousing
Sometimes “by the book” guidance for agile teams doesn’t easily apply to data warehousing (DW) teams. For instance, slicing user stories can cause a conundrum when we face two options: (1) slice user stories by technical function/skill set or (2) slice stories into thinner, architecturally complete features that encompass all of the work necessary, from source to report, to meet the organization’s needs. If you choose to slice user stories by technical function or skill set, you will put your delivery of timely business value at risk. However, if you slice user stories into architecturally complete functionality, you keep your team focused on--and honor the agile commitment to--frequent and consistent business value.
When we say that we need to develop “architecturally complete” user stories, we find we can easily do that for the business intelligence (BI) front end of our BI/DW system--if the data already exists in the data warehouse. However, once we find ourselves facing a user story that requires data that has not yet been brought into the data warehouse, then we face a challenge: How will we get all that work done in just a few weeks? This challenge makes it tempting to fall back to our old ways of separating DW and BI work due to (among other reasons) differing skill sets needed for each.
If you choose to slice a user story by technical function or skill set, you would write a user story for the DW work (the “back end”) and one for the BI work (the user-facing “front end”). The business value, which lies in the results of the front-end BI work, is now separated from the heavy lifting that happens in the data warehouse. This may make the work clear for the delivery team(s), but it puts the data warehouse efforts at risk of losing focus on delivering business value. It also separates the delivery team members with DW skills from interacting with the business.
Let’s look at an example to illustrate the point:
User story: “As a call center manager, I would like to understand the percentage of total calls that each customer care rep in the company handles each month, as well as month-to-date for the current month, so that I can properly reward the top performers with special appreciations and provide the lower performers with additional training and support.”
Situation: In this case, the aggregations of calls by month are not yet in the data warehouse. The team members believe they cannot properly develop and test all of this in a single iteration, so they work with the business to slice this work into smaller chunks to fit into various iterations. They can see two options for slicing this relatively simple request. The first option slices the work by technical function:
- Story 1: Create the aggregations and percentage calculation in the data warehouse, which focuses on the modeling and ETL skills needed for this piece.
- Story 2: Create a report reflecting these calculations, which focuses on the metadata and report-building skills needed for the front end.
In this option, the DW work has to be done first before anything of value can be demonstrated to the business stakeholder(s). This means the DW team will spend several weeks working on something that doesn’t yet provide value to the business. It also means that by the time the BI work is completed in a future iteration and the stakeholders provide feedback on the whole effort, the DW folks have moved on to something else. This feedback would then be an interruption to their work.
The second option slices the work into three architecturally complete, if embryonic, features by doing the DW and BI work in the same story:
- Story 1: Monthly (past) aggregations
- Story 2: Month-to-date aggregations
- Story 3: Percentage calculations for each customer care rep
In this option, the business stakeholder(s) will see real, working results in the BI layer when the first story is completed. In addition, the biggest risk to the whole requirement is in getting the monthly aggregate calculations correct, considering source system anomalies around what is considered a “call.” The team and the business would like to focus on that first and prove it’s working before moving on to the relatively straightforward tasks of applying those business rules to the month-to-date aggregation as well as calculating the percentage of total calls by rep.
In Summary
BI/DW teams that take an agile approach to their work have interesting choices about how to proceed when the by-the-book guidance for agile teams doesn’t easily apply. My advice is to always look back at the Agile Manifesto’s Values and Principles for guidance in these decisions. This article focused on how to adhere to the agile principle “Working software is the primary measure of progress” by slicing stories into architecturally complete chunks of value rather than slicing by technical function/skill set.
Lynn Winterboer, CBIP--Mastery Level, has a proven background in a variety of data projects and agile practices and teaches and coaches BI/DW teams on how to effectively apply agile principles and practices to their work.
Watson and Siri: The Rise of the BI Smart Machine
Troy Hiltbrand
The past few years have seen a significant evolution in human-computer interaction. The era of smart machines is upon us, with automation taking on a more advanced role than ever before and permeating areas that have traditionally been unique to human interaction. This movement has the potential to fundamentally alter the way business intelligence (BI) is executed and deployed across industries as well as the role BI may play in all aspects of decision making.
“Smart machines” Watson and Siri have demonstrated that natural language understanding has the potential to change how end users interact with computational processing--and business intelligence systems.
Learn more: Read this article by downloading the Business Intelligence Journal, Vol. 19, No. 1
Highlight of key findings from TDWI's wide variety of research
BI/DW Salary Trends in 2013
The lion’s share of our respondents (47 percent) saw an increase of 1 to 3 percent in their wages in 2013. At the top end, 8 percent netted gains of 11 percent or more in their pre-withdrawal pay, while 20 percent saw either no change or a decrease. These figures are largely unchanged from 2012, indicating relative stability in compensation. Also unchanged, at 27 percent, were those BI professionals who enjoyed a wage increase higher than the standard in their organizations.
(Click for larger image)
Read the full report: Download the 2014 TDWI Salary, Roles, and Responsibilities Report
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.
Mistake: Making It About the Technology and Not About the Business Pain
Lyndsay Wise
Many organizations make BI one of their priorities because of top-down or executive direction. This is helpful in terms of project sponsorship, but the reality is that it can also present broader challenges when balancing the needs of the business versus technology adoption. Although technology does support business success, it is the people behind the technology that create real value for the business. Take advantage of newer technologies that enable your organization to scale and enrich application use based on native growth and future expansion. However, your true goal should be to address a real business pain using analytics.
Business-driven projects will focus on answering questions that are presented as challenges within your company. Looking at how to best address these questions--while maintaining regular insights into metrics and enough flexibility to drill deeper based on additional questions that may arise--will drive your technology choices. Thus, solution choices will be based on specific technical requirements. Both are tied together but need to be driven by the problems or operational gaps being experienced by the intellectual capital of your organization. This focus also improves collaboration between business departments and IT because both are required to make sure the most effective and efficient BI project choices are made.
Read the full issue: Download Ten Mistakes to Avoid When Bridging the Business/IT Divide (Q1 2014)
