Exclusive to TDWI Premium MembersView online: tdwi.org/flashpoint

TDWI FlashPoint Newsletter

Article Image

Feature

April 3, 2014

 

ANNOUNCEMENTS

NEW TDWI E-Book:
Forward-Looking Business Intelligence

NEW TDWI E-Book:
Modernizing Your Data Warehouse Architecture with Hadoop


NEW TDWI Checklist Report
Eight Considerations for Utilizing Big Data Analytics with Hadoop


CONTENTS

Feature
Why Build Architecturally Complete BI/DW User Stories?



TDWI Flashpoint Insight
Watson and Siri: The Rise of the BI Smart Machine



TDWI Research Snapshot
BI/DW Salary Trends in 2013



Flashpoint Rx
Mistake: Making It About the Technology and Not About the Business Pain



TDWI Bulletin Board
See what's current in TDWI Education, Events, Webinars,
and Marketplace


Why Build Architecturally Complete BI/DW User Stories?

Lynn Winterboer
Winterboer Agile Analytics

Topics: Business Intelligence, Data Warehousing

Sometimes “by the book” guidance for agile teams doesn’t easily apply to data warehousing (DW) teams. For instance, slicing user stories can cause a conundrum when we face two options: (1) slice user stories by technical function/skill set or (2) slice stories into thinner, architecturally complete features that encompass all of the work necessary, from source to report, to meet the organization’s needs. If you choose to slice user stories by technical function or skill set, you will put your delivery of timely business value at risk. However, if you slice user stories into architecturally complete functionality, you keep your team focused on--and honor the agile commitment to--frequent and consistent business value.

When we say that we need to develop “architecturally complete” user stories, we find we can easily do that for the business intelligence (BI) front end of our BI/DW system--if the data already exists in the data warehouse. However, once we find ourselves facing a user story that requires data that has not yet been brought into the data warehouse, then we face a challenge: How will we get all that work done in just a few weeks? This challenge makes it tempting to fall back to our old ways of separating DW and BI work due to (among other reasons) differing skill sets needed for each.

If you choose to slice a user story by technical function or skill set, you would write a user story for the DW work (the “back end”) and one for the BI work (the user-facing “front end”). The business value, which lies in the results of the front-end BI work, is now separated from the heavy lifting that happens in the data warehouse. This may make the work clear for the delivery team(s), but it puts the data warehouse efforts at risk of losing focus on delivering business value. It also separates the delivery team members with DW skills from interacting with the business.

Let’s look at an example to illustrate the point:

User story: “As a call center manager, I would like to understand the percentage of total calls that each customer care rep in the company handles each month, as well as month-to-date for the current month, so that I can properly reward the top performers with special appreciations and provide the lower performers with additional training and support.”

Situation: In this case, the aggregations of calls by month are not yet in the data warehouse. The team members believe they cannot properly develop and test all of this in a single iteration, so they work with the business to slice this work into smaller chunks to fit into various iterations. They can see two options for slicing this relatively simple request. The first option slices the work by technical function:

  • Story 1: Create the aggregations and percentage calculation in the data warehouse, which focuses on the modeling and ETL skills needed for this piece.
  • Story 2: Create a report reflecting these calculations, which focuses on the metadata and report-building skills needed for the front end.

In this option, the DW work has to be done first before anything of value can be demonstrated to the business stakeholder(s). This means the DW team will spend several weeks working on something that doesn’t yet provide value to the business. It also means that by the time the BI work is completed in a future iteration and the stakeholders provide feedback on the whole effort, the DW folks have moved on to something else. This feedback would then be an interruption to their work.

The second option slices the work into three architecturally complete, if embryonic, features by doing the DW and BI work in the same story:

  • Story 1: Monthly (past) aggregations
  • Story 2: Month-to-date aggregations
  • Story 3: Percentage calculations for each customer care rep

In this option, the business stakeholder(s) will see real, working results in the BI layer when the first story is completed. In addition, the biggest risk to the whole requirement is in getting the monthly aggregate calculations correct, considering source system anomalies around what is considered a “call.” The team and the business would like to focus on that first and prove it’s working before moving on to the relatively straightforward tasks of applying those business rules to the month-to-date aggregation as well as calculating the percentage of total calls by rep.

In Summary
BI/DW teams that take an agile approach to their work have interesting choices about how to proceed when the by-the-book guidance for agile teams doesn’t easily apply. My advice is to always look back at the Agile Manifesto’s Values and Principles for guidance in these decisions. This article focused on how to adhere to the agile principle “Working software is the primary measure of progress” by slicing stories into architecturally complete chunks of value rather than slicing by technical function/skill set.

Lynn Winterboer, CBIP--Mastery Level, has a proven background in a variety of data projects and agile practices and teaches and coaches BI/DW teams on how to effectively apply agile principles and practices to their work.

Article ImageFlashpoint Insight

Watson and Siri: The Rise of the BI Smart Machine
Troy Hiltbrand

The past few years have seen a significant evolution in human-computer interaction. The era of smart machines is upon us, with automation taking on a more advanced role than ever before and permeating areas that have traditionally been unique to human interaction. This movement has the potential to fundamentally alter the way business intelligence (BI) is executed and deployed across industries as well as the role BI may play in all aspects of decision making.

“Smart machines” Watson and Siri have demonstrated that natural language understanding has the potential to change how end users interact with computational processing--and business intelligence systems.

Learn more: Read this article by downloading the Business Intelligence Journal, Vol. 19, No. 1

Article ImageTDWI Research Snapshot
Highlight of key findings from TDWI's wide variety of research

BI/DW Salary Trends in 2013
The lion’s share of our respondents (47 percent) saw an increase of 1 to 3 percent in their wages in 2013. At the top end, 8 percent netted gains of 11 percent or more in their pre-withdrawal pay, while 20 percent saw either no change or a decrease. These figures are largely unchanged from 2012, indicating relative stability in compensation. Also unchanged, at 27 percent, were those BI professionals who enjoyed a wage increase higher than the standard in their organizations.

(Click for larger image)
Click to view larger

Read the full report: Download the 2014 TDWI Salary, Roles, and Responsibilities Report

Article ImageFlashpoint Rx
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.

Mistake: Making It About the Technology and Not About the Business Pain
Lyndsay Wise

Many organizations make BI one of their priorities because of top-down or executive direction. This is helpful in terms of project sponsorship, but the reality is that it can also present broader challenges when balancing the needs of the business versus technology adoption. Although technology does support business success, it is the people behind the technology that create real value for the business. Take advantage of newer technologies that enable your organization to scale and enrich application use based on native growth and future expansion. However, your true goal should be to address a real business pain using analytics.

Business-driven projects will focus on answering questions that are presented as challenges within your company. Looking at how to best address these questions--while maintaining regular insights into metrics and enough flexibility to drill deeper based on additional questions that may arise--will drive your technology choices. Thus, solution choices will be based on specific technical requirements. Both are tied together but need to be driven by the problems or operational gaps being experienced by the intellectual capital of your organization. This focus also improves collaboration between business departments and IT because both are required to make sure the most effective and efficient BI project choices are made.

Read the full issue: Download Ten Mistakes to Avoid When Bridging the Business/IT Divide (Q1 2014)

TDWI Bulletin Board


EDUCATION & EVENTS

TDWI Seminar
Washington, DC

April 7–10, 2014

TDWI World Conference
Chicago, IL

May 11–16, 2014

Analytics World
Chicago, IL

May 12–15, 2014


WEBINARS

Seven Data Discovery Steps for Improving Information Delivery and Accuracy

Analytics at the Speed of Business: Delivering Real-Time Insight from Data Streams

Evolving Data Warehouse Architectures in the Age of Big Data Analytics


MARKETPLACE

TDWI Solutions Gateway
Information Integration and Governance

TDWI White Paper Library
Extreme Warehouse Speed for Fast Business Decisions

TDWI White Paper Library
IBM WebSphere MQ: The Right Information in the Right Place and Time

PREMIUM MEMBER DISCOUNTS

As a Premium Member of TDWI, you are eligible for a $275 discount off the price of TDWI World Conferences. Register for the upcoming Chicago World Conference here.

MANAGE YOUR TDWI PREMIUM MEMBERSHIP

Renew your Premium Membership by: [-ENDDATE-]

Renew | FAQ | Edit Your Profile | Contact Us

 

TDWI Home | About Us | Research
Publications
| Certification | Education
Partners | Premium Membership

Copyright 2014. TDWI. All rights reserved.