View online: tdwi.org/flashpoint
April 7, 2016
ANNOUNCEMENTS
NEW TDWI Salary Report
2016 TDWI Salary, Roles, and Responsibilities Report
NEW Best Practices Report
Data Warehouse Modernization in the Age of Big Data Analytics
NEW Ten Mistakes to Avoid
In Preparing for IoT Analytics
NEW Business Intelligence Journal
Business Intelligence Journal,
Vol. 21, No. 1
NEW TDWI E-Book
Shaping the Future of Data Warehousing through Open Source Software
NEW TDWI Infographic
Operationalizing and Embedding Analytics for Action
NEW Checklist Report
Seven Best Practices for Adopting Data Warehouse Automation
CONTENTS
3 Trademarks of a Good Agile BI Coach
Stream Intelligence: Taking Tips from Social Media
Compensation: Job Satisfaction
Mistake: Failure to Develop an IoT Architecture
See what's
current in TDWI Education, Events, Webinars,
and Marketplace
3 Trademarks of a Good Agile BI Coach
Lynn Winterboer
Agile Analytics Educator and Coach
What BI teams need in an Agile coach.
Working in Agile data warehousing (DW) and business intelligence (BI) over many years, I’ve interacted with many teams and coaches. There are three hallmarks of a good Agile BI coach that teams should consider when looking for a coach.
1. A good Agile BI coach understands the unique challenges faced by BI teams.
Agile coaches accustomed to working with software development teams (e.g., Web or mobile) often struggle to get their points across when coaching BI teams. The biggest hurdle is the difference in semantics. Although a Web team, for example, might refer to the regular movement of data from one place to another as an API (application programming interface), the BI industry refers to this function as ETL (extract, transform, and load). Coaches who have a deep repertoire of software examples to illustrate Agile concepts to their teams will find that many of their examples will frustrate the BI teams because of the disconnect between the Web/mobile examples and the daily data-focused work of the BI team. What’s more, without understanding the various aspects of BI work, it’s difficult for a coach to understand and answer questions.
2. A good Agile coach prevents BI teams from making exceptions.
The strongest coaches I've seen excel at helping teams "hold the Agile process." They help their teams resist the temptation to make exceptions. A BI team with multiple stakeholders, specialized tools and related skill sets, and large volumes of data certainly faces more challenges than other IT teams. I can understand why it’s tempting to say, "We're different, and therefore we can't adhere to [Agile value here]." A good coach, however, will help the team think this through and find ways to adhere to the core values and principles of an Agile approach.
For instance, many BI teams don’t think it’s possible to frequently deliver working BI results for business review and feedback. Yet, a coach who understands the BI value chain—from source system all the way through to reports and dashboards—can help the team break the work effort into smaller, testable pieces of functionality that the business can review every few weeks.
3. A good Agile coach will ensure testing at the beginning of the project.
I’ve seen data teams try to develop a "mini waterfall" within each iteration, leaving all testing of functionality to the end. This practice shortchanges the testing activities (just as it does in a traditional waterfall process), and it goes against the grain of the Agile principle of continuous attention to technical excellence. It's very difficult to deliver technically excellent code when it hasn't been well tested. A good Agile coach will help the team understand how to start thinking about testing from the very start, as well as how to finish the higher-priority functionality, including testing, before starting work on the next-in-line priority.
Conclusion
Many BI teams recognize the value their software development colleagues are getting from using Agile approaches to their work. They want that experience for themselves, but they need the right coaching to help them on their journey. Among other talents, a good Agile coach will understand BI challenges and semantics; keep the team from making exceptions to any of the core principles of Agile; and ensure the team focuses on testing from the beginning of each development effort.
Lynn Winterboer teaches Agile courses through TDWI Onsite Education and coaches DW/BI teams on how to effectively apply Agile principles and practices to their work. She publishes at www.WinterboerAgileAnalytics.com and can be reached by email and @AgileLynn on Twitter.
Stream Intelligence: Taking Tips from Social Media
Troy Hiltbrand
We Are Social, an Internet research firm that tracks the use of social media around the world, has identified almost 2.5 billion people on social media today. That is a huge portion of the population. With so many people on social media, what are they doing? What is the draw to social media that makes it different from media venues of the past? Is there something that social media has uncovered that can be transitioned into the arena of business intelligence? This is a fundamental question today.
To understand why social media is successful and extrapolate those lessons for use in BI, we must understand what social media is at its core. First and foremost, social media is about information. It thrives on having a continuous feed of information to bring it to life.
The flow of information is the pulse of social media. As long as it flows freely, social media lives. If the information were to slow, interest in social media would wane; if it stopped, social media would die. Social media behemoths, such as Facebook, Twitter, and Instagram, are all worthless without the information they hold. This is apparent by looking at their predecessors, MySpace and Orkut, which failed because their audiences abandoned them and dried up their source of information. Google+, launched in 2011, struggles to attract the number of users needed to generate a critical mass of information and continues to reinvent itself, trying to attain that healthy flow of information.
Learn more: Read this article by downloading the Business Intelligence Journal (Vol. 21, No. 1).
Highlight of key findings from TDWI's wide variety of research
Compensation: Job Satisfaction
The level of job satisfaction held by BI/DW professionals has changed little through the years. Forty-nine percent of respondents rated their satisfaction level as “high” or “very high” in 2015, nearly identical to the 50 percent who thought the same a year ago. The ratio of those who rate their satisfaction as “low” or “very low” has remained flat as well, 11 percent in 2015 compared to 10 percent in 2014.
TDWI interprets this fairly high level of satisfaction as a measure of the industry itself. As our analysts meet with people in the field, we find them to be generally optimistic about their chosen careers. They work in a field that is constantly evolving and offering new areas for learning. In addition, they engage with the business on a collaborative and peer level far more often than most other disciplines in IT.
The number of people in the BI/DW field who consider themselves “fairly compensated” dipped a bit in 2015 (46 percent) compared to 2014 (52 percent). Those who believe they deserve more rose by 4 percentage points, from 31 percent in 2014 to 35 percent in 2015. The difference in average salary earned by those disparate groups is dramatic.Those who said they were satisfied earned on average $119,181; those who said the opposite earned $95,583, a difference of $23,598.
Feelings of job security remained fairly strong, with just 14 percent of respondents characterizing their jobs in 2015 as “not very secure,” up a bit from 2014.
(Click for larger image)
Read the full report: Download the 2016 TDWI Salary, Roles, and Responsibilities Report.
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.
Mistake: Failure to Develop an IoT Architecture
Fern Halper and Asim Razvi
Analytics can happen at multiple points in an IoT topology. For instance, it might make sense to deploy a model in a gateway that aggregates information from multiple sensors, or it may make sense to run the model in a cloud or even on premises. The same holds true for other kinds of analytics. Some analytics can even be deployed at the very edge of the network—in the “thing” itself. These analytics are often slimmed down depending on the capabilities of the edge device.
Often the analytics might have to occur in real time, in the data stream, before the data is stored for offline processing. These analytics must be managed for speed and scale. Your enterprise may need complex event analysis as part of the data stream (e.g., in fraud detection). Commercial solutions, along with Apache Spark and Kafka, are beginning to be leveraged in such applications. Current streaming and capture solutions use different methods to capture data. Some, such as Kafka, write the data to multiple nodes so that reconstructing lost data is usually possible to a certain point in time, but this also comes at a cost to speed; other systems may stream data faster but at the risk of potential data loss.
The point is that an architecture will be needed to determine what software is required to support IoT analytics as well as where the analytics should occur. Will it be on an edge device such as the “thing” or a gateway? How will data be handled? More often, organizations are finding that they need to take an ecosystem approach to analyze data. That means that there may be different components as part of the environment. For instance, there may be a stream processing engine as well as a data store. Of course, there may be devices and gateways involved in this architecture.
Read the full issue: Download Ten Mistakes to Avoid in Preparing for IoT Analytics (Q1 2016).
