View online: tdwi.org/flashpoint
November 7, 2013
ANNOUNCEMENTS
NEW TDWI Best Practices Report:
Managing Big Data
NEW TDWI Checklist Report:
Hadoop Best Practices for Data Warehousing, Data Integration, and Analytics
CONTENTS
2013 TDWI BI Benchmark Report: Critical Factors for BI Success
Maximizing Insight from Unstructured Data
BI Benchmark Report: Staffing, Development, and Usage
Mistake: Lack of Ongoing Communication
See what's
current in TDWI Education, Events, Webinars,
and Marketplace
2013 TDWI BI Benchmark Report: Critical Factors for BI Success
Mark Hammond
TDWI Contributing Analyst
Topics:
Business Intelligence, Data Analysis and Design, Program Management
Comparing your organization’s BI approach to that of your peers can be helpful in charting your path to success. How does your BI team rank against the industry at large in terms of reporting structures, development methodologies, budgets, staffing, maturity, and value realized from BI?
The annual TDWI BI Benchmark Report is a good way to find out. Now in its eighth year and available exclusively to TDWI Premium Members, this in-depth, survey-based report reflects trends in BI environments and prevailing approaches, and quantifies such data points as year-over-year changes in capital and maintenance budgets.
Our 2013 study, based on a survey of 461 BI professionals, finds that BI maintenance spending is up this year at 48 percent of organizations, while the median maintenance budget has risen 29 percent since 2011, to $400,000.
Another data point is the perceived business value that organizations achieve with BI. Nearly 24 percent of respondents reported a high degree of business value from BI, down from 28 percent in 2012. On the other end, the incidence of low business value from BI rose from 13 percent in 2012 to 19 percent in 2013. We suspect that new demands and rising expectations contribute to the shift.
For example, long-standing report-oriented implementations suddenly seem less valuable when new requirements for analytics are added. Meanwhile, new challenges such as big data are emerging. Defining business objectives for BI on big data is not always straightforward; nor is measuring value in the early days of a big data BI initiative. As the industry evolves, value from BI is a moving target.
The challenge is to keep processes and systems aligned with these challenges. To size up which approaches are most effective in delivering high value, the TDWI BI Benchmark Report measures perceptions of value against select dimensions, including development methodologies, reporting structures, and resource organization.
Development methodology. Organizations are realizing the best results with an agile development methodology compared to the waterfall approach or a combination of the two. At the same time, the use of agile is on the rise, from 18 percent of organizations in 2011 to 27 percent in 2013.
Reporting structures. BI teams that report to business leaders, rather than IT, are more successful in delivering high business value from their environments. Yet 55 percent of BI teams report to IT rather than the business side.
Resource organization. Organizations net the greatest results with a mix of centralized and decentralized resources. This mixed-model approach is also the most prevalent, in place at 53 percent of organizations.
BI team size. As a proportion of overall IT, more resources dedicated to BI tend to produce better results. Although that is unsurprising, the differential is notable. At organizations where BI team size was just 1–3 percent of overall IT, only 15 percent achieved high value. When the BI team size grew to at least one-fifth that of IT, 41 percent of organizations achieved high value.
The TDWI BI Benchmark Report also takes a look at the scope and maturity of environments, the number of distinct BI teams and subject areas supported, the use of third-party contractors, the time needed for BI to deliver a positive financial return, and more. Download a copy of the 2013 TDWI BI Benchmark Report (exclusive to TDWI Premium Members).
Mark Hammond is a veteran contributor to TDWI, including a number of research reports, Business Intelligence Journal, What Works, and more.
Maximizing Insight from Unstructured Data
Sundar Varadarajan and Soundarapandian
Extracting meaningful information from unstructured data, in particular text data, is a complex process. Although text analytics engines exist that enable extraction with sufficient accuracy, the process must be supplemented with domain- and problem-specific knowledge and enhanced to handle the nuances and peculiarities of text sources.
The approach we describe in this article involves an ontology, taxonomy definitions, and extraction, as well as specific preprocessing steps. The domain- and problem-specific knowledge that is defined as an extension of the core linguistic engine is a key part of the information extraction layer and plays a critical part in achieving accurate results. We have worked with a variety of text sources, such as e-mail exchanges, chats, tweets, and Facebook and YouTube comments.
We present the approach, the framework employed for supplemental domain knowledge, and the enhancements necessary to significantly improve the accuracy of text analysis results. We illustrate all this using specific examples and scenarios we have successfully applied for some of our customers.
Read the full article and more: Download Business Intelligence Journal, Vol. 18, No. 3
Highlight of key findings from TDWI's wide variety of research
BI Benchmark Report: Staffing, Development, and Usage
Nearly two-thirds (63 percent) of organizations utilize per-project teams with three or fewer full-time equivalent (FTE) employees; up from 56 percent in 2011, this illustrates that many BI executives prefer to attack projects with a small, nimble team of members who are typically cross-trained in multiple disciplines, such as data modeling, data integration, reporting, architecture, and subject matter expertise. Similarly, our 2013 survey finds the average number of per-project FTEs at 5.1, down from 5.6 in 2011.
(Click for larger image)
Read the full report: Download 2013 TDWI BI Benchmark Report: Organizational and Performance Metrics for Business Intelligence Teams
FlashPoint Rx prescribes a "Mistake to Avoid" for business intelligence and data warehousing professionals.
Mistake: Lack of Ongoing Communication
Laura Reeves
BI projects are typically run by IT. Most teams attempt to gather some type of business requirements, then design and build a solution. If you don’t have major progress to report, you may hesitate to convene a meeting. Sometimes you must call a meeting to ask for more time or money. In the meantime, the business community is left wondering what is being done and why it is taking so long. Business-driven BI requires more than gathering requirements at the beginning of a project; it means working together throughout the entire project.
A better approach is for one or two key representatives from the business to come on board as part of the core project team. These individuals work side by side with the project team and participate in weekly project status meetings. Project updates should also be provided to stakeholders on a regular basis, regardless of how much (or little) progress has been made. The message can simply be that the team is working hard and the project is on schedule. If there are concerns about meeting project deadlines, share them. Be honest. If there is anything that could make things better (access to a critical person, adding another ETL developer, a bigger server for development)--speak up. You won’t get any help unless you ask.
Read the full issue: Download Ten Mistakes to Avoid When Delivering Business-Driven BI (Q3 2013)
