TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Text Analytics to the Rescue

A recent survey suggests a clear and inescapable rise in the consumption -- and importance -- of unstructured data

• By Stephen Swoyer
• August 12, 2009

Text analytics guru Seth Grimes, a principal with consultancy Alta Plana, recently published the 2009 edition of his Text Analytics: User Perspectives on Solutions and Providers survey. Like its predecessor, the 2009 version is chock-full of intriguing nuggets -- particularly on the demand side.

Not only is text analytics usage increasing (along with an ever-expanding constellation of applications), but the number and variety of sources that text analytics practitioners say they're consuming also continues to expand.

All in all, the most recent Text Analytics survey suggests a clear and inescapable increase in the consumption -- and importance -- of unstructured data.

For example, Grimes notes, nearly half (47 percent) of current text analytics practitioners say they're consuming data from blogs or social networks; slightly less (44 percent) cite data from news articles, e-mail messages (36), online forums (35 percent), or customer/marketing surveys (34 percent). All of this suggests that such unstructured data -- which has been viewed as a potentially valuable (but nonetheless unverified) information source -- is beginning to pay off. "These are on-line and other feedback-rich sources. Their high rate of selection suggests that veteran users have found significant benefit in these sources," writes Grimes. The rub, he says, is that shops that aren't currently using text analytics probably don't know what they're missing.

"By contrast, only three information-type categories [e.g., e-mail and correspondence, customer or marketing surveys, and contact center notes/transcripts] were selected by over 26 percent of respondents who are not yet using text analytics," Grimes continues. "It's easy to infer that the value of online materials … has not yet become clear to prospective users. That only a minority chose any particular category suggests … [that] prospective users are more broadly distributed across application categories … [or that] prospective users are cautious about how many different sources they tackle initially."

Such sources will undoubtedly come later, Grimes suggests, after text analytics adopters first master traditional materials: "[T]he plurality -- the largest portion -- of prospective users will focus initially on materials they have on hand that involve interactions with known stakeholders. Web sources can come later."

One downside to text analytics success is hype. Grimes found that potential adopters often have unrealistic expectations on the text analytic tip. For example, he notes, prospective text analytics users tend to cite a different -- and, not surprisingly, more explicitly remunerative -- set of ROI drivers than do existing practitioners. This is both good and bad, he explains.

"Of prospective-user respondents, almost a quarter are already using 'increased sales to existing customers' as an ROI measure, which make[s] sense. Sales are easily tracked and analyzed by current systems where items such as satisfaction ratings are not," Grimes writes.

On the other hand, he continues, a total of five ROI drivers are accorded disproportionate weight (cited by 38 percent or more of respondents) by prospective adopters, which suggests that prospects tend to be more exuberant -- irrationally so -- about the perceived benefits of text analytics than do seasoned users. "These prospective users, and the folks who advise them, would do well to manage and focus their expectations."