April 28, 2014
Organizations that aspire to be data driven cannot afford to limit
users’ data analysis to structured data: that is, the alphanumeric
data types that have been carefully defined, modeled, and stored in
standard spreadsheets, relational databases, and data warehouses.
In the course of business processes, operations, and customer
interactions, enterprises generate far more than just structured
data—in particular, most are swimming in massive amounts of text.
Drawn from both internal and external sources, text files form a
potential gold mine of insights that could help organizations reduce
costs, improve customer relationships, speed response to events, and
innovate with products and services. Business intelligence (BI) and
data warehousing (DW) systems are adept at delivering the numbers,
but if users do not have access to unstructured and semi-structured
text such as customer comments, field personnel notes, and social
media, they will be blind to contextual information that could help
answer questions about the numbers. Exploratory analysis of text
could reveal emerging trends that do not show up in BI reports.
Text analytics covers a range of technologies and practices for
analyzing text, extracting relevant information, and transforming
sources by applying structure so that analysis can be repeated and
adjusted over time. Software solutions combine techniques from
natural language processing, statistics, and machine learning. One
of the goals of text analytics is to accurately extract entities, facts,
concepts, themes, and sentiment. With the popularity of social media,
many enterprises are interested in performing sentiment analysis.
However, interest is expanding across industries to other uses, such as
in healthcare, where text analytics helps doctors understand the full
context of patient symptoms and engage in evidence-based medicine.
Making use of modern data visualization, text analytics solutions
enable users to explore text on desktops and mobile devices. Business
users, not just specialized text analysts, can use the solutions
to derive value—for example, by aggregating findings from text
with their analysis of structured data. Following TDWI’s previous
Checklist Report on this topic, How to Gain Insight from Text, this
Checklist discusses the relevance of text analytics for improving user
experiences with the semi-structured and unstructured textual big
data now at their disposal.