TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

The Future of Text Analytics

As enterprises look to put their best foot forward in 2017, many are increasingly turning to text analysis to improve customer experiences and business processes.

• By Terry Lawlor
• February 1, 2017

The volume and variety of data generated across feedback channels continues to expand at an exponential rate, providing businesses with a wealth of information about their customers. Looking to improve customer experiences and business processes, organizations are increasingly turning to text analysis to leverage the unstructured text held in open-ended survey questions, contact center notes, social media posts, and other sources of feedback data.

Text analytics can help businesses listen to the right stories by extracting insights from free text written by or about customers, combining it with existing feedback data, and identifying patterns and trends. Manual analysis alone is unable to capture this level of insight due to the sheer volume and complexity of the available data. Sifting through all this content would be too time-consuming to be done manually, but understanding the insights held in this text is critical to get an accurate view of the voice of the customer.

The Next Chapter of Text Analytics

Text analytics is already developing a solid foothold, and it will continue to be a necessity in 2017 and into the future. Text analytics implementation was originally slow to gain traction, as there were only loosely integrated, standalone solutions available and in many cases it took some time for companies to realize the true value behind the sophisticated analytical solution. In 2016, however, we've seen adoption rates increase, and we expect this to continue through 2017 and beyond.

Why are we seeing this uptick? In part, it's the increase in the sheer quantity of free-form text data available. Add to that the huge strides in technology -- text analytics can now find value in this data as part of an enterprise-ready, integrated solution. Humans analyzing text by hand, although sometimes very accurate, can be highly variable; some text analytics solutions are more than 90 percent accurate -- and the speed is exponentially faster.

Not Everyone Is Jumping On Board

Although we expect an increase in text analytics adoption rates in the coming year, not every company will adopt technology to analyze unstructured data. There are many reasons why, including:

• Structured data is core to existing research practice and changing these practices to include unstructured data requires planning, discipline, and additional resources
• It's harder to collect, measure, and analyze unstructured data as it exists in many different sources
• Capturing and analyzing unstructured feedback from certain sources (e.g., social media) can potentially present challenges to the codes of conduct to which organizations must adhere

The Benefits of Modern Text Analytics

The above challenges will still be present in 2017. However, the exponential rise in unstructured text data means that it can no longer be ignored if companies want to stay competitive. Text is, after all, the primary method used for recording thoughts and feelings, for expressing ideas and reasoning. More than ever before, customers are finding they have a voice.

The idea of being able to use comments -- sometimes tens of thousands of comments -- to get genuine insight into what someone is thinking may sound too good to be true. Many argue that technology can never pick up the nuances of language (sarcasm, irony, etc.) as well as a real person.

However, people themselves can have vastly different interpretations of the same statement. The right text analytics solution can produce results that achieve accuracy levels the same as or better than a human analyst in a fraction of the time.

As organizations take on text analytics, they should utilize tools that allow them to mine the text data for relevant categories of content, automatically determine sentiment by category, and correlate insights using the same categorization and sentiment model across all feedback channels.

Using available text analytics solutions, businesses can reach a whole new level of insight into what is being said about them. These tools allow organizations to match sentiment from both positive and negative feedback, analyze sentiment across categories that suit their business, receive timely alerts about sentiment changes, and align those insights with key customer metrics.

In 2017, expect new research to bring deep-learning techniques to text analytics -- this should improve its adaptability to different domains and languages, accelerate initial configuration, reduce maintenance, and improve accuracy. Deep learning can automatically determine the meaning of words and their association with other words, which can help to categorize sentences into topics and determine the sentiment associated with that topic.

How to Achieve Success in 2017

As text analytics grows in popularity, organizations should be aware of these key considerations:

• One size does not fit all. Look for flexibility in analyzing your data according to your business needs. Seek expert help when starting out, but look to take ownership as soon as you can.
• Mine and model your data. Ensure your model works with the way your customers talk to capture key insights.
• Make sure you use all of your data. Don't just focus on the quantitative feedback or results -- verbiage across all sources of text feedback can truly drive the action.
• Start small and grow the program at a pace you can handle.
• Act on your results.

The last step is especially critical. Technology isn't the be-all and end-all for text analytics. Organizations must do something with the resulting data to turn it into meaningful and actionable information that adds value for both their customers and their own bottom line.

As businesses look to put their best foot forward in 2017, using text analytics to derive structure from unstructured data is an essential place to start.