TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Advanced Analytics: A Look Back at 2016 and What's Ahead for 2017

Greater interest (and deployment of) AI, advanced analytics, and open source are among the trends you'll be seeing in 2017.

• By Fern Halper
• December 16, 2016

In 2016, advanced analytics maintained its momentum in the enterprise. Technologies such as predictive analytics, geospatial analytics, and text analytics continued to become mainstream. Other technologies such as machine learning and IoT analytics became hot market topics. TDWI expects these technologies to continue to grow in importance. We also anticipate several other potential advanced analytics hotspots in 2017.

2016 Trends

The Year of Machine Learning

Machine learning -- the science of getting computers to act without being explicitly programmed -- was, and will continue to be, an extremely hot topic in 2016. Although the technology has been around for decades (think decision trees), it received renewed interest as data volumes continued to increase and organizations looked to machine learning to help in tasks such as predictive analytics. For instance, machine learning can be applied to predict the probability of churn, a part failing, or a patient being readmitted to a hospital. However, machine learning is also being used for voice and image recognition, in gaming, and in self-driving cars. It is an important component of AI (artificial intelligence) that we see as a hot spot in 2017 (see below).

IoT Continued to Develop Momentum

The Internet of Things (IoT) -- a network of connected devices that can send and receive data over the Internet -- continued to interest organizations. The idea behind IoT has also been around for years, but the combination of cheap computing resources, advances in microprocessors, and more advanced software continued to make this a reality. This network is a trend in and of itself, but the analytics that can be performed on IoT-generated data is where the value is. Although only a small percentage of organizations surveyed in our research are utilizing IoT data today (17 percent in our recent TDWI Best Practices Report: Big Data and Data Science) that number is slated to triple in the next few years if users stick to their plans, which is one reason Philip Russom cites this as a hot spot in 2017.

The Citizen Data Scientist Continued to Grow in Popularity

The terms data science and citizen data scientist grew in prominence in 2016. Citizen data scientists are the next generation of statistical explorers, sometimes from nontraditional backgrounds, who are variously self-taught, self-starting, self-sufficient, and self-service in orientation. These are often business users or analysts who may not have formal training in statistics or math but perform advanced analytics using some of the easy-to-use advanced analytics software being marketed by analytics vendors.

In a recent TDWI survey, more than half of respondents stated that they would grow their data science effort internally using these citizen data scientists. There is good and bad here. It is great that tools are so easy to use; they pick the right model and they explain the results. However, it is still incumbent on whomever is building the model to make sure they know what is happening behind the scenes and that controls exist to validate models before they are put into production. Otherwise, we expect to see cases where models end up making big mistakes and data science is targeted as the culprit rather than poor training.

2017 Anticipated Hot Spots

The three trends above will continue to be hot trends in 2017. However, there are other emerging trends that are also worth noting.

The Growth of AI and AI Applications Will Continue

The term artificial intelligence (AI) was first coined by John McCarthy in 1956 during a conference to explore ways to make a machine that could reason like a human. AI declined in popularity but has seen a recent resurgence in interest as data size and diversity continue to grow and the cloud becomes a popular option for scaling compute power and data storage. Today the term AI is defined as the science of creating intelligent computer systems that can form tasks without human interference.

AI and its subcomponents (machine learning and natural language processing are two) are being woven into the analytics arsenal of various departments across industries. New companies are forming around AI. It is being used in marketing for next-best offer and real time customer support. It is used to determine fraud. It is being used in applications -- e.g., to inspect infrastructure, as virtual assistants in casinos, in "farmbots" (robots in farming), and to detect anomalies in X-rays and MRIs in oncology. We expect to hear more about AI and AI embedded in applications in 2017.

Adoption of Real-time Data and Stream Analytics Will Continue to Increase

Real-time streaming analytics, which involves analyzing data in motion, is also poised to grow if users stick to their plans. TDWI is hearing from companies that are looking to analyze streaming data utilizing more sophisticated algorithms, and we expect to hear even more in 2017. Streaming data and real-time analytics often goes hand-in-hand with IoT and we expect those applications to grow. Streaming analytics will also be used in other applications -- for example, in security analytics.

Open Source Will Become Hotter for Analytics

Open source provides a community of innovation for analytics. It has been hot and it is getting hotter. In recent TDWI surveys, we're seeing a lot of interest in R, Spark, and python for data science and big data analytics projects. We expect this to continue to grow.

As-A-Service Analytics Will Gain Momentum

As more organizations look to embed analytics into their application development, we expect more vendors will make "as-a-service" cloud marketplaces available. These marketplaces will provide analytics applications such as fraud-detection-as-a-service or churn-reduction-as-a-service. They will also provide smaller services, such as text sentiment services that can be embedded into applications.

The Bottom Line

The upshot of these trends? Many organizations are embracing analytics and moving to use newer and more advanced analytics techniques, both for analytics and analytics application development. Those who have not will want to. As organizations mature in their use of analytics, they will also have to become more mature in issues such as data security, data privacy, and data governance. These are some of the topics TDWI will be researching more in 2017.