Executive Summary | Practical Predictive Analytics
- By Fern Halper, Ph.D.
- May 29, 2018
Predictive analytics is now part of the analytics fabric of organizations. TDWI research indicates that it is in the early mainstream stage of adoption. Yet, even as organizations continue to adopt predictive analytics, many are struggling to make it stick. Challenges include skills, executive and organizational support, and data infrastructure issues. Many organizations have not considered how to practically put predictive analytics to work, given the organizational, technology, process, and deployment issues they face. Addressing these issues involves a combination of traditional and new technologies and practices. Some practical considerations highlighted in this report include:
Skills development. Organizations are concerned about skills for predictive modeling. These skills include understanding how to train a model, interpret output, and determine what algorithm to use in what circumstance. In fact, skills ranked as the biggest barrier to adoption of predictive analytics, with 22% of respondents citing this as the top challenge. To address the challenge, organizations are looking to improve the skills of current employees as well as hire externally. They are also looking to use some of the new breed of automated, easy-to-use predictive analytics tools that contain embedded intelligence. Although only 16% use these tools today for predictive analytics and machine learning, an additional 40% are planning to use them in the next few years. Analytics platforms that provide interfaces for multiple personas are also at the top of the list of technologies organizations think can help.
Model deployment. In this study, respondents are using predictive analytics and machine learning across a range of use cases. Those exploring the technology are also planning for a diverse set of use cases. Yet, many respondents are not considering what it takes to build a valid predictive model and put it into production. Close to 80% have some controls in place (such as any model built is reviewed by an expert), but only about 50% have a DevOps team or another group that puts machine learning models into production, maintains versioning, or monitors the models. Fewer than 25% register their models. Respondents report it can take months to put models into production.
Infrastructure. On the infrastructure side, the vast majority of respondents use the data warehouse, along with a variety of other technologies such as Hadoop, data lakes, or the cloud, for building predictive models. The good news is that these respondents appear to be looking to expand their data platforms to support predictive analytics and machine learning. The move to a modern data architecture to support disparate kinds of data makes sense and is needed to succeed in predictive analytics.
This TDWI Best Practices Report examines how organizations using predictive analytics are making it work. It looks at how those exploring the technology are planning to implement it. Finally, it offers recommendations and best practices for successfully implementing predictive analytics and machine learning in organizations.
Hortonworks, SAP, and Tellius sponsored the research and writing of this report.
About the Author
Fern Halper, Ph.D., is vice president and senior director of TDWI Research for advanced analytics, focusing on predictive analytics, social media analysis, text analytics, cloud computing, and other “big data” analytics approaches. She has more than 20 years of experience in data and business analysis and has published numerous articles on data mining and information technology. Halper is a coauthor of Big Data for Dummies as well as other “Dummies” books on cloud computing, hybrid cloud, service-oriented architecture, and service management. She has been a partner at industry analyst firm Hurwitz & Associates and a lead analyst for Bell Labs. Her Ph.D. is from Texas A&M University. You can reach her at [email protected], @fhalper on Twitter, and on LinkedIn at linkedin.com/in/fbhalper.
About the Author
Fern Halper, Ph.D., is vice president and senior director of TDWI Research for advanced analytics. She is well known in the analytics community, having been published hundreds of times on data mining and information technology over the past 20 years. Halper is also co-author of several Dummies books on cloud computing and big data. She focuses on advanced analytics, including predictive analytics, text and social media analysis, machine-learning, AI, cognitive computing, and big data analytics approaches. She has been a partner at industry analyst firm Hurwitz & Associates and a lead data analyst for Bell Labs. Her Ph.D. is from Texas A&M University. You can reach her by email ([email protected]), on Twitter (twitter.com/fhalper), and on LinkedIn (linkedin.com/in/fbhalper).