Executive Summary | Data Science and Big Data: Enterprise Paths to Success
- By Fern Halper, Ph.D.
- December 21, 2016
Big data and data science can provide a significant path to value for organizations. These technologies, methodologies, and skills can help organizations gain additional insight about customers and operations; they can help make organizations more efficient, be a new source of revenue, and make organizations more competitive.
Although many companies are still analyzing structured data, “newer” data sources such as text data, streaming data, and geospatial data are becoming part of an evolving data landscape. However, businesses often struggle with putting an effective analytics and data science strategy together. Part of the issue is with technology. Additionally, there are organizational challenges that must be addressed, which often include building cultures, hiring the right people, and organizing to execute.
TDWI Research finds that there are many paths to value. On the technology front, our survey respondents are utilizing open source and commercial packages to drive big data value. They are using a mix of on-premises and cloud technologies. They are using data warehouses together with newer technologies such as Hadoop and Spark to manage and process big data. They are deploying appliances, MPP (massively parallel processing) databases, and other solutions to meet their big data management needs. A new ecosystem is evolving to support big data and data science.
On the analytics front, data scientists and others needed to succeed in big data are often hard to find. Survey respondents are using different approaches to build data science skills. They are hiring from the outside as well as trying to grow talent internally. They are often looking to business analysts to become more sophisticated analytically to supplement data science expertise. Some are using a team approach. Many organizations are creating centers of excellence to provide analytics and big data expertise and to disseminate learning. Few are hiring chief analytics officers or chief data scientists. Most look to the VP of analytics or the CIO to help them in their efforts.
This TDWI Best Practices Report examines organizations’ experiences with and plans for big data and data science including both technology plans and organizational strategies. It also looks at various big data challenges and how organizations are overcoming them. It examines the importance of new open source models. Finally, it offers recommendations and best practices for successfully implementing big data programs in the organization.
A unique feature of this report is its examination of the characteristics of companies that have actually measured either top-line or bottom-line impact with big data and data science. In other words, it explores how those companies compare against those that haven’t measured value.
IBM, MapR, OpenText, and Snowflake Computing sponsored the research and writing of this report.
About the Author
Fern Halper, Ph.D., is vice president and senior director of TDWI Research for advanced analytics. She is well known in the analytics community, having been published hundreds of times on data mining and information technology over the past 20 years. Halper is also co-author of several Dummies books on cloud computing and big data. She focuses on advanced analytics, including predictive analytics, text and social media analysis, machine-learning, AI, cognitive computing, and big data analytics approaches. She has been a partner at industry analyst firm Hurwitz & Associates and a lead data analyst for Bell Labs. Her Ph.D. is from Texas A&M University. You can reach her by email ([email protected]), on Twitter (twitter.com/fhalper), and on LinkedIn (linkedin.com/in/fbhalper).