How to Improve Big Data ROI
ROI may look to be a high bar for specific data projects, but a holistic approach to data can provide ample opportunities to recoup big data technology investments.
- By Chuck Currin
- February 10, 2016
The hype around big data appears to be at a peak right now, and yet the ROI of big data projects has typically not met expectations. A Wikibon project determined that there was only a 55-cent return on each dollar spent on big data projects in a study published in August 2013 (Floyer, 2015).This determination was made after surveying a pool of technical managers who had implemented big data projects. Additionally, these managers shared the fact that they were expecting a three to four times return on their investment. Clearly, expectations and reality were far apart.
Furthermore, after surveying the users, the researchers identified several main reasons that big data initiatives failed, including a lack of relevant business use cases, a shortage of skilled big data talent, and immature technology (Kelly, 2013). We’ll examine each of these reasons and identify how they can be minimized to increase the chance of achieving ROI.
Business use cases are essential to successfully extracting value out of big data technologies. In the article Finding Use Cases for Big Data published in BI This Week, author Matt Lindsay explains that “An effective use case is one in which the application of analytics to a business challenge provides benefits sufficient to justify the investments required to acquire, prepare, analyze, and act on the data. Even as the cost of data capture and storage and analytical processing power fall, this ROI threshold is a high bar for potential use cases to exceed.”
Business use cases should drive the big data project, so business stakeholders should be involved from day one, just like in any other data initiative. All too often with big data, a data initiative is driven by the IT organization without a measurable business outcome in mind. By enlisting the business, the true stakeholders will have “skin in the game” and will be focused on measurable outcomes. An example of an application of analytics with a good chance for success is introducing a customer-churn model. Once the model is in place, the business will be able to baseline churn numbers and the impact of pricing changes on its monthly churn rate.
Another issue that often has an impact on the success of big data projects is a shortage of talent. The skills necessary to crunch unstructured data and to apply complex statistical modelling to data are not keeping up with industry demand. As a result, big data projects are often driven by consulting resources. Once the project is delivered and the consultants leave, there’s no one left behind to maintain or further the big data initiatives. Any enterprise with a plan to introduce big data projects into its organization should plan ahead for staffing. This could include a mix of “trusted advisors” from the consulting world and internal staff. A good approach to building the right staff is to work closely with consultants to hire employees with an aptitude for big data problems and groom them to be staff members. Significant project risk is mitigated by having the right staff on hand, a plan for acquiring more talent with the right aptitude, and a program for effective training.
Much of big data technology, particularly the Hadoop stack, is a work in progress. Unlike conventional data warehousing, which has reached market maturity and has many commercial vendors, many of the big data technologies are based on open source frameworks. This requires technology management that monitors the maturity of the technologies and works with the business on adopting big data technologies only when they are mature and useful. Keeping an eye on business goals, big data technologies should be adopted when they can have a measurable impact on the business. This will involve a combination of solid business use case and proper technology resources when it makes sense relative to the industry adoption of the technology.
Much like big data projects, traditional date warehouse projects have had issues with high failure rates. As recently as 2005, the Gartner Group was reporting that 50 percent of all data warehouse projects failed. One of the most common reasons was lack of involvement of the business, so the first step in getting value from a big data project is to engage the business and ensure that project goals are aligned with the business. Additionally, a staffing plan with skills aligned with the business goals will help mitigate project risk.
Finally, the right way to assess any technology adoption is within the context of your particular business and by weighing the risks and rewards of early adoption. Big data is no different in this regard. Decision makers must make the call for being an early or late mover, depending on whether there are relevant use cases that align with company goals and whether the risk of being an early adopter can create a sustainable competitive advantage.
ROI may look to be a high bar for specific data projects, but a more holistic approach to data, particularly the indiscriminate pooling of data into a data lake, can provide ample opportunities to recoup big data technology investments through continuous improvement of business models. The earlier churn model that was described is likely to take multiple iterations and observations to optimize. Having the data lake will enable technical staff to expose and integrate additional metrics to the model much more quickly.