Big Data in its Infancy
A new survey highlights how enterprises are collecting customer and product data and their use (and projected use) of external big data.
- By James E. Powell
- April 9, 2013
There's no question that big data is of big interest, yet according to the results of a new survey of 339 data management professionals involved in their organization's data management tools and processes, most organizations still haven't developed or implemented a big data strategy.
Few organizations are taking advantage of product, customer, or other data sources. For example, just 12 percent say they are "currently executing against a big data strategy in daily operations." What's holding the others back? Respondents say they don't know enough about big data (21 percent), don't understand the benefits (15 percent), lack business support, or lack data quality in their existing systems (both at 9 percent).
In the survey, conducted by SAS and SourceMedia, respondents (70 percent of whom work in IT) say that of the data used to make decisions for the organization, 43 percent is customer data, 18 percent is product data, and 10 percent is employee data. When it comes to customer data, two-thirds of respondents (66 percent) collect business consumer data (that is, business-to-business data) and 59 percent collect end-consumer data.
On the business-data side, 62 percent of respondents collect data from the selling side (outward-looking aspects of product companies, such as sellable SKUs, customer purchases of products, stocking details, sales, etc.) and nearly as many (61 percent) collect product data from the buying side (the internal-looking product data, such as product parts numbers and suppliers).
As an example, purchasing and operations managers are mostly interested in product information. Sales and marketing are mostly interested in customer information.
Madhu Nair, global product marketing manager for data management at SAS, a firm specializing in business analytics, told BI This Week that where organizations can derive the most value depends on several factors, including where the most business benefit resides, which is easier to tackle for quick wins, the company's analytical maturity, and the size of product line versus customer base. (Nair is responsible for managing SAS's data management products.)
When it comes to what impacts how an organization collects and manages its customer and product data, 76 percent said the need for more detailed analysis to support business objectives had a very high or the highest impact for those using customer data; 70 percent of respondents using product data cited the same factor. Close behind was an organization's increased need for internal reporting (at 68 and 69 percent respectively), the growing need for information access (at 67 and 63 percent), and its organizational or business growth (at 66 and 56 percent).
Unfortunately, enterprises are encountering problems with their customer and product data. For example, when asked about the management of their data, nearly 6 in 10 reported that data quality and accuracy were problematic at their organization. The difference between product data and customer data users was just one percentage point. Also on their list of concerns: accessing the right data, reconciling disparate data, and a lack of an organization view into the data. The timeliness of the data was also mentioned by at least 46 percent of respondents.
Given the pressures put on organizations to collect data and the problems they have, the most interesting results were about the state in which organizations are using external big data to make business decisions. (The survey defined big data as the "massive amounts of data collected over time that are difficult to analyze and handle using common database management tools," noting that "Big Data may include business transactions, photos, surveillance videos and activity logs such as sensor data as well as unstructured text posted on the Web, such as blogs and social media.")
Only 12 percent are actively using external big data; 5 percent are in the testing phase, and 11 percent are in the planning stage of their project. Although 39 percent claim to be exploring or considering use of external big data, 23 percent are not (9 percent don't know). Of those actually employing a big data strategy, 23 percent say their operations/processing area has such a strategy; 22 percent say it's marketing employing the strategy, and research/development and customer service both came in at 20 percent. Of those not actively using external big data, only 14 percent of organizations in the survey are very likely (and another 18 percent are likely) to use external big data by next year to make business decisions.
Nair explained that the numbers, although low, are not surprising. "Actually, numbers about big data usage are all over the map still. Big enterprises have some budget to explore big data. Startups are starting off with big data and the cloud. Midsize companies are the ones that are not able to get started as quickly. The costs are higher for skills and they still don't have personnel to execute on a big data strategy." Nair also noted that the enterprise's industry can also be a factor. "Financial services are one area adopting big data strategies early. The education industry is yet to start really using big data."
For those organizations that have implemented a strategy, only 64 percent have implemented a database management program, and just over half (53 percent) have implemented data security management. Nearly as many (52 percent) have a data warehousing or business intelligence management program in place. Other data management programs fare far worse: only a third (33 percent) have a data governance program in place for their customer or business data; less than one-quarter (23 percent) have MDM in place, and 21 percent are using metadata management.
When asked about the low figure for data governance, Nair told BI This Week that "Both big data and data governance are relatively new concepts. Organizations are still maturing in both these fields. In big data governance, security guidelines are still being drawn to establish data ownership and customer data privacy. Furthermore, with big data, everything is big. When operating at scale, benefits are big and challenges are big. Big data governance will be a fast-changing field as cloud, big data, and governance all mature."
Even for those with a strategy, use of data is still in its infancy; 36 percent say no more than a quarter of their customer data comes from external big data sources; the figure is 50 percent for product data.
When asked about the low percentage of enterprises using external big data, Nair pointed out that "Big data is here to stay and SAS In-Memory technologies are already delivering big benefits to organizations around the world including ContactLab, Cosmos Bank, Hong Kong Efficiency Unit, and The Internal Revenue Service. Those who kick off big data strategies early will see greater competitive advantage than those who wait."
The report is available at no cost here, although registration is required for access.