That Was the Year That Was: Major Data Warehousing Events of 2013 (and Predictions for 2014)
Analyst Mike Schiff looks back at the accuracy of his 2013 predictions and offers a short list of trends to watch in 2014.
- By Mike Schiff
- December 17, 2013
As 2013 draws to a close, it's time to once again review some of the major events of the year and speculate on what might occur in 2014. First, however, let's review my predictions from last year.
Results of Last Year's Predictions
In December of 2012, I predicted that the following would occur this year.
Industry Consolidations Will Continue:
My prediction about additional consolidations proved correct. Companies large and small continued to acquire other companies to gain complementary technology and/or additional market share. See the "Major Data Warehousing Events of 2013" section (below) for details. However, what I thought would be the most likely acquisition target, Informatica, once again continued to remain independent.
Predictive Analytics Will Become Better Predictors:
Technologies such as in-memory databases, commodity-based MPP processing, NoSQL databases, solid state storage, text analytics, and analytic appliances have continued to increase in power while decreasing in cost. Harnessed to analyze big data sources such as social media, clickstreams, RFIDs, search logs, etc., this has resulted in more accurate predictive analytics and, thanks to data visualization technology, has made it much easier for business users to utilize and comprehend. In addition to traditional revenue generation applications, there has been increased deployment of predictive analytics in healthcare, risk analysis, and fraud detection and prevention.
Cloud-Based Computing Growth Will Accelerate:
This was certainly the case. Once viewed as a way for small-to-medium-sized organizations to minimize capital expenditures and avoid the need for expensive staff resources associated with on-premises solutions, the cloud is now a mainstream channel for software, computing power, and storage. Many software vendors that once offered token cloud-based solutions in order to claim a cloud presence now aggressively market them if only to compete at the departmental level against entrenched on-premises competitors. Several hardware vendors also now offer cloud-based platform services to attract new customers and allow their on-premises customers the flexibility to accommodate peak demands.
Ease-of-Use Will Become Even More Important:
This prediction was proven by the success of several self-service data discovery BI products. BI has become much more pervasive, and business users are seeking solutions they can deploy without having to understand the details of underlying algorithms and technology.
Windows 8 Won't Set the Business World on Fire:
This has proven to be correct. According to a Computerworld article based on data from Net Applications, as of November 2013, Windows 8 and 8.1 have achieved a combined share of slightly over 10 percent of the Windows market. Although this exceeds Vista's market share, it greatly lags the market share of Windows XP. Furthermore, Windows 7 is still installed on the plurality of PCs. Although Windows 8.x with its touch screen capabilities ships with Microsoft's Surface tablets, business users have not rushed to upgrade their laptops and desktops, and many PC vendors still offer Windows 7 as an option. However, the second half of my prediction, "Windows 8 tablets will eat into Apple's market share in the consumer market and may drive organizations with liberal 'bring your own device' to ultimately migrate to Windows 8 on their PCs," was clearly overly optimistic.
Major Data Warehousing Events of 2013
Industry Consolidations Continued:
Among the consolidations in 2013 were IBM's acquisitions that included SoftLayer Technologies for cloud computing infrastructure, Now Factory for big data analysis, and Star Analytics for financial data integration; its planned acquisition of Fiberlink Communications and its mobile management and security technology will further support customer BYOD initiatives and is expected to close by year-end.
Oracle acquired cloud vendors Nimbula and configure, price, quote (CPQ) vendor Big Machines. Cisco acquired data virtualization vendor Composite Software, and Software AG acquired complex event processing software vendor Apama from Progress Software. SAP acquired data mining vendor KXEN. Microsoft bought social-monitoring analytics vendor Netbreeze and, in an effort to expand its mobile technology, announced plans to acquire Nokia's smart devices and mobile phones business units. Actian acquired analytic database vendor ParAccel as well as data integration and embedded database vendor Pervasive Software. Informatica, in my opinion a potential acquisition target itself, remained on the offensive and acquired process automation vendor Active Endpoints.
Big Data is Really, Really BIG:
Almost every data warehouse vendor has jumped aboard the "big data bandwagon," even if they don't all define it the same way or even agree on the number of Vs (e.g. volume, velocity, variability, veracity, vulnerability, value). It seems that the sub-headline of almost every vendor press release mentioned new or enhanced big data capabilities even when functionality to handle increased data volumes was not necessarily a new feature.
IPOs Were Active Again: With Facebook recovering from its disappointing IPO in 2012, high-tech companies more actively embraced going public as a way of rewarding their early investors. Among these taking this route were Tableau (with the very clever symbol DATA) which (as of late November) had more than doubled from its May IPO price of $31. Twitter, which went public on November 7, closed its opening day with a 73 percent gain.
Cloud Storage Provider Nirvanix Shuts Down:
After originally announcing on September 16 that it would close its doors at the end of the month, Nirvanix filed for Chapter 11 bankruptcy on October 1. As clients rushed to migrate to other storage platforms, concerns have been raised about the stability of non-giant cloud vendors and how their users could protect themselves. At the very least, potential cloud users will look more closely at the financial stability of the vendors that make it to their short lists and should consider having a backup plan as a high priority best practice should their cloud vendors suffer similar fates.
IBM's October 30 announcement that it was closing its SmartCloud Enterprise service while advising customers that they would have to migrate (with IBM's help) to its acquired SoftLayer Technologies cloud service by January 31, 2014, demonstrated to users that even industry giants might not allow them to keep their original cloud plan.
Privacy Became a Very Public Issue:
With the revelation that the National Security Agency is monitoring and collecting call detail records, users are becoming increasingly concerned about privacy and misuse. Although the agency is claiming that it did not collect the actual conversations but only the "metadata" about the calls, in my opinion, one person's metadata may be another person's data. For instance, I believe that a call's origin, destination, and time length are data that can be used to discover user-to-user interactions while call length unit-of-measure and the codes for the origin and destination are metadata.
The Affordable Care Act Website was Not Production-Ready:
The rollout of the Obamacare website was a technical (and political) disaster. Most users were initially unable to sign up or even shop for the health insurance policies mandated by the act. Among the multiple causes for this fiasco were late scope changes, the need to integrate several databases, insufficient code reviews, multiple vendors and subsequent "finger pointing," and intense sponsor pressure to "go live" without adequate system and stress testing. Although almost every IT professional can cite similar horror stories (hopefully of lesser magnitude) from their own experiences, this one will serve as an object lesson to all future implementers to resist pressure to prematurely go live with data warehouse and operational systems.
Predictions for 2014
Industry Consolidations Will Continue:
Industry consolidations will continue as larger vendors try to augment their capabilities in order to deliver more pieces of a total data warehousing solution platform including data integration, database technology, business intelligence, and (in some cases), hardware. I expect vendors still lacking "big data," social media connections, and in-memory solutions to seek out and acquire smaller vendors that do. Yes, as I have been predicting for the past few years, I still expect data integration specialist Informatica to be a likely acquisition target
Desktop Virtualization Will Experience High Growth:
Although server virtualization (both on-premises and cloud-based) is well established and widespread, many organizations have yet to embrace desktop virtualization. However, as mobile devices become even more ubiquitous , additional organizations will now consider virtualizing their desktops as well. The Network Computer, evangelized by Oracle and Larry Ellison in the mid-1990s, may soon have its day!
BI Tools Will Work With More Data Types:
Although relational databases once ruled the analytic world, the growth of NoSQL (and NewSQL) data structures has accelerated with the increased need to analyze massive amounts of data. There is no standard definition of what constitutes a NoSQL database (and non-relational won't do as it would include legacy offerings like ADABAS and IMS), but there are well over 100 "NoSQL" products. BI vendors will need to be able to analyze data contained in these data structures and though many can now accommodate the key/value Hadoop Distributed File System, they will expand their functionality to also work with Graph and Document NoSQL structures. I expect to see many press releases mentioning "BI vendor X has been certified to work with NoSQL database Y."
The Internet of Things Will Expand:
The Internet of Things will move towards the Internet of Everything as objects including automobiles and home appliances join televisions, tablets, smartphones, and sensors (including medical and fitness devices) to become Internet-enabled. I once laughed at a refrigerator that included a television screen, but consumers might come to appreciate one that remotely alerted them when their milk was getting old or their egg bin was empty.
Privacy and Data Security Will Become Very Important:
The uproar over government access to what users considered their private data will place data privacy as a major factor in IT cloud implementations and outsourcing decisions. Some organizations may favor non-U.S. vendors in order to circumvent the reach of the Patriot Act and U.S. government agencies such as the National Security Agency. Of course, these organizations may be fooling themselves as offshore cloud resources may not be beyond the perusal of their own governments.
Analytics Will Expand, Even If IT Budgets Don't:
Although I expect a marginal expansion of IT budgets in 2014, I believe BI initiatives such as predictive analytics and the desire to access data from anywhere will gain more than their fair share. I consider the ability to comparison shop from a mobile device to be an example of user consumer intelligence, and business users will demand comparable functionality from the business intelligence applications. Furthermore, many organizations now realize that data mining is a necessity, not a luxury, and will step up their efforts if only to remain competitive.
A Government Database Will Suffer a Major Security Breach:
This year we saw numerous new malware and virus attacks, and I expect this to continue. Unfortunately, I see the Affordable Care Act website as a likely target due to design and implementation practices that include multiple vendors, late specification changes, numerous integration points, and insufficient code reviews that might have caught (hopefully) inadvertent security holes.
I'll report on the accuracy of these predictions next year, when I make new ones for 2015.