That Was the Year That Was: Major Data Warehousing Events of 2015 (and Predictions for 2016)
How accurate were our data warehousing observer's predictions, and what's ahead for 2016?
- By Mike Schiff
- December 14, 2015
As 2015 draws to a close, it's time to once again review some of the major events of the year and speculate on what might occur in 2016. First, let's review my predictions from last year.
Results of Last Year's Predictions
In December, 2014 I predicted that the following would occur in 2015:
Industry consolidations will continue: My prediction about additional consolidations proved correct. Companies large and small continued to acquire other companies to increase market share and gain complementary technology (in particular cloud and big data) and provide additional pieces of a total data warehousing solution platform. See the "Major Data Warehousing Events of 2015" section (below) for examples.
User interfaces will rely less on mouse clicks: The mouse was a vast improvement over command-line interfaces, arrow keys, or even joysticks. However, tablets, smartphones, and Windows 10 are causing it to be supplanted by other technologies, including touch screens, hand gestures, styluses, and voice commands. The growth of tablets and smartphones, neither of which uses a mouse, served to verify this prediction.
Quantum computers will leap closer to reality: Although governments, universities, and companies including D-Wave Systems (which announced the general availability of a 1000+ qubit quantum processor in August 2015), Google, IBM, Intel, Lockheed Martin, and Microsoft continue to invest in quantum computing, my prediction that there would be a "quantum leap" in its commercialization in 2015 was somewhat optimistic. I still believe, however, that we will see greater commercialization of quantum computers within a few years
Watson-based cloud applications will rain from the sky: In an effort to further encourage the development of Watson-based applications, IBM continued to augment its Watson Developer Cloud services expanded APIs and SDKs and launched Watson Health to further target the medical field. As of September 2015, IBM claimed to have a cadre of over 77,000 active Watson developers and in November 2015 claimed over 450 partners with over 100 of them having already delivered applications to customers. Although applications are not quite pouring from the sky, IBM's efforts are yielding more than a drizzle.
Data warehouse automation will thrive: Data warehouse design and implementation have transitioned from an art to a science. Increased demand for analytics and user frustration with implementation delays drove the development of new and/or additional products to automate data warehouse lifecycle phases including design, creation, operations, and change management. Companies such as Attunity (which acquired BIReady's data warehouse automation technology in December 2014), Dimodelo Solutions, TimeXtender, and WhereScape have successfully targeted this market. For example, in its latest fiscal year (period ending 2/28/2015) WhereScape's software sales were up 66 percent.
Windows 10 will cause an uptick in PC sales: Windows 10 is now shipping with new PCs and other devices and although the days of a "high-growth" PC market are over, Windows 10 acceptance is well ahead of the initial growth rate for Windows 8. However, 2015 is not yet over, and during the holiday season I still expect a minor surge in new PC sales.
The battle over net neutrality will not be resolved quickly: I was off the mark on this prediction. The FCC ruled in favor of net neutrality in February 2015 by reclassifying broadband access as a telecommunications service, thus applying Title II (common carrier) of the Communications Act of 1934 to Internet service providers. That said, I believe this may change depending on the results of the 2016 elections and/or how successful various opponents of the decision are in their efforts to have the FCC decision overturned.
Major Data Warehousing Events of 2015
Continuing industry consolidations: IBM was the most active with acquisitions that included Merge Healthcare (a medical imaging platform vendor that will be enhanced with Watson's advanced analytics and cognitive capabilities); Blue Box Group (a private cloud provider powered by OpenStack); AlchemyAPI (for its text and facial recognition analysis technology and its SaaS-based APIs that will be merged with Watson's cloud services); StrongLoop Inc (for its technology to allow enterprise processes to connect with front-end mobile, IoT, and Web-based applications; Compose Inc. (a cloud-based database-as-a-service vendor whose offerings include MongoDB and PostgreSQL); Gravitant, Inc (a cloud brokerage software vendor); and Cleversafe Inc. (a storage solutions vendor).
Less aggressive acquirers included Microsoft whose acquisitions included Datazen (a mobile BI vendor) and Revolution Analytics for its expertise in R-based analytic solutions, while announcing its intention to acquire Secure Islands for its data protection technology. Microsoft also wrote off $7.6 billion as an "impairment charge" for its 2014 acquisition of Nokia's phone technology. Teradata announced in 2015 that in December 2014 it had acquired Appoxee, a mobile marketing SaaS provider.
Informatica goes private: The year I finally stopped predicting that Informatica would be acquired, it went private in a leveraged buyout by Primera Funds and the Canada Pension Plan Investment Board and other investors that included Microsoft and Saleforce.com's venture wing.
Data lakes swamp the landscape: "Data lake" was the big data buzzword of 2015. Unfortunately, many data lakes became data swamps for a variety of reasons including lack of data quality and an inability to productively understand and use much of the collected data. This further increased the already high demand for data scientists and chief data officers, two of the year's hottest job categories.
The HP Way went astray: Hewlett-Packard, a Fortune 100 company founded in 1939, split into two Fortune 100 companies: Hewlett-Packard Enterprise focused on hardware, software, and services and HP Inc. focused on PCs and printers. After years of failed acquisitions, disappointing earnings, and numerous layoffs it will be interesting to see if the individual companies can now be more successful as two separate entities. HP's "the Machine," a promising new computer architecture (see "Predictions for 2016), will be inherited by HP Enterprise.
Data warehouses and advanced analytics became more mission-critical than ever: Originally considered a "nice-to-have" that complemented an organization's operational systems, data warehouses continued to become an integral part of the operations themselves. Furthermore, advanced analytics, which in many cases are now bundled with operational systems, have become a competitive requirement. Companies recognized the need to plan for the future rather than simply analyze the past. Examples include customer cross-selling, medical diagnosis, quality control and proactive preventive maintenance, lending decisions, and just-in-time supply chain management.
Dell moves to acquire EMC: In a deal expected to close in 2016 that is being described as the largest technology deal ever, privately-held Dell will acquire publicly-held storage vendor EMC for $67 billion. In addition to cash, EMC shareholders will receive tracking stock (an equity that mimics the underlying stock's performance but has no voting rights) in virtualization software vendor VMware, of which EMC holds 81 percent. Dell Computer may have started as a PC vendor in Michael Dell's dorm room, but now it is aggressively attempting to grow its enterprise prominence in both operational and analytic environments.
Oracle releases "Security in Silicon" chip-based security: Although Oracle's move to build security features into its SPARC M7 processor family features that include hardware-assisted encryption and memory access violation protection may be an attempt to retain its security-conscious SPARC customers who might be considering moving to a non-Oracle-based cloud platform, it points out the serious issue users have about security and malware. Oracle's move demonstrates how vendors are proactively taking steps to alleviate customer concerns and serves to incite other vendors (hardware and software) to enhance their security capabilities.
My Predictions for 2016
Industry consolidation will continue: Once again I expect major platform vendors to target smaller, more focused, companies in order to obtain additional big data analysis capabilities. They will also pursue vendors of cybersecurity, data protection, virus and malware defense, and privacy technology in order to harden and lock-down their platforms.
Cloud computing will limit the growth of on-premiseS data centers: The days of the showcase "glass house" data center have long passed as more companies realize the benefits of considering computing power (both hardware and software) as a utility that can be obtained on-demand. Although an early driver (even dating back to its time-sharing origins in the 1960s) was making it cost-effective for smaller organizations to obtain computing power with minimal capital expenditures, many organizations are now questioning their need for massive, on-premise, data centers. Security concerns and protection of company proprietary information are still a factor constraining cloud growth, but technology (and common sense!) will continue to evolve to better protect cloud data.
Artificial Intelligence (AI) applications will include virtual analysts in addition to their current role as virtual assistants: The inability for a human to distinguish interactions between another human being or a computer (i.e., the Turing Test for artificial intelligence) has existed at least since Joseph Weizenbaum of MIT released his ELIZA program in 1966. AI is now an accepted tool in many operational areas including "robo-advisors" for financial planning, patient medical diagnoses, risk assessments, and virtual office assistants. Its growth in analytic settings such as analyzing cause-and effect relationships in dashboards and management reports will expand AI's usage in virtual analyst applications.
The November political campaigns will enhance user awareness of big data: As candidates deploy data analytics to help them refine their tactics and messages, the media's mention of "big data" will be heard or read by the population daily. This will serve to further drive overall interest in data warehouses and data analysis. Election night will reinforce this awareness as preliminary voter results are analyzed across many dimensions to predict the winners.
Major cloud security breaches will occur: As the Internet of Things encompasses more and more web-connected devices, cloud sites will be compromised by hackers who gain access or plant malware via device vulnerabilities such as default passwords that have never been reset. Ease of connection will become a secondary concern as security and vulnerability become primary.
HP's "the Machine" will continue to advance but will not be commercialized for several years: With an architecture that uses (1) energy-efficient,non-volatile "memristors" as memory-storage, thus eliminating the comparatively slow operation of moving data between memory (for processing) and storage (where it is retained), and (2) lasers and optical connections rather than wires, this advanced architecture promises to speed up operational and analytical processing by several orders of magnitude. I expect HP to continue to advance "the Machine" technology, including a new operating system to support it, but I don't expect it to advance beyond the laboratory for several years.
IBM's commitment will help spark enterprise interest in Spark: IBM's Spark-as-a-Service offering (IBM Analytics on Apache Spark) and its re-architecting of numerous analytic and commerce solutions, including IBM SPSS Analytic Server and IBM SPSS Modeler to leverage Apache Spark, will serve as a catalyst for other enterprise organizations to also embrace it.
Security versus privacy will become a major source of contention: Just when the pendulum seemed to be swinging towards concerns about loss of individual privacy, incidents including ISIS October terrorist attacks in Paris, appear to have moved it the other way. Although federal and local law officials will fail in their efforts to get vendors to build "back doors" into their encryption technology, I expect that national security concerns will trump individual privacy, especially because many consumers appear willing to trade anonymity for free apps. Sun Microsystems' 1999 statement by Scott McNealy that "you have zero privacy anyway, get over it" will prove to be truer than ever.
I'll report on the accuracy of these predictions next year, when I make new ones for 2017.