That Was the Year That Was: Major BI Events of 2018 (And Predictions for 2019)
Industry analyst Michael Schiff shares the results of last year’s predictions and what he foresees in the coming year.
- By Mike Schiff
- December 17, 2018
As 2018 draws to a close, it's time to once again review some of the major events of the year and speculate on what might occur in 2019. First, let's review my predictions from last year.
Results of Last Year's Predictions
In December 2017 I predicted that the following would occur in 2018:
Industry consolidations will continue: My predictions about continuing industry consolidations once again proved correct. Companies of all sizes continued to enhance their ability to offer complete data warehouse and analytics solutions rather than individual point products. Companies without the necessary in-house expertise continued to rely on acquisitions rather than develop their own organic solutions. See the "Major Data Warehousing Events of 2018" section (below) for examples.
Data lakes will be drained: Although organizations continue to build and mine data lakes, many organizations have come to realize that their expectations exceeded the results because without proper data quality they were actually polluting their data lakes and building data swamps. That said, I believe my prediction was somewhat correct because many organizations took steps to better evaluate potential sources and archive, reject, or delete data that had little or no expected usage.
Data warehouse automation will extend to query automation: Although the automation of data warehouse design and population continues to be enhanced by vendor enhancements (including natural language capabilities), I was somewhat disappointed by the seeming lack of focus on machine learning and other AI technology to anticipate and suggest meaningful, and perhaps unexpected, user queries.
Security and identity theft prevention will be primary drivers: This was certainly proven true as various data breaches and associated scandals at organizations including, but certainly not limited to, the U.S. Centers for Medicare & Medicaid Services (CMS), Facebook, Marriott, Orbitz, and Under Armour reported breaches. People are more concerned than ever about the security of their data and individuals are beginning to actually read the fine print in service agreements and choosing to opt out of "free" services rather than share their personal data.
Monetization of smart-home device data: The Internet of Things with smart devices including thermostats, lighting, video door bells, and security devices -- as well as digital assistants such as Amazon Echo and Google Home -- yield a tremendous amount of data that can be used for purposes such as one-on-one marketing. As companies continue to seek ways to monetize the data they collect, concerns about user privacy have made them reluctant to publicize their efforts and successes.
Escalating demand for data scientist and AI practitioners: With the growth of analytics, artificial intelligence, and machine learning, the demand for data scientists continued to expand. This prediction was right on target and supported by a May 2018 Bloomberg article citing data scientist as "This Is America's Hottest Job" as well as numerous studies and articles by publications and organizations including Forbes, Fortune, IBM, and KDnuggets.
Major Data Warehouse Events of 2018
Industry consolidations continue: Among the acquisitions announced and/or completed in 2018 were Broadcom's acquisition of CA Technologies (see below) and Microsoft's acquisitions of cloud software vendor GitHub, conversational AI vendors XOXCO and Semantic Machines, and codeless development and implantation AI vendor Lobe.
IBM announced the pending acquisition of open source vendor Red Hat and SAP touted the pending acquisition of cloud-based survey specialist Qualtrics International. Oracle acquired cloud-based, AI-driven BI vendor DataFox and data science platform vendor Datascience.com.
Federal versus state net neutrality battles erupt: With the Federal Communication Commission's (FCC's) repeal of the Obama administration's net neutrality rules, several states have filed suit against the FCC to reestablish them. California has taken the further step of establishing its own net neutrality laws that are now on hold pending a Federal appeals court review of a challenge by the U.S. Justice Department.
Broadcom acquires CA Technologies: In an effort to establish a software presence, Broadcom, a communications hardware vendor, acquired CA Technologies, a software vendor with a multitude of mostly mainframe software tools, utilities, and other products that it acquired over several decades. CA Technologies has a reputation as the place where software companies with financial difficulties go to be bought lest their products disappear completely. By continuing to support these products, sometimes only in a zombie-like state, CA provided a safety net for IT organizations and allowed them time to possibly migrate to alternative technologies. Given the high price that Broadcom paid for CA, I expect the discontinuance of support for some of CA's unprofitable products and believe that reports of post-acquisition layoffs at CA are credible.
California enacts data privacy laws: Many organizations are still trying to ascertain the impact of the May 2018 European Union's (EU) General Data Protection Regulation (GDPR) which regulates the processing, collection, and retention of customer and employee personal data. They need to also consider the implications of the California Consumer Privacy Act of 2018 which is scheduled to go into effect on January 1, 2020.
Cloud computing vendor wage war: Although Amazon Web Services seems to be the leader in the public cloud infrastructure service market, Microsoft Azure, Alibaba, Google Cloud Platform, and IBM Cloud are still major players. Other vendors (such as Oracle) continue to strive to increase their market presence. The cloud is an expanding marketplace, not a zero-sum game, and even a decline in market share can still result in increased cloud revenue for participating vendors.
My Predictions for 2019
Industry consolidations will continue: Once again I expect major platform vendors to target smaller product and service specialists to obtain complementary technology to establish or augment their capabilities and expertise especially in technologies related to cybersecurity, privacy, analytics, cloud computing, artificial intelligence, and machine learning.
Privacy and security will be more important than ever: I expect a slew of legal challenges to the California Privacy Act of 2018 prior to it taking effect, signaling the importance that individuals and organizations place on the privacy of their data and this will likely lead to stricter federal data privacy standards. Just as some automobile manufacturers adhere to California pollution standards that are stricter than federal emission standards, companies that collect personal data about U.S. citizens will likely conform to the California standards. The November 30 revelation that Marriott's Starwood had suffered a data breach affecting half a billion guest records will serve as a catalyst for Congress to give the passage of consumer data privacy and data ownership rules and regulations a high priority.
Quantum computing becomes practical: Countries such as China, the European Union, and Russia and organizations including Google, IBM, and Intel are making significant investments in quantum computing. 2019 may be the year that resolves the entanglement barrier that has limited the number of qubits that can be linked together without interfering with each other. I expect quantum computing to move from demonstration and laboratory mode to practical implementations in applications including those concerned with climate and medical research.
Blockchain use will increase: Despite the volatility in Bitcoin's value, we must recognize that Bitcoin is an application that merely utilizes blockchain technology. Blockchain, itself, will see increased use for applications that requiring decentralized collection and verification of ledger transactions. By eliminating the "middleman," blockchain will facilitate the authentication of transactions in areas such as logistics, healthcare, fraud prevention, and finance while providing a huge data source for our data warehouses and analytics.
The jury will remain out on the SAP/Microsoft/Adobe Open Data Initiative: Announced in September 2018 at the Microsoft Ignite conference, the initiative is focused on developing a common customer data model. As of early December, I cannot find a list of other vendor partners, which leads me to believe that the initiative is Azure-centric marketecture; I don't expect it will be a raging success. It may, however, provide an impetus for the creation of a wider, vendor neutral, common data model.
Real-time consumer monitoring will improve financial and market analyses: Financial analysts have traditionally studied a company's quarterly results and associated conference calls, but tracking consumer behavior and geolocation in real time can be used to more quickly spot trends such as store visits or product sales. This will result in an increase in companies that collect and aggregate such data along with credit card transactions, website visits, etc. (hopefully after anonymizing them) for resale to financial and market analysts.
Looking Ahead to Next December
I'll report on the accuracy of these predictions next year, when I make new ones for 2020.