Emerging Technologies: And the Beat Goes On
We owe it to our users (and to our careers) to learn about new technologies.
- By Mike Schiff
- December 2, 2014
Perhaps the only certainty we have concerning data warehousing technologies is that existing ones will continue to evolve as new ones continue to emerge. Although the basic components of a data warehousing architecture (i.e., data collection, data storage, analysis, and distribution) will always exist, the technologies to do so will continue to improve.
In the early days of business intelligence (first known as decision support systems) and data warehousing, reports and analyses were generated on "green bar" paper on printers attached to mainframe computers. Today, we typically receive them via Web browsers and apps that might reside on our mobile devices.
I personally have observed input evolve from manually flipped sense switches to paper tape to punch cards to dumb terminals via line editors (and then full-screen text editors) to Web browsers to machine generated data. Along the way, the format has evolved from tables to graphs to today's interactive dashboards and other visualization techniques. We have certainly moved far away from the day of batch processing and overnight reports to today's on-demand real-time analyses.
Computer power has evolved from mainframes to minicomputers to microprocessors to personal computers to tablets, smartphones, and other mobile devices. Computing and storage platforms now include on-premises and the cloud (a much-improved reincarnation of time-sharing).
Database technologies have evolved from flat files to indexed sequential structures to chained DBOMP (originally disk bill-of-material processor but renamed by IBM marketing as Database Organization and Maintenance Processor) structures to networked and CODASYL models to relational -- and now to the current craze of NoSQL non-relational structures. Many data warehouse practitioners have learned that this is not an either/or choice. Rather. they now embrace multiple structures, in particular relational and the many NoSQL variants, within an overall data warehouse architecture.
The cry for a single physical enterprise data warehouse is no longer heard; in fact, we not-so-quickly learned that attempting to build one was comparable to attempting to "boil the ocean." Our data warehouse architectures now include data marts, data warehouses, operational data stores, and appliances with the definition delineating any two being somewhat fuzzy. For example, a special-purpose appliance could readily serve the platform for a single-subject data mart.
Among the relatively new technologies that are already present, but considered emerging, are robotics, quantum computing, 3D printing, smart watches, IBM Watson (and is ability to leverage natural language processing, data retrieval, massively parallel processing, and artificial intelligence), and the increasingly ubiquitous Internet of things.
As we have moved away from having to be in physical proximity to the source of computer power to input and delivery via increasingly sophisticated mobile devices including "wearables," I predict it is only a matter of time until we will be able to directly interface via our thoughts through implants and even non-invasive mind-machine interfaces. Although this has long been a topic of science fiction, we have already advanced to the point where thoughts can now control prosthetic devices in patients with spinal cord injuries.
Thanks to evolving technology, we are now able to undertake analyses that we could not have done several years ago. For example, we now analyze and mine complete (and vastly larger) data sets where in the past we might only have used samples. We can conduct our analyses from our mobile devices on the fly while we are on the move.
No one can predict with certainty what the future will bring, but I suspect that at the very least we will move away from the constraints imposed by binary technology when quantum computing becomes a commercial reality. We must at least recognize that technology will move forward and that today's solutions will seem primitive by tomorrow's standards. We owe it to our users (and to our careers) to continue to learn about new technologies so we can be prepared to embrace them as they mature.