Emerging DW Technologies: Some Things Old are New Again
As data warehousing has continued to evolve, many vendors have introduced technologies they marketed as new and innovative -- but are they?
- By Mike Schiff
- October 21, 2010
As data warehousing has continued to evolve, many vendors have introduced technologies they marketed as new and innovative. These technologies have served to facilitate the deployment of better, faster, and (in many cases) less costly data warehouses. However, many of them have existed for well over a decade.
[Editor's note: Emerging technologies is the subject of TDWI's World Conference in Orlando.]
Let's take a look at just how new some of these technologies really are.
Column-based Databases
In the last few years, several vendors (including Infobright, ParAccel, and Vertica) have entered the column-based database market. Although Sybase IQ is thought by some as being the first column-oriented database, it is actually based on technology acquired from Expressway Technologies, a company Sybase acquired in 1994. Sybase initially marketed its column-based capabilities as Sybase IQ Accelerator, an optional extension to Sybase SQL Server, the forerunner to Sybase Adaptive Server Enterprise. Prior to its acquisition by Sybase, Expressway marketed its technology as an add-on to other databases as well.
(Commentary continues below)
Related Resources
White Papers:
Business Intelligence: The Definitive Guide for Midsize Organizations
Operational Data Warehousing: The Integration of Operational Applications and Data Warehouses
Webinars:
Data Warehouse Appliances: An Update on the State of the Art
What’s Required for Enterprise Business Intelligence Deployments
(Commentary continues)
In-Memory Architectures
In-memory data access operations are orders of magnitude faster than disk-based storage systems as conventional disk storage devices (which have mechanical constraints on access speed). However, in-memory database technology is not a new concept, and companies such as Applix (acquired by Cognos which, in turn, was acquired by IBM) have been offering it for decades. Applix TM1 Server is an in-memory OLAP server that was first offered in the mid 1980s. Note: Although solid-state storage devices drastically accelerate disk access, data still needs to be moved to memory for processing. Furthermore, although I expect the future deployment of solid-state storage to grow exponentially, today they are still relatively expensive.
What has accelerated the use of in-memory databases was the advent of 64-bit computer architecture and its ability to directly address significantly larger in-memory databases. Although some people may think that 64-bit addressing doubles the amount of memory that can directly addressed by 32-bit addresses, this is a gross underestimation. Only one additional bit (i.e., 33-bit addressing) is needed to double the amount of memory that can be addressed by 32-bit architectures. In fact, 64-bit addressing squares the amount of memory that can be addressed by 32-bit addresses and, when combined with the declining price of memory, has led to current interest in, and feasibility of, very large in-memory database technology.
Cloud Computing
The concept of cloud computing, or utilizing remote and often third-party resources for an organization's computing needs, is far from new. It dates back to at least the 1960s when it was then known as timesharing (if interactive) or remote job entry (if jobs were submitted for batch processing). Of course back then most timesharing was done using dial-up modems or acoustical couplers and a modified IBM Selectric Typewriter was a state-of-the-art input/output device with an output speed of 15 characters per second.
Software-as-a-Service (SaaS)
Generally hosted in a cloud computing environment, SaaS refers to the subset of cloud computing concerned with application software. Since many timesharing vendors offered software (both operational and analytical) as part of their services, it, too, dates back to the 1960s. In the late 1990s, SaaS had a major resurgence under the application service provider (ASP) moniker.
Dashboards
Everyone (except, perhaps, those with red/green color blindness viewing dashboards where colors indicate status) loves dashboards. Their highly visual output make it easy for users to analyze data and quickly grasp status information, identify problem areas, and view metrics such as key performance indicators. I consider dashboards to be an evolution of the user interface for what used to be called executive information systems (EISes). Dashboards have helped make EIS so much more pervasive that perhaps EIS should now be considered an acronym for "everyone's information systems."
Mobile Access
In my opinion, one of the critical success factors of a data warehouse deployment is the ability to provide the right information to the right person at the right time so that they can make better decisions. In today's world, it is paramount that this information be delivered to the information consumer's device of choice. Mobile access technology makes this possible. While today's devices include technology such as smart phones, netbooks, and tablet PCs, companies such as MicroStrategy, with its DSS Broadcaster, offered mobile broadcasting in 1998.
The Last Word
As you can see, several so-called emerging technologies have in fact been around for quite a while. In many cases, new technology is not really emerging; rather established technology is continuing to evolve. What is emerging is that their usage is accelerating and they are being deployed in an ever-growing number of data warehouse deployments.
Michael A. Schiff is a principal consultant for MAS Strategies. He can be reached at [email protected]
Related Resources
White Papers:
Webinars:
Data Warehouse Appliances: An Update on the State of the Art
November 4, 2010
Speaker: Philip Russom
What’s Required for Enterprise Business Intelligence Deployments
December 7, 2010
Speaker: Mark Madsen