It’s a Brave New World for Business Intelligence: How the Cloud, Social Computing, and Next-Generation Analytic Technologies Are Redefining the Data Management Landscape
By Don DeLoach, CEO, Infobright Inc.
It's no secret that today's IT professionals need to help their organizations capture, track, analyze, and share more information than ever before. From mass quantities of transactional data, Web data, and huge and growing volumes of "machine-generated" information, such as sensor and log data, volumes are expanding into the terabyte (and even the petabyte) range.
At the same time, the ways that end users consume information are rapidly changing. Thanks to innovators like Facebook, Twitter, and LinkedIn, social computing technologies are spreading like wildfire. Companies are starting to look at how to harness social networks, blogs, wikis, and more to share business intelligence and collaborate more effectively.
As the data center strains under the need for more storage and faster performance (all while keeping costs in check), cloud computing, open source technologies, and other emerging approaches are presenting compelling new ways to manage data and consume IT services. How can IT practitioners best navigate today's rapidly changing BI landscape?
Tap into the Cloud
Although the data center is not going away anytime soon, cloud computing is certainly democratizing information access. In addition, a number of innovative software-as-a-service and "cloud-friendly" BI and analytic solutions are cropping up. There are, of course, some key considerations. Security and data privacy get the most press, but uptime, performance, and openness/portability are also important. Depending on your organization's specific requirements, there's more than one flavor of cloud, ranging from public, private, and hybrid. The best approach will ultimately depend on what's most important to your organization. Take the time to clearly define what the business needs to achieve before jumping on the bandwagon.
Fast, efficient, and flexible query performance is the Holy Grail for today's information consumers. There's no time to index or partition data or perform other tedious forms of manual configuration just to create and run new queries.
Speed the Data "Acquisition-to-Action" Cycle
In the coming years, organizational competitiveness will be increasingly defined by how quickly companies can synthesize the many sources of information coming their way. To do this, organizations must be able to master what I call the "acquisition-to-action" cycle. In other words, how fast can data be captured, stored, queried, analyzed, shared, and acted upon? Traditional BI, massive data warehouses, and databases that were originally designed to crank out pre-configured reports (e.g., sales histories, financials) are simply not nimble enough to handle today's urgent analytic needs, especially in the relentlessly expanding domain of machine-generated data. Pre-calculated, batch-based solutions (e.g., Hadoop, MapReduce, clusters) have a role here, but their utility is limited when it comes to ad hoc analysis (important for figuring out what to do now) and predictive analysis (essential for understanding what to do next).
Fast, efficient, and flexible query performance is the Holy Grail for today's information consumers. There's no time to index or partition data or perform other tedious forms of manual configuration just to create and run new queries. (Indexing and partitioning also increase database size—in some cases, by a factor of two or more.) This demands solutions that are optimized to deliver quick analysis of large volumes of data, with minimal administrative effort to set up, change, and customize analysis.
Columnar databases and other newer analytic approaches have emerged that enable significant data compression and accelerated query processing—no indexes or partitioning of data required. This is a particularly compelling capability in the big data era. In addition, there are a number of open source projects now focused on analytics, BI, data integration, and more, which gives IT practitioners the opportunity to test-drive more innovative analytic tools without risk. Open source tools can also offer the flexibility essential for optimizing rapidly changing query and reporting requirements. Finally, open source solutions are considerably more affordable than proprietary BI and database solutions.
This article originally appeared in the issue of .