The Magic of Abstraction: Hierarchy Management and Decision-Making

How an abstracted analysis layer can shield systems, enhance insights, and more

Speaking in the abstract might sound lofty, but conceptual expression provides the grounding for all manner of creative thinking. Indeed, the world’s greatest inventions owe their glory to the power of abstraction: the ability to draw back one or more layers from the tactile world around us, and then assemble, piece by piece, a process or construct that results in a desired creation. The more complex the construct, the more prodigious the final product can become.

Even our system of mathematics, which easily dates back some 5,000 years to the storied land of Mesopotamia (modern-day Iraq), relies on the abstract: all numbers exist one layer of abstraction from reality. Furthermore, the center of any number line, zero, doesn’t really exist, per se. It’s arguably a second-level abstraction, but one that nonetheless binds positive numbers with negative along an infinite spectrum which, in the world of business, often boils down to the fundamental reality of profit or loss.

In the realm of information management, and especially in the focused world of business intelligence and data warehousing, abstractions abound. Data models, ETL mappings, business metrics—these are all abstractions; they don’t exist in the corporeal world. So is transactional data, for that matter; it amounts to details about the real-world products that companies sell and consumers buy. And yet, these abstractions provide the framework necessary for organizing, analyzing, and ultimately “operationalizing” information—integrating insights gleaned from analysis into day-to-day business operations.

Which brings us to the essence of analytical protocol. The process of analysis obviously consists of thinking, but while the human brain is capable of highly complex thought patterns, there is a limit to how far out a person can abstract, or how many concepts someone can juggle in their mind at once. For these reasons, smart people for millennia have created tools to facilitate analytical thought, such as the abacus or the whiteboard.

In modern-day business intelligence, another method for facilitating analysis has emerged, one that creates and harnesses a new layer of abstraction: hierarchies.

Anatomy of a Hierarchy

Almost every set of data at some point distills into a list. General ledgers, charts of accounts, cost centers, organizational charts, data warehouses—all such information can be represented in list format. The more valuable a list becomes to a business, the more likely it will blossom into a full-blown taxonomy. Hierarchy management is essentially the discipline of defining and manipulating such taxonomies for the purpose of improving analysis of business processes, and also enforcing data quality via referential integrity.

For a tangible example of a hierarchy, consider an organizational structure: Salesperson John Doe reports to his immediate manager, who reports to a regional director, who reports to a line-of-business vice president, who reports to a senior vice president, who reports to the chief executive officer, who reports to the board of directors. That’s a seven-level hierarchy. Sometimes such hierarchies will be ragged (a street-level salesperson may report directly to a regional director); other times they’ll be balanced, where all branches contain the same number of levels.

That salesperson will also figure into a financial hierarchy, and will likely be linked with a product hierarchy. The financial hierarchy would include information about revenue generated by that salesperson, while the product hierarchy would be cross-referenced to facilitate analysis about which products comprise John’s profitability, and which products don’t move as quickly under his stewardship.

A product hierarchy might take the following form: Several sizes of a particular soda would exist at the bottom level of a hierarchy, such as a can, a glass bottle, a liter, a two-liter, and other sizes. Each of those products could roll up in several ways: a can could even be marked as a unit sent to a soda machine as opposed to a unit sold individually in a store. Six-packs, twelve-packs, and cases could be on the next level up, or they could comprise a bottom-level node in and of themselves.

That particular type of soda could then roll up into a grouping of carbonated drinks, such as regular soda, diet soda, cherry-flavored soda, and lemon-lime soda. That grouping could roll up into the next level of the hierarchy, consisting of all beverages. The next level up might include both beverages and snack foods. One more layer up might be all products carried by every line of business within a parent company. For large packaged goods companies, there could be hundreds or even thousands of hierarchical structures.

As different hierarchies interact, the number of permutations rises dramatically. With so many possibilities, this hierarchical abstraction becomes increasingly valuable. The more business requirements a company has for information managed in different ways, the more important hierarchy management becomes, not only due to the number of possibilities, but also because of the need to perform complex, often ad hoc analysis without jeopardizing the information architecture itself.

The Value of Hierarchical Abstraction

By abstracting the analysis layer away from operational or even analytical systems such as online analytical procession (OLAP) applications, several key benefits result:

• First, the analytical and operational applications themselves are shielded from business users, which protects those systems from being altered or broken by users who are not technically savvy. This is a key benefit because information technology (IT) professionals need not worry about business users meddling with mission-critical code.
• Second, by allowing business users to create their own alternate views of the organization, their understanding of the business greatly increases. After all, the value of analysis hinges on the ability to ask questions and get answers in rapid-fire fashion. Most analysts know from experience that desired answers rarely result from the first question asked; rather, the most valuable answers result from the third or fourth questions posed in succession. When business users must engage IT to create custom code or effect a merge of hierarchical structures, the rhythm of analysis is broken. Further, time is spent by both the user and the IT technician, and all for one change that may not produce the desired result. When the analyst can make such changes quickly without jeopardizing the system itself, the benefit of analysis can blossom.
• Third, if a hierarchy management system is maintained dutifully, change management becomes much easier, especially if a historical record is kept of hierarchies as they evolve over time. When necessary, a user, department, or entire organization can roll back the hierarchical construct to a particular version in order to analyze what changed and why. This type of historical analysis can prove invaluable when trying to reconcile numbers for a quarterly close or when producing information for auditors.
• Fourth, by diligently managing attributes assigned to particular nodes, an effective master data management solution arises. Such a solution can be highly valuable for dealing with mergers and acquisitions, as well as multinational corporations that must reconcile product sales across multiple regions and countries.

Hierarchy management serves as a translation layer between values in a data warehouse or other business intelligence environment, and the reporting or analysis view that business users see. By abstracting this translation layer, individual business users can manipulate variables within that layer without overwriting data in source systems or requiring hard-coded changes to those systems. This abstraction opens up an entire realm of analysis that can help organizations maintain competitive advantage in an increasingly complex and demanding marketplace.