The Premier Website for Data Warehousing and Business Intelligence
Members Only

Wayne's World

By Wayne W. Eckerson

Blog archive

Humpty Dumpty and the CEO

There is a dirty, little secret about data warehouses: we wouldn’t need them if top executives ran their organizations properly.

A data warehouse (DW) reflects the organization—the more fractured and disintegrated the organization, the harder it is to create a robust, highly functional data warehouse. These reporting repositories really are tools to reintegrate a fractured enterprise and provide a holistic and consistent view of data where none exists.

Most organizations are like Humpty Dumpty teetering and tottering on top of a big wall. With the slightest gust of wind, Humpty crashes and breaks into dozens of pieces. And DW teams are “all the king’s horses and all the king’s men” who are charged with putting Humpty Dumpty back together again.

Today, most companies have fragmented into dozens or hundreds of largely unconnected business units, departments, and workgroups, each with their own strategies, policies, processes, systems, IT staffs, and data. It’s the job of the CEO to bring order to this chaos. If the CEO provides the business with a clear, coherent strategy, integrated processes and systems, and standard definitions and rules governing those processes, then there is almost no need for a data warehouse.

The good news is that many DW teams do succeed in gluing their companies back together, at least for awhile, until the next merger, acquisition, or other upheaval blasts everything apart. But while it lasts, these unsung heroes should be congratulated, not outsourced, offshored, or reassigned to the IT-equivalent of a Siberian gulag.

The moral of the story is this: don’t blame the DW team if it can’t deliver a successful data warehouse; blame the CEO. The data warehouse is merely a messenger, a reflection of the state of organizational dysfunction.

Editor's note: this article originally appeared in the Washington Post.

Posted by Wayne W. Eckerson on 09/23/09 at 9:52 PM


Comments

Thu, Oct 15, 2009 Wayne Eckerson

Art and Arthur, Thanks for your comments. Arthur, you are right to suggest that the BI team needs to adapt to changes Phoenix-like rather than pick up the pieces every time the organization changes. I guess the Humpty Dumpty metaphor is not the best--albeit a lot of fun--unless the king's men are using superglue to get Humpty back together again. Certainly, I'm a big advocate of adaptable DW and BI architectures. That's a given I should have noted! Wayne

Mon, Sep 28, 2009 For: Arthur Ritche Montreal

Thanks for starting this conversation again Wayne. It's an important one and I'd like to share some thoughts. "Most organizations are like Humpty Dumpty teetering and tottering on top of a big wall. With the slightest gust of wind, Humpty crashes and breaks into dozens of pieces. And DW teams are “all the king’s horses and all the king’s men” who are charged with putting Humpty Dumpty back together again." Whether we’re talking about “Humpty Dumpty” in terms of the enterprise as a whole, or the data within a given warehouse, agreed -- DW teams are doing the best they can with what they have. But often so are the CEOs, who are facing battles in the boardroom, battles between the shareholders, the company’s bankers, the boards of directors, the various C-Levels within the organizations and some of their powerful subordinates. Never mind vacuums created when key executives leave, or when mergers and acquisitions, divestitures, change the nature of the business. Unfortunately, most current Data Warehouses are built in such a way that this Humpty Dumpty dilemma will repeat itself over and over again. The real dirty little secret is that the same tricks used to make DWs efficient for reporting purposes (aggregation, indexing, and the subsequent discarding of underlying details) are the ones that make them difficult — and expensive — to change and update. So what’s the answer? First, we must realize that there is no such thing as a “single version of the truth” but merely a convenient and workable one. Next, we must break out of the “Humpty Dumpty” paradox and its tragic ending and find a better story -- a better model, like the “Phoenix” that can “rise from the ashes” overnight to meet all the new KPI’s to support the business needs. The good news is that technological developments in Nearline 2.0 storage, RDBMS federation capabilities and high-performance ETL allow companies to transition from “Humpty Dumpty” to the new “Phoenix”-like approach -- without resorting to a “rip and replace” strategy. More to follow on my next post...

Mon, Sep 28, 2009 Arthur Ritchie Montreal

Thanks for starting this conversation up again Wayne. It's an important one, and I'd like to share some thoughts. Whether we’re talking about “Humpty Dumpty” in terms of the enterprise as a whole, or the data within a given warehouse, agreed — DW teams are doing the best they can with what they have. But often so are the CEOs, who are facing battles in the boardroom, battles between the shareholders, the company’s bankers, the boards of directors, the various C-Levels within the organizations and some of their powerful subordinates. Never mind vacuums created when key executives leave, or when mergers and acquisitions, divestitures, etc. change the nature of the business. Unfortunately, most current Data Warehouses are built in such a way that this Humpty Dumpty dilemma will repeat itself over and over again. The real dirty little secret is that the same tricks used to make DWs efficient for reporting purposes (aggregation, indexing, and the subsequent discarding of underlying details) are the ones that make them difficult — and expensive — to change and update. So what’s the answer? First, we must realize that there is no such thing as a “single version of the truth” but merely a convenient and workable one. Next, we must break out of the “Humpty Dumpty” dilemma and its tragic ending and find a better story — a better model, like the “Phoenix” that can “rise from the ashes” overnight to meet all the new KPI’s to support the business needs. The good news is that technological developments in Nearline 2.0, RDBMS federation capabilities and high-performance ETL tools offer a way for companies to transition from “Humpty Dumpty” to the new “Phoenix”-like approach — without resorting to a “rip and replace” strategy. Arthur

Thu, Sep 24, 2009 Art Conroy Washington DC

Wayne, I was pleasantly surprised to find the Post in the driveway this morning and read the BI insert. You hit the nail on the head with humpty dumpty. It is even worse in the government sector. I spent many years building data warehouses and was fortunate enough to work for Dr Hammer at ETI. (http://www.fastcompany.com/articles/archive/hammer_bio.html) After several years, I realized the reason I was so successful at designing warehouses was that I looked first to understand the dysfunction of the organization , then and only then can you know what model (schemas etc) to use in building the edw…to your point. In every case, the root cause went to the leadership of the organization. Thanks for addressing the dike. The holes are in the mind, not the wall. Art C

Add your Comment

Your Name:(optional)
Your Email:(optional)
Your Location:(optional)
Comment:
Please type the letters/numbers you see above

Share

Connect with TDWI

Follow TDWI on Twitter Follow TDWI on Facebook Follow TDWI on LinkedIn Follow TDWI on RSS

Get up-to-the-minute BI/DW news, articles, and special discounts

Wayne's Speaking Schedule