How to Avoid the Allure of Shiny BI Objects
Education and critical analysis, according to TDWI World Conference keynote speakers Marc Demarest and Mark Madsen, are the best tools with which to keep vendors honest -- and to combat hypermarketeering.
- By Stephen Swoyer
- December 10, 2013
This year's TDWI World Conference in Orlando opened with a celebration of “shiny objects.”
To think of shininess or of things shiny at this time of year is to conjure up visions of “pretty paper, pretty ribbons of blue,” as the late, great Roy Orbison once put it. This isn't what TDWI keynote co-presenters Marc Demarest and Mark Madsen had in mind, however.
Demarest, a principal with information management consultancy Noumenal Inc., explained that marketing has always had a loose correspondence with reality. The danger, Demarest argued, is that in any context where things are changing rapidly -- e.g., where architectural prescriptions are being revisited and revamped, best practices questioned, and rulebooks ostensibly rewritten -- hype becomes increasingly disconnected from reality. There's little, in fact, to keep vendors honest. Buyers are especially susceptible to the lure of “hypermarketeering” -- or, as Demarest put it, “shiny objects.”
Madsen, a research analyst with IT strategy consultancy Third Nature Inc., used a different analogy: that of signal-to-noise ratio. In the current clime, he argued, the ratio of genuine signal to spurious noise is at an all-time low. This is particularly true of the business intelligence (BI) and data warehousing (DW) space. Upstart -- or “disruptive” -- entrants would have customers believe that the old rules no longer apply; that rulebooks aren't just being rewritten but thrown out entirely; that “new” technologies -- such as Hadoop -- are rip-and-replace successors to “legacy” tools such as the DW. (Hadoop itself is 10 years old -- a point that Madsen has made in other contexts.)
The reality is supplementation or augmentation, not culmination and/or succession, Madsen argued.
“We're building systems not with one technology but with a whole host of them,” he said. “A data warehouse is a bunch of moving parts and they're layered on top of each other. What we have is a pile of objects, some shiny and some not. The 'not' stuff, like relational databases, is being superseded by [shiny] things like Hadoop, and what we've forgotten is that you have to design and architect systems. Design and architecture means fit, form, and function.”
Most discussion about Hadoop and big data focuses on the technology, not the way it's architected, implemented, and used, Madsen argued. However, what's most interesting and helpful about Hadoop is the way it fits into -- and, to a significant degree, requires the reconfiguration or reimagining of -- an information architecture. “Hadoop is a great example of a labeled 'disruptive' technology that is really in some ways just a very small improvement ... married to something new,” Madsen argued.
“What's really intriguing about Hadoop is ... the arrangement of the components. The disruption in the data warehouse market has nothing to do with specific technologies, it has to do with taking the parts that we have … and rearranging them in a different way so that they create new, emergent behaviors. That is the whole gist of being an architect.”
Don't Believe the (Manufactured) Hype
Madsen and Demarest are by no means data management (DM) bigots. Both regularly present at venues such as O'Reilly Inc.'s Strata, as well as at the combined Strata + Hadoop World. They get big data and Hadoop. They aren’t reactive agents; they’re not trying to circle the proverbial wagons and protect the DM status quo. In fact, they described big data as just one of several shiny objects.
Demarest and Madsen also discussed -- and demolished -- cloud (which Madsen described as both a “noun and an adjective!”), decision management, in-memory, “advanced” data visualization, and manufactured concern over privacy and security. “Manufacturing” is a key component of hypermarketeering, according to Demarest and Madsen: vendors (or entire industries) try to manufacture problems for which they can serve up ready-made solutions.
Demarest, for example, assessed both the promise and the inescapable limitations of enterprise decision management (EDM). “There is no doubt in the minds of people who look at decision-making in the enterprise ... that there are many decisions made by people that could be made more cost-effectively by technology, [i.e., which] do not require judgment,” he pointed out.
“However, there's a fundamental problem with decisions that do require judgments -- and it is this: they cannot be effectively automated. Judgment is not algorithmic; judgment is heuristic.”
As Madsen put it, “There's a bunch of parameters around [a decision-making process] that at the end of the day, a human makes [the judgment]. But [these parameters] are repeatable. These are the things that kind of work in a decision management context.”
These tend to be “minimum value decisions,” however, he argued. “Outside of those fundamental, repeatable situations for which you can analyze the parameters,” Madsen said, “EDM is much less useful, if not completely unrealistic. It's great for 'no-anomalous-state' decisions [i.e., decisions that don't involve or trigger exceptions], but these generally don't correlate with value.”
With a full slate of educational tracks -- including a full-day, first-ever “Data Warehouse Automation” course -- TDWI Orlando could credibly bill itself as a hype-busting corrective. As Madsen pointed out, education, interrogation, and critical analysis are the best tools with which to keep vendors honest -- and to combat hypermarketeering.
“All of this marketing exists for one of two reasons. It's either generating noise to hide something by labeling it with something. Because if you stick this label on [something], then obviously it's a mobile BI tool. What they're doing [in this case] is they're hiding the fact that it's the same old shlock,” he suggested.
“The noise itself tells you something. It tells you where to look and where to start sniffing -- [e.g.] when you see these [shiny] new adjectives being applied to stuff.
Vendors, he concluded, are trying “to defer your questions. The way that you get around [this] is that you ask the obvious questions.”
About the Author
Stephen Swoyer is a technology writer with 20 years of experience. His writing has focused on business intelligence, data warehousing, and analytics for almost 15 years. Swoyer has an abiding interest in tech, but he’s particularly intrigued by the thorny people and process problems technology vendors never, ever want to talk about. You can contact him at
[email protected].