Q&A: Cutting through the Glitz of BI's Shiny Objects
What's hype and what's real? What BI shiny objects have attracted our attention -- and perhaps distracted us? Marc Demarest and Mark Madsen explain what's hot and what's not.
- By James E. Powell
- November 19, 2013
Marc Demarest, CEO and principal at Noumenal, Inc. and Mark Madsen, president of Third Nature, Inc. take an irreverent look at, in their words, "the buzzwords, and big ideas that fill our lives and our inboxes and keep us up at night." Decision management, memory-resident DBMS, cloud BI, and predictive analytics are among the subjects that may be in play during their typical freewheeling and irreverent talk.
We asked them to give us a preview of their upcoming keynote address at the TDWI World Conference in Orlando, Florida (December 8-13, 2013).
TDWI: Gentlemen, what's the top shiny object on your list that's causing the most distraction for BI and data warehousing professionals?
Demarest: A year ago, I would have said -- I probably did say, a hundred times or so -- Hadoop, but I think the shine on Hadoop is significantly dulled in the past year. The same thing with NoSQL databases, but I feel certain we'll have a thing or two to say about that topic anyway, because the trajectory of the NoSQL market -- from "replacement" to "augmentation" to "alternative, for some use cases" -- is very much the trajectory of shiny objects, generally. In any case, my candidate for shiny-object-of-the-year is: privacy, but I think we should save that topic for the show.
Madsen: I'd nominate the Internet of Things [IoT], and machine-to-machine communications generally, as they pertain to BI and analytics, as well as transactional applications, automation, and infrastructure.
Vendors are showing us pretty YouTube clips of Dick Tracy watches and self-navigating vehicles, but no one's talking, as far as I can tell, about the hard problems of devices communicating in an uneven and spotty transport fabric, discovering services, or anything of that nature. Any discussion of basic service-neutral, non-centralized protocols would be nice, or the protocols for store-and-forward, on-demand data requests and processing. Neither of us is aware of any coordinated work in this area, and we sense an impending train wreck in an area that -- unlike so many shiny object areas -- actually has huge economic impact, and significant potential for competitive differentiation, for early adopters.
Trouble is, many companies fired their architects a few years ago because Oracle (or SAP or Cisco) was going to tell them what they needed to know in all areas pertaining to infrastructure and design, so who needed architects? Oops. The Feds are driving infrastructure work in some areas -- such as smart transportation -- but after this NSA debacle, I don't think anyone can look at the Feds as a credible source of architectural guidance or technology specification -- not for a long time to come.
Demarest: And the lawyers. Where are they? Imagine the details of the Dick Tracy Universe. The whole question of rights-in-data in that IoT world is a huge festering sore. Is the telemetry coming off my refrigerator and washing machine my data or Frigidaire's data, or do we both have rights in that data? And what are they, exactly?
We know that millions of people will click "OK" when presented with terms of service that say, in effect: we own your data and can do with it what we like without notifying you. That's what Web 2.0 is, at some level: a wholesale transfer of data rights. It's one thing when the data in question is a thoughtless tweet about something stupid you did last night. It's another thing when the data in question is the entire dwell graph of your brand-new OnStar-enabled car.
What's the fascination with the Internet of Things all about? What problems are we hoping it will solve, and why will we be disappointed the closer we look?
Madsen: Wearable computing. Instant communications with all of the people and assets that make up your life. Instrumentation of your body, your activities, your children, and pets. You can know anything you want: what's in the refrigerator at your beach house, whether what you're looking at right now is the building where you're supposed to have that meeting in six minutes. Where's my child, and what is she doing? Surveillance, when it's us, surveilling the things we own and the people we love, is a good thing. In that respect, the future's so bright, we'll all have to wear tinted Google Glasses. And it's all technically feasible, provided the infrastructure on which all of these marvels depend is properly designed, implemented, and managed.
The commercial side is more interesting: look up Bruce Sterling's "spimes" for early thoughts and cans of worms. The idea of things communicating to provide services without human involvement, or to monitor operations, isn't all that new, but the idea of intelligence in the network and dynamic, adaptive systems is.
Demarest: Verizon just announced a "converged health management" platform which is gobbledygook shiny-object marketese for: remote patient vital signs monitoring. Anything "converged" is shiny, these days. It's a good idea -- being able to send bodily telemetry to your healthcare providers, particularly if you have serious or rare health conditions. This Verizon system is an Internet-of-Things usage model in miniature. Blood pressure cuffs, pulse oximeters, glucose meters and scales -- all sensors, capturing and transmitting data wirelessly to a cloud-based application. Cloud. Shiny. Ooooh.
It's also Verizon wireless broadband, Verizon cloud services, Verizon defined interfaces. Hospitals are lukewarm to the idea as they want to control the collection and presentation of the data for reasons that have nothing to do with my wellness. Insurance companies want the data, I'm sure, ultimately so they can figure out how to be more profitable than they are already. Healthcare providers want the data. Medical instrument companies want the data. The Feds want the data.
The patient -- whose information it is, after all -- is lost in the squabbling over access and control. Mark's right: the infrastructure is largely ignored, unplanned -- or designed for integration-on-my-terms-only, which history tells us doesn't work, ever. Furthermore, in practice, the beneficiary is also largely ignored. The pretty pictures of Dick Tracy watches and health hobbyists monitoring their morning run is a cover story for control strategies: capture the data, own the data, control the consumer, or capture the data, own the data, sell the data, directly or indirectly. Your vital signs, in a Google pay-for-placement advertisement. Mark Madsen uses Acme glucose management technology; shouldn't you?
Is big data a shiny object? Should it be? After all, hasn't big data been a concern of IT for years? It seems to me that IT has been worrying about how to store more data, how to conduct back-up in ever-shrinking processing windows, and worrying about applications being powerful enough to run in real time (and environments scalable enough to handle the workload)?
Demarest: Big data isn't shiny anymore. Fast data is shiny now. Get with the program. I think you're right when you list those real problems, none of which is characteristic of shiny objects. Shiny objects -- if you read the marketing literature and the trade press -- have no problems, no limitations, no sharp edges. They might, they may, they can change everything.
Big data is clearly a big problem, operationally, now, and a big problem commercially. The promise of big data seems to be, still, just that: a promise. Invest now -- in what is essentially a wholly new and foreign technology stack, as well as in a new class of employee or two -- and together the new tech and the new brains will discover a "treasure chest" -- I see that phrase used a lot -- that will catapult your organization into a market leadership position. The pony is in there somewhere; you just have to find it.
Needle. Haystack. Mining. Pick your metaphor. The poster children for big data are usually companies far more well-funded and well-resourced than the companies Mark and I typically work with. They're companies that can afford to go on pony hunts, that can buy the technology and people they need to make the pony hunts work, technically. They're companies that can afford to run experiments and fail. I envy them. That's not the world I work in.
Does "in-memory databases" make your list? They claim to solve a number of performance problems, especially when it comes to processing big data. What is it BI professionals are missing with this new technology?
Madsen: You can always throw hardware at problems and get some kind of acceleration from that act. We've done this for 30 years. That's not a problem-solving strategy; that's a problem-shifting strategy. Storage too slow to keep up with the database management system's demands? Try in-memory. Application suite so gargantuan and complex that simple queries take hours to return? Try in-memory. Manage large in-memory complexes? We're working on that; maybe next year.
For sure, there are valid applications for in-memory data management technologies, and many of them are analytical, and for sure, in-memory is a shiny object at present thanks to some of the market analysts and their Top 10 lists. We'll talk about this more during our keynote, but I have a teaser for you: are we really going to build, again, what Tandem and Stratus and Sequoia and others built in the 1980s?
Cloud BI -- we can't seem to get enough information about applications running in -- or data stored in -- the cloud. When you factor in the lack of (or lower) CapEx or OpEx up-front expenses, and unlimited scalability, what's not to like?
Demarest: You're bull-baiting. I think cloud BI makes sense for companies that have their whole OLTP portfolio in the cloud -- if they can get those cloud-based OLTP apps to cough up their data. I think cloud BI makes sense for companies with no in-house BI expertise and a history of expensive failed projects. The cost of failure in cloud BI is relatively low.
I think cloud BI makes sense when the constituency being served by the BI application is one that wants a few simple metrics displayed in pretty ways on mobile devices, but I continue to believe that cloud BI makes no sense when the core information assets targeted for analysis are inside the company firewall and are considered assets: of use to the company, and of great value to competitors, if they had it.
One of my clients is a company that specializes in data breach response, so I have a view into the large, and largely under-reported, number of data breaches involving personally identifiable information, financial data, and healthcare data that occur each year. The problem is bigger and more expensive than people understand. The companies being breached are well-run companies -- in some notorious cases, in the information security industry. They're following industry best practices in their IT operations, employing the latest in intrusion detection and prevention technologies money can buy, doing all the right things. Yet, they breach. I wonder what will happen when we have the first breach in which the historical financial information of dozens or hundreds of companies is let loose into the wild? Madsen said it once or twice, during our Big Data Wonderland days: cloud security is an oxymoron.
Is there a common theme to all the objects that you'll be talking about? What features or promises do they share? Why are we so often disappointed by these objects?
Madsen: Caveat lector. Reader, beware. Everything on the technology landscape today has to be deconstructed. Nothing you read says what you think it says, and what isn't said is usually more important than what is said. Hyperscale marketeering is in effect everywhere.
Demarest: And, caveat cohort. Beware of your colleagues, who may not be following the caveat lector rule. What I see, again and again -- and certainly this is the case with Hadoop still -- is that uncritical internal champions of shiny object technologies often steamroll more thoughtful folks, answering their "Why?" questions with: "It's a revolution! A game-changer!" Good rhetorical strategy -- who wants to miss the revolution? -- but it doesn't actually answer the question: what's in this for us?
How can we recognize shiny objects ourselves, and what's the best way to protect ourselves from succumbing to their spell?
Demarest: We're butchering Latin, so let's keep going. Quid pro quo. What for whom? Once you realize that everything's shiny, that there's hyperinflation in technology marketing, and that nearly every technology on offer is "revolutionary," "game-changing," and -- worst of all -- "disruptive," then the key question becomes: what's in it for us, in our particular situation, with our particular set of needs and objectives?