Q&A: BI Leaders Must Evolve to Survive
Former TDWI director Wayne Eckerson discusses industry trends that make it necessary for BI directors to reinvent themselves.
- By Linda L. Briggs
- June 2, 2015
Wayne Eckerson, founder and principal consultant with The Eckerson Group, served as director of education and research at TDWI for many years. He is an internationally recognized thought leader who has worked in the business intelligence and analytics field since the early 1990s. Eckerson is the author of 2012's The Secrets of Analytical Leaders: Insights from Information Insiders" and earlier Performance Dashboards: Measuring, Monitoring, and Managing Your Business. He is currently working on a book about data governance.
In this first of two interviews with Eckerson, he discusses the need for BI directors to reinvent themselves. "Bottom line, [the BI team] needs to make a case as to why they're still relevant," Eckerson says. "In my opinion, they are very relevant. They're still at the heart of business intelligence; they're still at the heart of what it takes to turn data into insight and action."
BI This Week: You've been working in this area of technology a long time. What trends are you seeing in data warehousing and BI?
Wayne Eckerson: First, the role of the BI director -- the person in charge of the BI and data warehousing environment -- is changing as the technology environment is changing. There's a convergence of things they have to deal with. One is that line-of-business self-service tools have gained ascendency. They can no longer dictate to the line of business or force them to wait to have their needs serviced. The line-of-business folks have more autonomy organizationally and more self-service technology, which enables them to meet their own needs. So on that score, the BI director has to reach out and partner with the line of business instead of holding them at bay.
Second, there are the analytics teams, which have all the sex appeal now. They are getting more of the funding from corporate, and there is a perception "why do we even need the BI or the data warehousing team anymore?" We have advanced analytics on one hand and self-service tools for the business on the other hand.
[To counter that perception], BI directors have to learn how to partner with the line of business, with the advanced analytics team, and even with people and groups in the IT department, like the project management office. ...
At the same time, data warehouse technology has evolved, and a lot of the data warehouses that were built 10 years ago are woefully out of date, just as the practices that were used to build them are superseded by new or more agile development techniques.
Bottom line, [the BI team] needs to make a case as to why they're still relevant. In my opinion, they are very relevant. They still are at the heart of business intelligence; they are still at the heart of what it takes to turn data into insight and action.
People are saying that BI should concede application development, and leave that to the business side. That's true to an extent, but on the other hand, the business units are often building a lot of garbage. They take self-service tools and build applications that don't scale, that aren't secure, that don't deliver accurate data a lot of the time, and that are very vulnerable if someone [in the business unit] leaves. Of course they're building them using all kinds of different tools, so oftentimes when the thing blows up because it's not scalable, or secure, or performing well, it falls in [the BI team's] lap to fix it.
I think the pendulum will swing back one day soon; companies will start to emphasize governance over speed, standards and consistency over agility, individual preference, and business unit autonomy.
As I said, I do believe BI is still very much part of the picture. It's interesting to listen to Gartner talk about this thing they call "bi-modal BI." I've been saying that same thing for the past five years. I call it "top-down, bottom-up BI." Whatever you call it, it's absolutely true that there are two worlds out there and they need to coexist.
We shouldn't believe that one world can co-opt or usurp the other. Currently, the business thinks that self-service is going to co-opt everything in the data warehouse, and big data is part of the whole picture. Big data is partly an IT play and partly a data warehousing play. It's something of a Swiss Army knife -- you can use it to do anything. Business units are latching on to it, however, as a low-cost way to bring in all this non-standard data and do something with it from an analytics perspective.
We can have something faster and cheaper, but ultimately, if we're all doing our own thing with our own data in our own way, nothing adds up, and eventually the CEO says, this doesn't work. I can't get answers to simple questions such as how many customers we have, or what did we do in sales yesterday ... so eventually the pendulum will swing back, as I said, and BI will become more relevant.
In the meantime, [BI leaders] really need to sell their value to the company, they need to learn how to partner, they need to learn how to create an ecosystem and not an architecture -- an ecosystem of different parts and different architectures that are designed to support the same end, which is turning data into insight and action.
It sounds like BI directors really need new strategies to survive.
As I look at this playing field -- from a leadership position and what it's going to take to survive, I think yes, BI directors need new strategies in order to not just survive but to drive in this new world order. A lot of this gets wrapped up in two things on the architecture side. First is how to build a big data ecosystem, not just BI architecture, which is only part of it. Second is on the organizational side -- how to build a BI center of excellence -- not just a BI team or a data warehousing team. That center of excellence is really an organizational web that embodies those partnerships I talked about. It supports the big data ecosystem for the purpose of putting data into insights and action.
That's what I'm focused on these days, and it seems to resonate pretty well with the folks I'm meeting with.
In terms of the BI director partnering with the line of business -- which goes along with the top-down, bottom-up BI you've been talking about for some time now -- how well are organizations doing that?
It's a good question. If they aren't partnering by now, honestly, they're out of work. Early on, with the advent of self-service BI tools, there was a push by many in BI to resist these things. They were popping up uncontrolled -- business users were building data marts or spreadmarts that in many cases undermined the data consistency of the warehouse and created dueling spreadsheet problems, just aggravating things. Ultimately, as I like to say, what you resist, persists. Fighting these spreadmarts was a self-defeating proposition -- a pyric victory at best.
What most people now realize -- at least I hope -- is that they should be enabling self-service BI and bringing in these self-service tools instead of having them pop up unbeknownst. They need to get ahead of the wave instead of behind it. They need to set these environments up and point them to the warehouse -- to the source of data that they've built, at least for the data that exists. ... Don't reinvent the wheel, don't re-create metrics that already exist. Help these business units grow faster but leverage what's already there, so that there is some semblance of consistency.
Where does the BI center of excellence that you mentioned come in?
The harder thing, which very few people have done, is to create a real BI center of excellence, one that really weaves together business units and the corporate BI -- and advanced analytics teams -- into a coherent whole. This requires some degree of finesse, it requires matrix reporting relationships, it requires a federated organization -- which, actually, has precedent in most companies because a lot of project management offices and app dev teams are federated. We just have to learn how to do that in BI so that BI experts in the field have shared responsibility both to business units and to corporate.
We want them to be acting locally but thinking globally. In other words, they're building locally with the tools that have been given to them from corporate, using standards and processes that inherit best practices gleaned from across the organization and standardized by the corporate team. However, they can be building fast because they don't need to go through this [prolonged] requirements-building process, and ultimately build something that users don't want.
You just can't build from the outside. Corporate can't build -- corporate's role is more of support. They are the first line of support for BI and data folks in the field. They have the experts, the specialists. They can get all the groups together to define best practices and standards, to collectively build a road map for the data warehouse and the BI environment, to provide ample training, and so on.
It's a real two-way street, and part of it is simply identifying who does BI already. That extended BI-data warehousing team already exists, it's just hidden inside the business unit. ... The hard part is finding who they are and restructuring their relationship a bit so that they have some corporate skin in the game and can be both efficient and effective in business units, without undermining corporate consistency.
It's a big, big challenge. People in these areas get it -- they understand. Some even say to me, "Yeah, we've done that all that, and it really works," but it's very hard to do. It's also very hard to do continuously -- but if you don't, you're just swinging from one end of the spectrum to the other, always dealing with major problems.