Q&A: The Human Factors of BI
Top-level management support is vital for project success, but it won't guarantee it. Len Silverston explains why human factors in BI projects are just as -- if not more -- important.
- By James E. Powell
- September 17, 2013
[Editor's Note: Len Silverston is presenting an all-day course at the 2013 TDWI World Conference in Boston on Wednesday, October 23. There Is a Magic Bullet: Leveraging Personal Relationships for BI/DW Success covers many of the human dynamics topics, frameworks, and techniques for human dynamics discussed in this article. Silverston is the author of The Data Model Resource Book series and president of Universal Data Models, LLC.]
BI This Week: In your opinion, if you were to pick one factor that is the most important in data warehouse and business intelligence success, what would this be and why?
Len Silverston: Many people say the most important factor in BI/DW success is top-level management commitment, demonstrated by financial resources and investment in expertise as well as in state-of-the-art tools and technology. This seems like a reasonable answer. However, I have been on several engagements where we had these factors in place and failed terribly. For example, on one effort, we had top-level management commitment, hundreds of millions of dollars allocated to the data integration and data warehousing effort, top experts working on it, and after several years of effort of not delivering substantial value, they cancelled the program.
I have also been on data warehouse engagements where we did not have these factors in place. At one of the most successful data warehouse efforts, to which I provided consulting, there was skepticism at top levels of management, there was only $250,000 allocated to the program at first (for hardware, software, and consulting), and very basic tools available -- yet they succeeded in developing a very robust data warehousing program and were written up in several magazines!
The difference between these efforts was how people worked together. In my example of a failed project, there was great competition and infighting between groups. In the example of the successful project, people worked in an extremely collaborative, productive fashion. Both of these effort involved highly political environments. However, the way people worked together was vastly different.
In my opinion, the one factor that I think is most important is the way we work together, or in other words, the human dynamics within the organization and their BI/DW program.
What are some of the biggest pitfalls that you see in data warehouse and business intelligence efforts?
One of the greatest pitfalls is not investing in people and ways that we can more effectively work together on these efforts. For many years, I have been teaching the human dynamics involved in data warehousing and business intelligence. The great majority of people agree that the human factors are most important. However, managers often feel that they cannot fund or invest in training people in how to more effectively work together. They often say or think, "I can't get funding for this airy-fairy stuff!" However, I have seen programs that waste tens of millions of dollars and then fail because they have not invested on the human side of the equation.
For example, in my experience, I see five inevitable scenarios that occur over and over again on data warehousing/business intelligence efforts. If this is so, why not know what these are (see the TDWI article There is a Magic Bullet for BI and DW Success to learn about these scenarios) and have tools for dealing with them before they occur. My experience is that if we are not prepared to deal with them until they actually occur, we often hit a brick wall and it is often too late.
Another pitfall is lack of clarity. I have been on DW/BI programs that don't know how the overall picture works -- for example, how data governance, metadata, architecture, big data, master data management, and other aspects of information management fit together. There are often not clear plans showing how all these components work and what sponsors expect. Organizations also assume that the objectives, benefits, expectations, and details are understood, but they are also often unclear.
Lack of balancing short-term results and a long-term foundation is another pitfall. We need to deliver, yet we also need to have a solid foundation. I know of an organization that has 2000 data marts that provide completely inconsistent data. Each area of the business wanted their own data reporting solution, and although each department seemed to get what they wanted the organization, as a whole ended up with a mess. Most of the data marts were delivered relatively quickly, but they failed to see the consequences of an integrated architecture. On the other hand, if we build data warehouses using an impractical, ivory-tower approach that is too focused on the infrastructure without also considering the need to deliver, we are likely to fail as well.
Finally, a big pitfall is incorrectly estimating, setting expectations, and failing to deliver on promises. Data integration efforts such as data warehouse or BI efforts are often deceptive in how much effort is involved.
What are some solutions to handle these pitfalls?
Regarding investing in people, some enterprises set up an area within the enterprise information management (EIM) function focused on the people challenges because information integration often requires a change in how we see things. This may be called EIM change management (or another name). Of course, it is possible to set up a change management function across the entire enterprise as well and this can also work effectively. However, some of the challenges within information management and data warehousing are unique. The key is recognizing that it is important to understand and invest in people and the human dynamics involved. One can use case studies of successful organizations that have invested in this area as a justification. Another solution is to provide training to information management and data warehouse professionals on this important area of human dynamics.
Concerning clarity, it is important to start and continue with the end in mind. We must be laser focused and clear on what we and others want. This is very challenging. There are frameworks and techniques that we can use to get clear about people's motivations and expectations. At one client, we took the time to understand and document motivations using a specific approach we called a "sponsorship model." This led to great results, such as a $20+ million return on investment as well as winning an international industry achievement award.
Our approach must balance short-term results and a long-term foundation. We need to deliver incrementally while having a clear, compelling, and common (agreed upon) vision that all people involved understand.
The most successful efforts I have seen deliver what they promised and in the time frame that they promised. We must stay focused on what is important, create reasonable expectations, and then meet them. This sounds much simpler than it is. This involves deep listening, communications, expertise in estimating and setting reasonable expectations, and skill and commitment in working effectively on delivery.
What is the most important point that you want to stress in dealing with the human factors of business intelligence and data warehousing?
Invest in people and the human dynamics involved. The most successful companies I have seen have invested in this and gained huge returns. Invest in training about conflict management, how to develop trust, how to better understand motivations, how to get people on the same page, how to set reasonable expectations, how to create clarity, and how people can work together more effectively. Many say this is common sense, but skills in these areas are not common. Skills development in human dynamics is instrumental for success.