Minimizing Risk with the Best Practices for Data Management
By Philip Russom, TDWI Research Director for Data Management
I recently broadcast a really interesting Webinar with Rob Myers – a technical delivery manager at Informatica – talking about the many critical success factors in projects that migrate or consolidation applications and data. Long story short, we concluded that the many risks and problems associated with migrations and consolidations can be minimized or avoided by following best practices in data management and other IT disciplines. Please allow me to share some of the points Rob and I discussed:
There are many business and technology reasons for migrating and consolidating applications and data.
- Mergers and Acquisitions (M&As) – Two firms involved in a M&A don’t just merge companies; they also merge applications and data, since these are required for operating the modern business in a unified manner. For example, cross-selling between the customer bases of the two firms is a common business goal in a merger, and this is best done with merged and consolidated customer data.
- Reorganizations (reorgs) – Some reorgs restructure departments and business units, which in turn can require the restructuring of applications and data.
- Redundant Applications – For example, many firms have multiple applications for customer relationship management (CRM) and sales force automation (SFA), as the result of M&As or departmental IT budgets. These are common targets for migration and consolidation, because they work against valuable business goals, such as the single view of the customer and multi-channel customer marketing. In these cases, it’s best to migrate required data, archive the rest of the data, and retire legacy or redundant applications.
- Technology Modernization – These range from upgrades of packaged applications and database management systems to replacing old platforms with new ones.
- All the above, repeatedly – In other words, data or app migrations and consolidations are not one-off projects. New projects pop up regularly, so users are better off in the long run, if they staff, tool, and develop these projects with the future in mind.
Migration and consolidation projects affect more than applications and data:
- Business Processes – The purpose of enterprise software is to automate business processes, to give the organization greater efficiency, speed, accuracy, customer service, and so on. Hence, migrating software is tantamount to migrating business processes, and a successful project executes without disrupting business processes.
- Users of applications and data – These vary from individual people to whole departments and sometimes beyond the enterprise to customers and partners. A successful project defines steps for switching over users without disrupting their work.
Application or data migrations and consolidations are inherently risky. This is due to their large size and complexity, numerous processes and people affected, cost of the technology, and (even greater) the cost of failing to serve the business on time and on budget. If you succeed, you’re a hero or heroine. If you fail, the ramifications are dire for you personally and the organization you work for.
Succeed with app/data migrations and consolidations. Success comes from combining the best practices of data management, solution development, and project management. Here are some of the critical success factors Rob and I discussed in the Webinar:
- Go into the project with your eyes wide open – Realize there’s no simple “forklift” of data, logic, and users from one system to the next, because application logic and data structures often need substantial improvements to be fit for a new purpose on a new platform. Communicate the inherent complexities and risks, in a factual and positive manner, without sounding like a “naysayer.”
- Create a multi-phased plan for the project – Avoid a risky “big bang” approach by breaking the project into manageable steps. Pre-plan by exploring and profiling data extensively. Follow a develop-test-deploy methodology. Coordinate with multi-phased plans from outside your data team, including those for applications, process, and people migration. Expect that old and new platforms must run concurrently for awhile, as data, processes, and users are migrated in orderly groups.
- Use vendor tools – Programming (or hand coding) is inherently non-productive as the primary development method for either applications or data management solutions. Furthermore, vendor tools enable functions that are key to migrations, such as data profiling, develop-test-deploy methods, full-featured interfaces to all sources and targets, collaboration for multi-functional teams, repeatability across multiple projects, and so on.
- Template-ize your project and staff for repeatability – In many organizations, migrations and consolidations recur regularly. Put extra work into projects, so their components are easily reused, thereby assuring consistent data standards, better governance, and productivity boosts over time.
- Staff each migration or consolidation project with diverse people – Be sure that multiple IT disciplines are represented, especially those for apps, data, and hardware. You also need line-of-business staff to coordinate processes and people. Consider staff augmentation via consultants and system integrators.
- Build a data management competency center or similar team structure – From one center, you can staff data migrations and consolidations, as well as related work for data warehousing, integration, quality, database administration, and so on.
If you’d like to hear more of my discussion with Informatica’s Rob Myers, please
replay the Webinar from the Informatica archive.
Posted by Philip Russom, Ph.D. on March 11, 20150 comments
As my flight west from Orlando began its descent into San Francisco, I thought about how touching ground was a good metaphor for the just-completed TDWI World Conference. The theme of the conference was “Emerging Technologies 2014,” but one of my strongest impressions from the keynotes and sessions was the deflation of the hype surrounding those emerging technologies. Speakers addressed what’s new and exciting in business intelligence, big data, analytics, the “Internet of things,” data warehousing, and enterprise data management. However, they were careful to point out potential weaknesses in claims made by proponents of the new technologies and where spending on the new stuff just because it’s new could be an expensive mistake.
Setting the tone on Monday morning in their “Shiny Objects Show” keynote presentation, Marc Demarest and Mark Madsen debated pros and cons of new technologies, including cloud (the pursuit of “instant gratification”), in-memory computing, visualization, and Hadoop. Overall, they advised attendees to be wary of hype. “Strike out every adjective on the marketing collateral piece and see what’s left,” Demarest advised. The speakers were able to drill down to what are truly significant emerging trends, helping attendees focus on those instead of being distracted by the noise.
Evan Levy’s “Tipping the Sacred Cows of Data Warehousing” session was similarly educational. While deflating hype about various emerging technologies, Levy at the same time advised his audience to always question the value proposition of existing systems and practices to see if there might be a better way. He took particular aim at operational data stores (ODSs), noting that database and data integration technologies have matured to the point where maintaining an ODS is unnecessary.
I caught part of Cindi Howson’s session, “Cool BI: The Latest Innovations.” With guest appearances by some leading vendors to demo aspects of their products, the session covered promises and challenges inherent in several key emerging BI trends, including mobile BI, cloud BI, and visual data discovery. Cindi has just published the second edition of her book, Successful Business Intelligence, which offers a combination of interesting case studies and best practices advice to help organizations get BI projects off on the right foot and keep them going strong.
The Thursday keynote by Krish Krishnan and Fern Halper introduced TDWI’s Big Data Maturity Model Assessment Tool. Krish and Fern have been working on this project throughout 2013. It is a tool designed to help organizations assess their level of maturity across five dimensions important to realizing value from big data analytics: organization, infrastructure, data management, analytics, and governance. It is the first assessment tool of its kind. Taking such an assessment can help organizations look past the industry hype to gain a “grounded” view of where they are and what areas they need to address with better technologies and methods. Check it out!
Grounded: that’s where my plane is now, at SFO. Time to head home.
Posted by David Stodder on December 13, 20130 comments
By Philip Russom, TDWI Research Director
[
NOTE -- My new TDWI report “Integrating Hadoop into Business Intelligence (BI) and Data Warehousing (DW)” (Hadoop4BIDW) is finished and will be published in early April. I will broadcast the report’s Webinar on April 9, 2013. In the meantime, I’ll leak a few of the report’s findings in this blog series. Search Twitter for #Hadoop4BIDW, #Hadoop, and #TDWI to find other leaks. Enjoy!]
One way to get a sense of what kinds of technical specialists are working with HDFS and other Hadoop tools is to look at their job titles. So, this report’s survey asked a subset of respondents to enter the job titles of Hadoop workers. (See Figure 12 above.) Many users are concerned about acquiring the right people with the right skills for Hadoop, and this list of job titles can assist in that area.
Hadoop workers are typically architects, developers, data scientists, and analysts:
Architect. It’s interesting that the word architect appeared in more job titles than any other word, followed closely by the word developer. Among these, two titles stand out – data architect and application architect – plus miscellaneous titles like system architect and IT architect. Most architects (regardless of type) guide designs, set standards, and manage developers. So architects are most likely providing a management and/or governance function for Hadoop, since Hadoop has an impact on data, application, and system architectures.
Developer. Similar to the word architect, many job titles contained the word developer. Again, there’s a distinction between application developers and data (or BI) developers. Application developers may be there to satisfy Hadoop’s need for hand-coded solutions, regardless of the type of solution. And, as noted, some application groups have their own Hadoop cluster. The data and BI developers obviously bring their analytic expertise to Hadoop-based solutions.
Data Scientist. This job title has slowly gained popularity in recent years, and seems to be replacing the older position of business analyst. Another way to look at it is that some business analysts are proactively evolving into data scientists, because that’s what their organizations need from them. When done right, the data scientist’s job involves many skills, and most of those are quite challenging. For example, like a business analyst, the data scientist is also a hybrid worker who needs knowledge of both business and data (that is, data’s meaning, as well as its management). But the data scientist must be more technical than the average business analyst, doing far more hands-on work writing code, designing analytic models, creating ETL logic, modeling databases, writing very complex SQL, and so on. Note that these skills are typically required for high-quality big data analytics in a Hadoop environment, and the position of the data scientist originated for precisely that. Even so, TDWI sees the number of data scientists increasing across a wide range of organizations and industries, because they’re needed as analytic usage gets deeper and more sophisticated and as data sources and types diversify.
Analyst. Business analyst and data analyst job titles barely registered in the survey. Perhaps that’s because most business analysts rely heavily on SQL, relational databases, and other technologies for structured data, which are currently not well represented in Hadoop functionally. As noted, some analysts are becoming data scientists, as they evolve to satisfy new business requirements.
Miscellaneous. The remaining job titles are a mixed bag, ranging from engineers to marketers. This reminds us that big data analytics – and therefore Hadoop, too – is undergoing a democratization that makes it accessible to an ever-broadening range of end users who depend on data to do their jobs well.
Want more? Register for my Hadoop4BIDW Webinar, coming up April 9, 2013 at noon ET:
http://bit.ly/Hadoop13
Posted by Philip Russom, Ph.D. on March 25, 20130 comments
To succeed with business intelligence (BI), sometimes you have to buck tradition, especially if you work at a fast-paced company in a volatile industry.
And that’s what Eric Colson did when he took the helm of Neflix’ BI team last year. He quickly discovered that his team of BI specialists moved too slowly to successfully meet business needs. “Coordination costs [among our BI specialists] were killing us,” says Colson.
Subsequently, Colson introduced the notion of a “spanner”—a BI developer who builds an entire BI solution singlehandedly. The person “spans” all BI domains, from gathering requirements to sourcing, profiling, and modeling data to ETL and report development to metadata management and Q&A testing.
Colson claims that one spanner works much faster and more effectively than a team of specialists. They work faster because they don’t have to wait for other people or teams to complete tasks or spend time in meetings coordinating development. They work more effectively because they are not biased to any one layer of the BI stack and thus embed rules where most appropriate. “A traditional BI team often makes changes in the wrong layer because no one sees the big picture,” Colson says.
Also, since spanners aren’t bound by a written contract (i.e., requirements document) created by someone else, they are free to make course corrections as they go along and “discover” the optimal solution as it unfolds. This degree of autonomy also means that spanners have higher job satisfaction and are more dedicated and accountable. One final benefit: there’s no fingerpointing, if something fails.
Not For Everyone
Of course, there are downsides to spanning. First, not every developer is capable of spanning. Some don’t have the skills, and others don’t have the interest. “We have lost some people,” admits Colson. Finding the right people isn’t easy, and you must pay a premium in salary to attract and retain them. Plus, software license costs increase because each spanner needs a full license to each BI tool in your stack.
Second, not every company is well suited spanners. Many companies won’t allocate enough money to attract and retain spanners. And mature companies in regulated or risk-averse industries may work better with a traditional BI organization and development approach.
Simplicity
Nonethless, experience shows that the simplest solution is often the best one. In that regard, spanners could be the wave of the future.
Colson says that using spanners eliminates much of the complexity of running BI programs and development projects. The only thing you need is a unifying data model and BI platform and a set of common principles, such as “avoid putting logic in code” or “account ID is a fundamental unifier.” The rest falls into the hands of the spanners who rely on their skills, experience, and judgment to create robust local applications within an enterprise architecture. Thus, with spanners, you no longer need business requirement analysts or requirements documents, a BI methodology, project managers , and a QA team, says Colson.
This is certainly pretty radical stuff, but Colson has proven that thinking and acting outside the box works, at least at Neflix. Perhaps it’s time you consider following suit!
Posted on October 21, 20100 comments
Every once in a while, you encounter a breath of fresh air in the business intelligence field. Someone who approaches the field with a fresh set of eyes, a barrel full of common sense, and the courage to do things differently. We were fortunate at TDWI’s BI Executive Summit this August in San Diego to have several speakers who fit this mold.
One was Ken Rudin, general manager of analytics and social networking products, at Zynga, the online gaming company that produces Farmville and Mafia Wars, among others. Ken discussed how Zynga is “an analytics company masquerading as an online gaming company.”
Centralized High Priests
One of the many interesting things he addressed was how he reorganized his team from a centralized “high priest” model to a distributed “embedded” model. When he arrived at Zynga, studio heads or project managers would submit requests for analytical work to the corporate analytics team. According to Ken, this approach led business people to view analytics as external or separate from what they do, as something delivered by experts whose time needed to be scheduled and prioritized. In other words, they didn’t view analytics as integral to their jobs. It was someone else’s responsibility.
Another downside of the centralized model (although Ken didn’t mention this) is that analysts become less efficient. Since they are asked to analyze issues from a variety of departments, it takes them more time to get up to speed with germane business and technical issues. As a result, they can easily miss key issues or nuances that lead to below par results. Moreover, analysts serving in a “high priest” approach often feel like “short order cooks” who take requests in a reactive manner and don’t feel very engaged in the process.
Embedding Analysts
To improve the effectiveness of his analytical team, Rudin “gave away” his headcount to department heads. He said he had to go “hat in hand” and ask department heads to assume funding of his analysts. He thought the department heads would push back, but the opposite occurred: the department heads were thrilled to have analysts dedicated to their teams, and even agreed to fund more hires than Rudin proposed.
Embedding analysts in the departments helped cultivate a culture of analytics at Zynga. Each analyst became part of the team involved in designing the games. Their role was to suggest ways to test new new ideas and examine assumptions about what drives retention and longevity. By providing scientific proof of what works or doesn’t work, the embedded model created a perfect blend of “art and science” that has helped fuel Zynga’s extraordinary growth. “Art without science doesn’t work, says Rudin.
To maintain the cohesiveness of an analytical team that doesn’t report to him, Rudin holds 30 minute scrums where all analysts meet daily to share what they’ve learned and develop ideas about approaches, metrics, or tests that might apply across departments. Rudin also maintains two senior analysts who perform cross-departmental analyses.
Rudin had other gems in his presentation so stay tuned for analytical insights in this blog.
Posted on September 21, 20100 comments
Designing dashboards is not unlike decorating a room in your house. Most homeowners design as they purchase objects to place in the room. When we buy a rug, we select the nicest rug; when we pick out wall paint, we pick the most appealing color; when we select chairs and tables, we find the most elegant ones we can afford. Although each individual selection makes sense, collectively the objects clash or compete for attention.
Smart homeowners (with enough cash) hire interior decorators who filter your tastes and preferences through principles of interior design to create a look and feel in which every element works together harmoniously and emphasizes what really matters. For example, the design might highlight an elegant antique coffee table by selecting carpets, couches, and curtains that complement its color and texture.
To optimize the design of your performance dashboard, it is important to get somebody on the team who is trained in the visual design of quantitative information displays. Although few teams can afford to hire someone full time, you may be able to hire a consultant to provide initial guidance or find someone in the marketing department with appropriate training. Ideally, the person can educate the team about basic design principles and provide feedback on initial displays.
But be careful: don’t entrust the design to someone who is a run-of-the-mill graphic artist or who is not familiar with user requirements, business processes, and corporate data. For example, a Web designer will give you a professional looking display but will probably garble the data—they might use the wrong type of chart to display data or group metrics in nonsensical ways or apply the wrong filters for different user roles. And any designer needs to take the time upfront to understand user requirements and the nature of the data that will populate the displays.
Ideally, report developers and design experts work together to create an effective series of dashboard displays, complementing their knowledge and expertise. This partnership can serve as a professional bulwark against the sometimes misguided wishes of business users. Although it’s important to listen to and incorporate user preferences, ultimately the look and feel of a dashboard should remain in the hands of design professionals. For example, most companies today entrust the design of their Web sites and marketing collateral to professional media designers who work in concert with members of the marketing team. They don’t let the CEO dictate the Web design (or shouldn’t anyway!)
There are many good books available today to help dashboard teams bone up on visual design techniques. The upcoming second edition of my book, “Performance Dashboards: Measuring, Monitoring, and Managing Your Business” has a chapter devoted to visual design techniques. Many of the concepts in the chapter are inspired by Stephen Few whose “Information Dashboard Design” book is a must read. Few and others have drawn inspiration from Edward R. Tufte, whose book, “The Visual Display of Quantitative Information,” is considered a classic in the field. Tufte has also written “Visual Explanations,” “Envisioning Information” and “Beautiful Evidence.”
Posted by Wayne Eckerson on July 27, 20100 comments
(Caution: This blog may contain ideas that are hazardous to your career.)
I’ve argued in previous blogs that business intelligence (BI) professionals must think more like business people and less like IT managers if they are to succeed. However, while many BI professionals have their hearts in the right place, their actions speak differently. They know what they need to do but can’t seem to extricate themselves from an IT mindset. That takes revolutionary thinking and a little bit of luck.
Radical Thinking
So, here’s a radical idea that will help you escape the cultural bonds of IT: don’t upgrade your BI software.
Now, if you gasped after reading that statement, you’re still an IT person at heart. An IT person always believes in the value of new software features and fears losing vendor support and leverage by not staying fairly current with software licenses and versions.
Conversely, the average business person sees upgrades as a waste of time and money. Most don’t care about the new functionality or appreciate the financial rationale or architectural implications. To them, the upgrade is just more “IT busywork.”
Here’s another radical idea: stick with the BI tools you have. Why spend a lot of money and time migrating to new platform when the one you have works? So what if the tools are substandard and missing features? Is it really a problem if the tools force your team to work overtime to make ends meet? Who are the tools really designed to support: you or the users?
In the end, it’s not the tools that matter, it’s how you apply them. Case in point: in high school, I played clarinet in the band. One day, I complained vociferously to the first chair, a geeky guy named Igor Kavinsky who had an expensive, wooden clarinet (which I coveted), that my cheap, plasticized version wasn’t working very well and I needed a replacement. Before I could list my specific complaints, he grabbed my clarinet, replaced the mouthpiece, and began playing.
Lo and behold, the sound that came from my clarinet was beautiful, like nothing I had ever produced! I was both flabbergasted and humiliated. It was then I realized that the problem with my clarinet not the instrument but me! Igor showed me that it’s the skill of the practitioner, not the technology, that makes all the difference.
Reality Creeps In
Igor not withstanding, if you’re a good, well-trained IT person, you probably think my prior suggestions are unrealistic, if not ludicrous. In the “real world,” you say, there is no alternative to upgrading and migrating software from time to time. These changes—although painful—improve your team’s ability to respond quickly to new business needs and avoid a maintenance nightmare. And besides, many users want the new features and tools, you insist.
And of course, you are right. You have no choice.
Yet, given the rate of technology obsolescence and vendor consolidation, your team probably spends 50% of its time upgrading and migrating software. And it spends its remaining time maintaining both new and old versions (because everyone knows that old applications never die.) All this busywork leaves your team with precious little time and resources to devise new ways to add real value to the business.
Am I wrong? Is this a good use of your organization’s precious capital? What would a business person think about the ratio of maintenance to development dollars in your budget?
Blame the Vendors. It’s easy to blame software vendors for this predicament. In their quest for perpetual growth and profits, vendors continually sunset existing products, forcing you (the hapless customer) with no choice but to upgrade or lose support and critical features. And just when you’ve fully installed their products, they merge with another company and reinvent their product line, forcing another painful migration. It’s tempting to think that these mergers and acquisitions are simply diabolical schemes by vendors to sell customers expensive, replacement products. Just ask any SAP BI customer!
Breaking the Cycle
If this describes your situation, what do you do about it? How do you stop thinking like an IT person and being an IT cuckold to software vendors?
Most BI professionals are burrowed more deeply in an IT culture than they know. Breaking free often requires a cataclysmic event that rattles their cages and creates an opening to escape. This might be a change in leadership, deregulation, a new competitor, or a new computing platform. Savvy BI managers seize such opportunities to reinvent themselves and change the rules of the game.
Clouds Coming. Lucky for you, the Cloud—or more specifically, Software as a Service (SaaS)—is one of those cataclysmic events. The Cloud has the potential to liberate you and your team from an overwrought IT culture that is mired in endless, expensive upgrades and painful product migrations, among other things.
The beauty of a multi-tenant, cloud-based solution is that you never have to upgrade software again. In a SaaS environment, the upgrades happen automatically. To business and IT people, this is magical: cool new features appear and no one had to do any work or suffer any inconvenience. SaaS also eliminates vendor lock in since you can easily change cloud vendors (as long as you maintain your data) by just pointing users to a new URL. The Cloud is a radical invention that promises to alter IT culture forever.
Getting Started. To break the cycle, start experimenting with cloud-based BI solutions. Learn how these tools work and who offers them. Use the cloud for prototypes or small, new projects. Some cloud BI vendors offer a 30-day free trial while more scalable solutions promise to get you up and running quickly. If you have a sizable data warehouse, leave your data on premise and simply point the cloud BI tools to it. Performance won’t suffer.
Unless you experiment with ways to break free from an IT culture, you never will. Seize the opportunity that the Cloud affords and others that are sure to follow. Carpe diem!
Posted by Wayne Eckerson on June 25, 20100 comments
Business analysts are a key resource for creating an agile organization. These MBA- or PhD-accredited, number-crunchers can quickly unearth insights and correlations so executives can make critical decisions. Yet, one decision that executives haven’t analyzed thoroughly is the best way to organize business analysts to enhance their productivity and value.
Distributed Versus Centralized
Traditionally, executives either manage business analysts as a centralized, shared service or allow each business unit or department to hire and manage their own business analysts. Ultimately, neither a centralized or distributed approach is optimal.
Distributed Approach. In a distributed approach, a department or business unit head hires the analyst to address local needs and issues. This is ideal for the business head and departmental managers who get immediate and direct access to an analyst. And the presence of one or more analysts helps foster a culture of fact-based decision making. For example, analysts will often suggest analytical methods for testing various ideas, helping managers become accustomed to basing decisions on fact rather than gut feel alone.
However, in the distributed approach, business analysts often become a surrogate data mart for the department. They get bogged down creating low-value, ad hoc reports instead of conducting more strategic analyses. If the business analyst is highly efficient, the department head often doesn’t see the need to invest in a legitimate, enterprise decision-making infrastructure. From an analyst perspective, they often feel pigeonholed in a distributed approach. They see little room for career advancement and few opportunities to expand their knowledge in new areas. They often feel isolated and have few opportunities to exchange ideas and collaborate on projects with fellow analysts. In effect, they are “buried” in departmental silos.
Centralized Approach. In a centralized approach, business analysts are housed centrally and managed as a shared service under the control of a director of analytics, or more likely, a chief financial officer or director of information management. One benefit of this approach is that organizations can assign analysts to strategic, high priority projects rather than tactical, departmental ones, and the director can establish a strong partnership with the data warehousing and IT teams which control access to data, the fuel for business analysts. Also, by being co-located, business analysts can more easily collaborate on projects, mentor new hires, and cross-train in new disciplines. This makes the environment a more rewarding place to work for business analysts and increases their retention rate.
The downside of the centralized approach is that business analysts are a step removed from the people, processes, and data that drive the business. Without firsthand knowledge of these things, business analysts are less effective. It takes them longer to get up to speed on key issues and deliver useful insights, and they may miss various nuances that are critical for delivering a proper assessment. In short, without a close working relationship with the people they support and intimate knowledge of local processes and systems, they are running blind.
Hybrid Approach
A more optimal approach combines elements of both distributed and centralized methods. In a hybrid environment, business analysts are embedded in departments or business units but report directly to a director of analytics. This sounds easy enough, but it’s hard to do. It’s ideal when the company is geographically consolidated in one place so members of the analytics team can easily physically reconvene as a group to share ideas and discuss issues.
Zynga. For example, Zynga, an internet gaming company, uses a hybrid approach for managing its analysts. All of Zynga’s business analysts report to Ken Rudin, director of analytics for the company. However, about 75% of the analysts are embedded in business units, working side by side with product managers to enhance games and retain customers. The remainder sit with Rudin and work on strategic, cross-functional projects. (See “Proactive Analytics That Drive the Business” in my blog for more information on Zynga’s analytics initiative.) This setup helps deliver the best of both centralized and distributed approaches.
Every day, both distributed and centralized analysts come together for a quick “stand up” meeting where they share ideas and discuss issues. This helps preserve the sense of team and fosters a healthy exchange of knowledge among all the analysts, both embedded and centralized. Although Zynga’s analysts all reside on the same physical campus, a geographically distributed team could simulate “stand up” meetings with virtual Web meetings or conference calls.
Center of Excellence. The book “Analytics at Work” by Tom Davenport, Jeanne Harris, and Robert Morison describes five approaches for organizing analysts, most of which are variations on the themes described above. One approach, “Center of Excellence” is similar to the Hybrid approach above. The differences are that all (not just some) business analysts are embedded in business units, and all are members of (and perhaps report dotted line to) a corporate center of excellence for analytics. Here, the Center of Excellence functions more like a program office that coordinates activities of dispersed analysts rather than a singular, cohesive team, as in the case of Zynga.
Either approach works, although Zynga’s makes it easier for an inspired director of analytics to shape and grow the analytics department quickly and foster a culture of analytics throughout the organization.
Summary. As an organization recognizes the value of analytics, it will evolve the way it organizes its business analysts. Typically, companies will start off on one extreme—either centralized or distributed—and then migrate to a more nuanced hybrid approach in which analysts report directly to a director of analytics (i.e., Zynga) or are part of a corporate Center of Excellence.
Posted by Wayne Eckerson on May 23, 20100 comments
To create high-performance BI teams, we need to attract the right people. There are a couple of ways to do this.
Skills Versus Qualities
Inner Drive. First, don’t just hire people to fill technical slots. Yes, you should demand a certain level of technical competence. For example, everyone on the award winning BI team at Continental Airlines in 2004 had training as a database administrator. But these days, technical competence is simply a ticket to play the game. To win the game, you need people who are eager to learn, highly adaptable, and passionate about what they do.
The bottom line is that you shouldn’t hire technical specialists whose skills may become obsolete tomorrow if your environment changes. Hire people who have inner drive and can reinvent themselves on a regular basis to meet the future challenges your team will face. If you need pure technical specialists, consider outsourcing or contracting people to fill these roles.
Think Big. To attract the right people, it’s important to set ambitious goals. A big vision and stretch targets will attract ambitious people who seek new challenges and opportunities and discourage risk-adverse folks who simply want a “job” and a company to “take care” of them. One way to think big is to run the BI group like a business. Create mission, vision, and values statements for your team and make sure they align with the strategic objectives of your organization. Put people in leadership positions, delegate decision making, and hold them accountable for results. Reward them handsomely for success (monetarily or otherwise) yet don’t punish failure. View mistakes as “coaching opportunities” to improve future performance or as evidence that the team needs to invest more resources into the process or initiative.
Performance-based Hiring
Proactive Job Descriptions. We spend a lot of time measuring performance after we hire people, but we need to inject performance measures into the hiring process itself. To do this, write proactive job descriptions that contain a mission statement, a series of measurable outcomes, and the requisite skills and experience needed to achieve the outcomes. Make sure the outcomes are concrete and tangible: such as “improve the accuracy of the top dozen data elements to 99%” or “achieve a rating of “high” or better from 75% of users in the annual BI customer satisfaction survey” or “successfully deliver 12 major projects a year on time and in budget.”
If done right, a proactive job description helps prospective team members know exactly what they are getting into. They know specific goals they have to achieve and when they have to achieve them. A proactive job description helps them evaluate honestly whether they have what it takes to do the job. It also becomes the new hire’s performance review and a basis for merit pay. For more information on writing these proactive job descriptions, I recommend reading the book, “Who: A Method for Hiring” by Geoff Smart and Randy Street.
Where are they? So where do you find these self-actuated people? Steve Dine, president of DataSource Consulting and a TDWI faculty member, says he tracks people on Twitter and online forums, such as TDWI’s LinkedIn group. He evaluates them by identifying the number of followers they have and assessing the quality of advice they offer. And he avoids the major job boards, such as Monster.com or Dice.com. A more direct and surefire method is to hire someone on a contract basis to see what kind of work they do and how they get along with others on your team.
Retaining the Right People
Finally, to retain your high-performance team, you need to understand what makes BI professionals tick. Salary is always a key factor, but not the most important one. (See TDWI’s annual Salary, Roles, and Responsibility Report.) Above all else, BI professionals want new challenges and opportunities to expand their knowledge and skills. Most get bored if forced to specialize in one area for too long. It’s best to provide BI professionals with ample opportunity to attend training classes so they can pick up new skills.
One way to retain valuable team members is to create small teams responsible for delivering complete solutions. This gives team members exposure to all technologies and skills needed to meet business needs and also gives them ample face time with the business folks who use the solution. BI professionals are more motivated when they understand how their activities contribute to the organization’s overall success. Another retention technique is to give people opportunities to exercise their leadership skills. For instance, assign your rising stars to lead small, multidisciplinary teams where they define the strategy, execute the plans, and report their progress to the team as a whole.
Summary. By following these simple techniques, you will attract the right people to serve on your team and give them ample opportunity to stay with you. For more information on creating high performance BI teams, see a recent entry at my Wayne’s World blog titled “Strategies for Creating High Performance Teams” and the slide deck by the same title at my LinkedIn Page.
Posted by Wayne Eckerson on May 20, 20100 comments
Author’s Note: Given the positive reaction to my last blog, “Purple People: The Key to BI Success”, I thought I’d publish an excerpt from the second edition of my book, “Performance Dashboards: Measuring, Monitoring, and Managing Your Business” due out this fall. This excerpt focuses on the responsibility of the business to close the business-IT gap, something that doesn’t get enough attention these days.
Aligning Business and IT
Tension Abounds. There has always been distrust between the business and the technical sides of an organization, but performance dashboard projects seem to heighten the tension to extreme levels. I have been in the technology industry for 20 years, and frankly, I’ve been shocked by the intensity of the distrust that I have witnessed between these two groups while researching this book.
Although there is much talk about the need to align business and information technology (IT) departments, little progress has been made. Part of the problem is systemic to IT departments and technical people, but another part involves the willingness of business executives and managers to engage with IT constructively on a long-term basis….
Counseling for Business
Although IT groups generally get the lion’s share of the blame for misalignment between business and IT, it takes two to tango, as they say. The business shares equal responsibility for the tension between the two groups—perhaps more so, since it does not always recognize how its actions and behavior contribute to the problem.
The business needs to understand that it often moves too fast for IT to keep up. It harbors a short-term bias toward action and rarely takes a long-term view toward building sustainable value. This is especially true in U.S. companies, whose Wild West heritage makes them notorious for acting first and asking questions later. The business needs to slow down sometimes and ask whether change is really needed or if they are reacting in knee-jerk fashion to the latest event or issue of the day. They need to prioritize their needs and evaluate what they need now and what can wait. By working incrementally, the business discovers that it gets what it needs faster than by trying to build a solution all at once.
Decentralized organizations magnify this behavior, parceling out authority to divisions and departments to make decisions faster and in the context of local markets. Although there are advantages to decentralization, there are considerable downsides that contribute to the perpetual misalignment of the business and IT. The scores of analytical and operational silos, including the hundreds and thousands of pernicious spreadmarts that hamstring corporate productivity, testify to the business’ fixation with speed and decentralized decision making.
Lack of Self Discipline. Finally, the business has the upper hand in its relationship with IT, and it often leverages its position in a high-handed and capricious manner. In many organizations, executives threaten to outsource or offshore IT when it does not deliver sufficient value, ignoring the possibility that their own actions and decisions may have crippled IT’s ability to function effectively. The business also lacks a reasonable degree of restraint and self-discipline when it comes to IT projects. One IT manager said his company’s annual technology planning process is a sham because the business cannot discipline itself to live within its limits.
“Prior to the beginning of every calendar year, the business prioritizes IT projects for the next 12 months. Out of 90 projects, they identify 60 of them as ‘high priority’ and we create a schedule to deliver them,” says the beleaguered IT manager. “But even before January 1st arrives, the business adds 20 more ‘high-priority’ projects to our list and adds another 20 projects before April. And then they tell us in March that we are already two months behind schedule!”
There are many ways to align business and IT and create a lasting sustainable partnership. In the world of BI, these range from gathering better business requirements and adopting new agile development methodologies to creating a BI governance program and creating a BI Center of Excellence that leverages business analysts and super users. My book and blogs go into detail on these and other techniques. Stay tuned!
Posted by Wayne Eckerson on May 6, 20100 comments