Data Ops: What Is It? Do We Need It? (Part 2 of 2)
Six common pitfalls for busy data management teams.
- By Jill Dyché
- April 17, 2018
In my previous column, reader Clayton wondered whether he and his enterprise data management team should be threatened by the pending launch of an adversary's new data ops team.
My answer was that Clayton had nothing to worry about if he and his team had every phase of the data life cycle covered. How to know for sure? I've identified six common pitfalls for busy data management teams. Use this as an abbreviated health check for your own team to see where you might be falling short.
Pitfall #1: Lack of project breadth
Many data teams in high demand work on only the most visible projects. Be careful not to put all your data skills into a few buckets. Your team should be contributing to a range of business initiatives, from supporting basic revenue reports to preparing and enriching data for machine learning. Focusing too narrowly on high-profile or complex projects can unfairly pigeonhole a data team, making room for interlopers.
Pitfall #2: Need for workflow
Workflow is the unsung hero of successful data management. It doesn't just formalize effective processes; it circumscribes inputs and outputs, clarifying roles and ownership boundaries. Ideally, it's automated via workflow management or data curation software. Absent workflow, data teams can become myopic and insular, ignoring their data's transcendent value.
Pitfall #3: Manual processes
Every time I decide data automation software has finally become a commodity, I meet a business professional working from a spreadsheet, inevitably emailed by a colleague who has tinkered with it before forwarding it. Sometimes these spreadsheets contain sensitive data. Too often they're considered authoritative sources. Manual and paper-based processes deter efficiencies, and they compromise data quality and jeopardize data security.
Pitfall #4: Missing standards
Data stewards, wranglers, and scientists who have been knee-deep in data tend to devise effective and repeatable processes, making it easier to quickly glean value from data. Once proven, these processes should be adopted more broadly, formalized, automated, and annotated, their artifacts organized for easy access. Standardizing data-specific processes is not only a data management best practice, it's a rallying cry of the data ops crusaders.
Pitfall #5: Lack of consistent quality improvement
By combining the collaboration endemic to agile development with deliberate delivery processes, data teams can, in theory, discover and remedy quality issues more quickly. Error detection in data and its associated metadata and business rules is a critical component of data ops (as well as its forebear, DevOps), never mind a data management best practice. Likewise, quality monitoring should be a discrete and repeatable exercise in a data ops environment.
Pitfall #6: Static cycle times and delivery bottlenecks
That time frames should improve with repeatable processes and optimized tools is a fundamental tenet of data ops. I've said it before and I'll say it again: maturity is not found in technology but in delivery. This is as true for data as it is for strategy, team effectiveness, or your blockchain technology du jour. If the lack of continuous -- agile, if you must -- delivery has impeded an important business program, your team is likely in the crosshairs of its rivals or worse, its constituents.
Additional Benefits
Did this list make you nervous? Maybe you have some holes to plug. Maybe, if you're lucky, Ken is still devising his business plan, giving you time to remedy your team's shortcomings and fix what might be broken.
However, let's give Ken the benefit of the doubt and assume his new data ops team is an earnest effort to enhance your company's data-driven culture. Consider that you and your team might actually benefit from Ken's effort.
Make a list of the tasks your team performs that are operationally focused. For instance, maybe data acceptance tests should be more routine. Perhaps contract negotiation with third-party data providers consumes half a headcount, or there's a backlog of data correction requests that never ends. These tasks could well fall under a data ops team, and you might not even miss them. Delegating them to Ken's team would free your team members to engage more with businesspeople who use the data in the context of their day jobs. As many have learned the hard way, the (data) expert most aligned to the (data) consumer always wins.
Aim high, Clayton, and let me know how it goes!
About the Author
Jill Dyché has advised clients and executive teams on their analytics and data programs for as long as she can remember. Longer, in fact. She’s the author of four books on the business value of technology and regularly talks to teams about what keeps them up at night. Ambivalent about analytics? Maddened by management? Constricted by your culture? Check out Jill’s Q&A column, Q&A with Jill Dyché, here.