Data Ops: What Is It? Do We Need It? (Part 1 of 2)
Data ops applies agile and collaborative techniques to data delivery to align various and often far-flung data deployment tasks.
- By Jill Dyché
- April 16, 2018
If you're a regular visitor to TDWI's Upside, it's a good bet that you've found yourself in the data trenches. You've fought a few skirmishes, or maybe even been the embattled data scientist forging a path toward data ingestion, quality, and deployment victory.
This is no time to hide in a foxhole. Fighting a political battle at his company, Clayton recently asked me for some ammunition:
Hi, Jill.
I head up the data management center of excellence at a multinational media company. I was promoted into the role four years ago, chosen over two of my peers. One of the candidates now works for me as head data steward. The other candidate -- I'll call him Ken -- left the team and joined IT.
Our company considers itself data-driven. Every year I've received the budget and headcount I've requested, and every year my team meets its goals. We have been involved in some really strategic projects, and I think we deserve credit for their success. It's given my team and me some well-deserved visibility with upper management.
Recently Ken announced that he would be heading a "data ops" team reporting to the CIO. I can't help thinking that this is a political maneuver intended to dilute my team's responsibilities and, frankly, sully my reputation. I have asked Ken to meet with me several times, but he's ignored my emails. A member of Ken's staff explains he's busy working on his new team's charter.
Assuming I'm right to be suspicious of Ken's motives, can you describe data ops to me and how it's different from the classic data management team -- in other words, what my team is doing?
-- Clayton, Atlanta
Hi, Clayton. I usually like to assume that people are acting in the best interest of the companies that employ them. However, let's say you're right about Ken. To know whether he's launched a data ops team because of sour grapes or because he sees a genuine gap in data development depends on what your team is currently doing -- and maybe not doing.
Traditionally I define data management as the tactical execution of data governance policies -- with data governance being the business-driven policy making and oversight of corporate data.
Data ops tilts toward similar objectives. Piggybacking on the "DevOps" trend that reached its apotheosis in the last few years, data ops applies agile and collaborative techniques to data delivery with the goal of aligning various and often far-flung data deployment tasks.
Data ops rests on the assumption that sometimes data will be sitting, static, in a platform within the four walls of your data center. Sometimes it will be in the cloud. Sometimes it will be traveling across a network. Regardless of where it lives, data needs to be managed -- and secured -- in all those locations. Synchronizing heterogeneous activities -- including data sourcing, data prep, data cleansing, data loading, data testing, deployment, analysis, and science -- evolves toward an interconnected ecosystem for streamlining and accelerating data delivery, and hence business value. Data ops is, by definition, operational. However, like DevOps before it, data ops relies heavily on agile methods and interconnected processes.
DevOps exposes (indeed, it celebrates) operational transparency, allowing business people to peer under the covers to find process consistency across erstwhile delivery silos. Data ops likewise emphasizes data development rigor. Companies that are embracing data ops consider data-as-an-asset a given. As such, data warrants investment, organizational structure, delivery precision, and speed.
But, you may argue, my team has already mastered those challenges!
Fair enough, Clayton. Let's assume for a minute that Ken is diabolical and out to take you down. He's probably made a point of inventorying your weaknesses. He could be listing areas you and your team haven't addressed, planning to expose those shortcomings in order to justify his end run. The question is: Are you covered?
In my next article I'll list six potential Achilles' heels of data management teams that might provide an opening for political sabotage. Hopefully you can use them as a checklist to ensure your data management team has all its bases covered. Otherwise, you can watch as Ken builds his new data ops team, citing the deficiencies of the enterprise data management team -- your team, Clayton -- as his defense.
About the Author
Jill Dyché has advised clients and executive teams on their analytics and data programs for as long as she can remember. Longer, in fact. She’s the author of four books on the business value of technology and regularly talks to teams about what keeps them up at night. Ambivalent about analytics? Maddened by management? Constricted by your culture? Check out Jill’s Q&A column, Q&A with Jill Dyché, here.