How Modern Metadata Management Can Boost BI Productivity
Data warehousing and analytics are changing, so metadata management can't stand still. We explain what's driving the push to modernize metadata management and marketplace trends that ease the process.
- By James E. Powell
- December 11, 2018
If yours is like most enterprises, you're constantly modernizing -- from introducing new data types to incorporating new data platforms. You're modernizing analytics, too, moving beyond OLAP and into more advanced analytics for different forms of mining, clustering, graphs, statistics -- you name it. So why not modernize your metadata management?
As Philip Russom, senior research director for data management at TDWI, noted in a recent webinar, "Modern Metadata Management: Boosting BI Capacity with Automation and Machine Learning," traditional metadata management is notoriously manual, so it's time-consuming. Furthermore, it's siloed. "We often populate metadata on a per-tool or per-database basis, which really narrows your visibility into data through metadata and it limits your BI capacity."
By BI capacity, Russom is referring to your ability to churn out large numbers of reports, analytics, data sets, new subject areas for your data warehouse, and new data integration or data quality solutions. "Modernizing metadata can give you automation, intelligence, and centralization, much of which is achieved through the metadata tool and the infrastructure that goes with it."
Through metadata that's centralized and enterprise in scope, you can find more of your customer data. With greater sharing of metadata, coworkers can enjoy a productivity boost because people don't have to reinvent the wheel when it comes to finding and exploiting your data assets.
Why Modernize?
Why modernize metadata and why do it now? Why is there a sense of urgency?
Developers, Russom points out, need to accelerate data-driven development. Business decision makers can no longer wait weeks or months while new reports are built or a new subject area is added to the warehouse. Competitors are moving faster, putting even more pressure on developers. Amid this push to change development methods and become more agile and lean, the right kind of metadata and other semantics (such as business metadata glossaries) can help you do just that -- and therefore get the job done more quickly as a data-driven developer.
Another part of the equation involves new tool functionality. "We're seeing some better automation built into tools from the vendor community, but also the open source community. Automation can be done through machine learning, which can enable a kind of predictive analytics within, say, a metadata management tool. That way, metadata management can parse data and do a better job of predicting how you want to represent that data in the metadata realm."
Finally, modernizing metadata can enable new practices -- in particular, helping self-service users. This isn't limited to business users. "Yes, metadata is for businesspeople who know enough about data to go browse and do a certain amount of data prep on their own -- but don't forget tech people, even someone as powerful as a data scientist. Sometimes they just need to get a real quick read of data before diving into a technical solution."
Metadata Management Advancements
In his presentation, Russom takes note of several changes that enable this management modernization. For example, tool automation based on various types of intelligence is getting better at predicting data mappings, especially in data integration tools. Even if developers must slightly modify the mappings, "you're working faster. Eventually for certain types of mappings, we'll get to the point where the tool just does it and maybe tells you about it later, but you don't really have to get involved." The same goes for data modeling.
Russom also points to analytics folded into data management tools (including analytics visualization) and new platforms as modernization enablers. He explains a new type of intelligence called modern metadata scanning, the importance of cloud tools, and software-as-a-service in particular.
"Modern metadata can enable new practices. In my mind, self-service is top of that list, but I know some people ... want to do more virtualization. Modern metadata can help you build up with that, onboarding new systems for IoT and so forth. [Also,] consolidating metadata silos helps you put together a wide range of views, from that single customer view I know many of you are trying to build to that very broad enterprise view of data assets for governance and other compliance reasons."
Time to Get Started
Metadata is more relevant than ever, Russom emphasizes. "We're seeing so many changes in IT, changes in technology, especially in this broad realm of data management and all the analytics and reporting that goes with it. Yet metadata is still relevant because all data-driven action goes through metadata. If you want to browse data, you're actually browsing metadata. Want to run a query? You're going to structure the query using metadata, and then metadata turns around and does double duty as an interface level."
Metadata is mission-critical if you want to be a data-driven business, improve change management, manage your data inventory, and ensure you're compliant with the growing crop of security and privacy regulations. "If you have modern tools for metadata management, they can improve your productivity using automation, intelligence, centralization, and cloud-based systems. Metadata really can increase developer productivity, and less time per project means the business is faster and more agile.
"The sooner you modernize metadata, the sooner you're going to apply those productivity gains to making your business move faster."
About the Author
James E. Powell is the editorial director of TDWI, including research reports, the Business Intelligence Journal, and Upside newsletter. You can contact him
via email here.