The Benefits of Balancing Self-Service and Enterprise Data Systems
If organizations use enterprise systems to solve common challenges, users can focus on resolving business questions and improving performance rather than on how to query and access data.
- By David Stodder
- July 21, 2020
Amid turbulent times, decision makers at all levels need the best and most relevant data possible. To respond to an economy unexpectedly reshaped by the coronavirus pandemic, organizations are having to retool business processes, services, and products to align with changes in customer demand and engagement. Organizations are keenly aware of the importance of empowering decision makers to use data effectively to analyze current situations in context and uncover trends and patterns for better forecasting and predictive insights.
It continues to be a high priority for most organizations to expand users' self-reliance with business intelligence (BI), visual analytics, and data preparation, according to TDWI research. Yet, organizations are also finding that self-service technologies alone do not cause good decisions and greater satisfaction with data. This requires a careful balance with (typically IT-led) enterprise systems and governance.
Without a good balance, organizations have trouble solving common challenges such as lack of standardization, inconsistent data quality, difficulty getting a single view of the truth, and unsatisfactory performance when users compose queries poorly or are unaware of the data access complexities and processing power needed to answer their queries. These are problems that enterprise platforms (including modern, cloud-based services) can help resolve. Organizations can use enterprise platforms to gain a perspective on users' data interactions across the organization rather than just within a single department or project team using self-service technologies.
Modernizing both enterprise and self-service technologies with artificial intelligence (AI) will be important to enabling this balance. Enterprise data management, data integration, and data catalogs have often involved too much manual effort on the part of IT. Using AI techniques such as machine learning (ML) allows enterprise systems to increase scalability and flexibility to handle bigger volumes and more diverse data. AI/ML is also key to enhancing these aspects of self-service technologies and improving data exploration, analytics, and development of actionable insights provided to users in the form of recommendations.
A Balanced Approach Improves Governance
At the enterprise level, organizations can establish data governance and security rules and data quality processes for all personnel to follow. IT can use an enterprise data catalog to make common metadata definitions available to all users, developers, and applications. The data catalog can make it easier for IT to find data faster across on-premises and cloud-based data systems to meet data inventory requirements in regulations such as the European Union's General Data Protection Regulation (GDPR), the California Consumer Privacy Act (CCPA), and other data privacy regulations within the U.S. and around the world.
Organizations can use an enterprise metadata resource such as a data catalog as part of a larger objective of ensuring that users are accessing and sharing trusted, well-governed data through their dashboards and reports. The resource will help resolve disputes among users about which data sources are most trusted, which calculated values and aggregations are best to use, and the accepted definitions for higher-level entities such as "customers."
If enterprise platforms can enable organizations to iron out these and other quality and consistency issues -- and IT can effectively communicate and integrate these standards with departments employing self-service technologies -- users will have greater confidence and trust in their dashboards and analytics. It will be easier to gain a single view of the truth where necessary for reports or specific performance metrics. Joint governance committees can ensure that similar or related performance metrics developed at the departmental or operational level are integrated with those used at the enterprise level.
The key, however, is balance. Users want -- and expect -- greater freedom and empowerment. Traditional enterprise BI reporting systems have a reputation for inflexibility and "one size fits all" sets of features and functionality. They may give users the benefits of standardization but at the cost of a lack of freedom in how they can visualize and interact with data. Modern enterprise systems must be less intrusive; they must fill the gaps in terms of standardization and governance but not overtake users' freedom to use self-service technologies to personalize visualizations, filters, and other parameters, and even to explore different types of data that they might want to incorporate.
Along with addressing data quality, consistency, and governance concerns, there are two other issues enterprise systems are able to address.
Promoting reuse and predefinition for efficiency and performance. A common byproduct of increased self-service is that users create their own dashboards and ad hoc queries that may be the same as or similar to those others use. Data systems are then swamped by nearly redundant and poorly composed queries. Quality standards weaken as best practices are insufficiently shared.
Organizations can reduce quality and consistency problems by employing enterprise systems. They can focus on eliminating redundancy and applying best practices for increasing reuse. At the enterprise level, IT can define common needs and create and execute predefined queries to optimize overall performance.
Enabling easier development and embedding in applications. Today's applications need to be more data-driven, which means that BI reporting and analytics systems need to be better integrated within them. Then, data insights can inform application processes so that operational users or automated systems make faster and better decisions.
Enterprise systems are a good place to formalize integration with applications across the organization. Organizations should evaluate opportunities to embed interactive dashboards and visualizations. These must be supported by enterprise governance and the use of standard integration technologies and application programming interfaces (APIs).
By using enterprise systems effectively to solve common challenges, organizations will make it easier for users to focus on resolving business questions and improving performance rather than on how to query and access data. With a good balance between self-service capabilities and enterprise governance and data quality, organizations can gain the benefits of both types of platforms. A good balance will enable organizations to keep total cost of ownership in check by reducing inefficiency and redundancy while giving users the flexibility and personalization they need.
A Final Word
I invite members of the TDWI community to participate in a research survey for our upcoming Q3 2020 TDWI Best Practices Report: The Future of Business Analytics: Augmented, Self-Service Front Ends Supported by Modern Data Management. For this report, I will be examining issues discussed in this article as well as how organizations can achieve a better balance between users' interests in self-reliance and agility and enterprise IT responsibilities for governance, data quality, consistency, and performance.
Along with self-service BI and visual analytics, the research will look at technology trends in data catalogs, metadata management, data integration and virtualization, cloud data management, and AI for BI -- all of which are critical to delivering what users want in self-service capabilities. Thank you in advance for your participation. You can access the survey at https://www.surveymonkey.com/r/2GNGFPJ