TDWI | Training & Research | Business Intelligence, Analytics, Big Data, Data Warehousing

Executive Summary | Responsible Data and Analytics

The executive summary for the TDWI Best Practices Report: Responsible Data and Analytics

This TDWI Best Practices Report examines where organizations are today in terms of responsible data and analytics and what work they still need to do. It also addresses organizational imperatives and technologies to help organizations become more responsible.

• By Fern Halper, Ph.D., James G. Kobielus, David Stodder
• December 22, 2022

Enterprises are under increasing pressure to address social, environmental, and other responsible outcomes in their digital operations. To stay ahead of the curve, data management and analytics professionals have begun to address such topics as ethics, equity, fairness, safety, and sustainability in their strategic planning and operational practices.

For years, TDWI Research has covered such responsible data and analytics topics as data compliance, quality, privacy, security, protection, and bias mitigation. In order to manage data and analytics responsibly, enterprises often leverage platforms, tools, skills, and techniques that have proven useful in data compliance, data governance, and other established practices. At the same time, organizations are finding they need to evolve and extend these practices in order to serve a different set of stakeholders and use cases.

Responsible data and analytics considers the ethical, societal, compliance, legal, and environmental ramifications of using data in a wide variety of applications and processes. This TDWI Best Practices Report discusses the diverse range of use cases under the heading of responsible data and analytics. It defines the scope of this paradigm, discusses key business drivers, and baselines the current state of enterprise practices, platforms, and tools for addressing responsible data and analytics use cases.

This report uses findings from a survey of 214 data management and analytics professionals worldwide. It illustrates how enterprises are addressing the multifaceted requirements included under the paradigm of responsible data and analytics:

• Data quality, privacy, security, and compliance are the highest priorities. Data management and analytics professionals continue to place the highest priority on established requirements, such as data governance, trustworthiness, privacy, security, and regulatory compliance.
• Data transparency and explainability are important for enterprises to focus on. Transparency—the requirement that average humans be able to easily review and understand why predictive and other analytics models behave as they do—is a clear priority in a majority of organizations. Over half of those respondents using machine learning (ML) reported that their data and analytics organizations were comfortable explaining AI/ML algorithms.
• Data ethics and equity considerations are on the enterprise radar, but still take lower priority. Practitioners are starting to address newer imperatives such as data ethics, but rank these as less important drivers of their responsible data and analytics initiatives. Matters of data equity—such as whether analytics application designs support a range of disabilities and preferences—are not yet a high priority in enterprises. They are being addressed in the enterprises of around one-third of respondents, with a majority either not knowing or stating outright that these aren’t being addressed.
• Data sustainability is not yet a substantial practitioner focus. Environmental sustainability concerns, such as the carbon footprint from the nonrenewable energy used in training machine learning models, are still a low priority with enterprise data and analytics practitioners. Sustainability was cited as a priority by less than 25% of respondents. Nevertheless, companies have begun collecting data on the energy consumption, carbon footprint, and resource sustainability of their business and technical operations. Slightly less than half of respondents stated that their companies were either collecting this data or had plans to do so in the near future as part of their company-wide environmental, social, and governance initiatives.

To some degree, the enterprise journey to responsible data and analytics depends on building awareness, knowledge, and skills on these matters among both business and technical professionals. However, the survey showed that knowledge and training in responsible data and analytics knowledge is still weak, scattered, and spotty in many enterprises. Less than one in five respondents reported that data professionals in their organizations have training or knowledge about a full range of responsible data and analytics practices, including data privacy, data ethics, and data bias mitigation. Slightly under half reported their organizations’ data practitioners have some knowledge on a smattering of these topics. Less than one-third report that their data practitioners have knowledge on data privacy, and less than one in five organizations are providing training on data ethics.

This TDWI Best Practices Report examines these issues. It looks at where organizations are today in terms of responsible data and analytics and what work they still need to do. It also addresses organizational imperatives and technologies to help organizations become more responsible.

Denodo, SAP, and Snowflake sponsored the research and writing of this report.

• Download the full report