The Future Is Bespoke: Why Data Is a Medium for Creativity in this New World
Can data be the new canvas upon which stories of innovation and exemplary adaptation will be drawn?
- By Rishi Diwan
- April 26, 2021
The pandemic and subsequent move to remote work has put pressure on data to solve complex organizational and business challenges faster than ever before—from digital transformation and boosting online sales to creating new products and improving operational efficiencies. As data volumes continue to explode, the time window to act on insights continues to shorten. To remain competitive, organizations must act more quickly relying more than ever before on quality data while embracing innovation from applying artificial intelligence (AI) and machine learning (ML).
No two organizations are the same, of course. Each faces unique challenges and has access to unique opportunities. What they do share are common constraints around data, especially around volume and performance, placing limits on broad-based access and creativity which, in turn, limits their ability to approach problem-solving and decision-making in a tailored way.
If it seems overwhelming, there is a silver lining. Innovations in the data and analytics landscape are unleashing creativity more effectively by lifting these technical constraints, demystifying analytics, and enabling collaboration on creative solutions that are tailored to each problem or challenge. When the power of creativity is harnessed, organizations will be able to move away from a one-size-fits-all approach to data analytics and towards a future in which creative, bespoke solutions to their (analytics) problems are possible.
Recent innovation in the analytics database market allows for data to become a medium for creativity in two ways. First, improved analytics performance at scale lowers the cost of exploration to develop new ideas and make them more effective. Second, it means more teams have access to data that enables them to quickly explore new ideas to solve problems.
Creativity in an organizational context requires collaboration. Productivity tools such as spreadsheets and slide decks have been the canvas of choice for this type of creative collaboration in the past, giving everyone the same information in a standard format. More recently, data visualization dashboards and technologies to support collaboration for data scientists have become widely adopted. As new tools emerge for ad hoc creative collaboration, teams will have more opportunities to tailor their questions and the data they use to answer them.
Working with data creatively also requires a broad base of literacy in the organization. AI and ML applications have remained at the edges of business because they have been hard to understand for non-technical experts. This is changing. Semiautonomous decision making is starting to be used to drive cost reduction such as when triaging service calls. As understanding and trust in AI and ML capabilities grow, AI is rapidly moving towards core business workflows that drive more top-line initiatives, such as automated lead nurturing. Although governance issues still remain around ethical applications, debiasing, explainability, and monitoring, AI's move into the core business will drive new levels of creative innovation.
In this "bespoke future," line-of-business users will be able to think outside the box -- be creative -- and work together to create custom solutions that fit the needs of every new problem they come across. Imagine a cross-functional team being able to come together and quickly spawn a tailor-made analytical warehouse on top of massive volumes of quality data in a data lake without involving IT teams. This bespoke approach enables teams to explore creative answers as they need them and then discard or evolve this custom analytics warehouse when faced with the next problem because they can do it again and it will be tailored to the new problem. In this future, the notion of a centralized or even a permanent warehouse will feel idealized, constrained, and unsuitable for innovation.
Moving to this bespoke future will not be easy for most organizations. It will require a shift in mindset, but it will be worth the effort. First, enterprises must become data-driven at their core, embracing new approaches that foster creativity, shedding centralized practices that no longer work. Organizations will need to modernize legacy architectures. They must remove data siloes and the outdated mindset that IT is the custodian of insight.
The effort will be worth it. When data becomes a medium for creativity in an organization -- because the constraints have been lifted and teams can cooperate and work in a coordinated way -- business units will start to look at innovative and bespoke ways to problem solve.
The old expression "may you live in interesting times" certainly describes the post-pandemic world and its many challenges. However, with challenge comes opportunity. The time is now to draw the bespoke future through more creative and collaborative approaches to data. There has never been a better time to embrace this change.
About the Author
Rishi Diwan is chief product officer at Exasol where he is responsible for leading strategy, design, product management, and product marketing. Most of Rishi’s 25 years in enterprise software have been in and around data and analytics products at companies including Salesforce, Stats Perform, and SAP. You can reach the author via Twitter or LinkedIn.