Executive Q&A: Research into the State of Digital Architects, Post-Pandemic
For many years, architects have struggled to satisfy their organizations’' needs for digital change. We spoke with Chin-Heng Hong, VP product management at Couchbase, about the new challenges architects face almost 20 months into the pandemic.
- By James E. Powell
- November 8, 2021
For many years, architects have struggled to satisfy their organizations' needs for digital change. Chin-Heng Hong, VP of product management at Couchbase, spoke to us about insights gleaned from a Couchbase survey about the drivers of digital transformation projects, why they fail (and the consequences of that failure), and what's needed to avoid disruptions and failures.
Upside: What does digital transformation mean as your research defines it?
Chin-Heng Hong: In the context of our research, digital transformation refers to the process of implementing new digital technologies that will enable organizations to improve business processes, enhance customer experience, encourage innovation, and ultimately operate as effectively and efficiently as possible while maintaining and advancing the ability to react rapidly to changing business conditions and market requirements.
With digital transformation near the top of CIOs' agendas for some years, many architects have struggled to satisfy their organizations' needs for digital change. Successful digital transformation requires the right technology to support evolving architectures, yet many architects face hurdles in adopting the tools they need to position their organizations for future success. To examine this further, the Couchbase team conducted a survey of 450 digital architects to look into the pandemic's profound effect on these efforts.
In the last five years, what were the primary drivers of digital transformation projects, and what were the expected benefits?
There have been multiple drivers fueling digital transformation, but perhaps none so pressing as the explosion of data and the need for highly interactive and personalized mobile applications that are always on no matter where they are. Big data has put enterprise legacy systems under increasing pressure as they struggle to manage not only massive volumes of data but also the variety of data (including semistructured data generated by users and devices) in a way that enables them to rapidly, reliably, and effectively extract intelligence to gain a competitive edge.
Ensuring a positive customer experience is critical. The "big data imperative," if you like, coupled with the race to the cloud and the growing need to push applications ever closer to the data sources at the network's edge, have undoubtedly been among the most significant drivers of digital transformation projects in the past five years.
As a result, many organizations have moved away from legacy databases to more modern NoSQL offerings and are seeing accelerated benefits. It's not just the shift from relational databases to NoSQL offerings that is underpinning the modernization of enterprise architectures. How databases are deployed is also important. Our research found that 80 percent of respondents said moving to cloud deployment models have helped them meet their digital transformation goals, and 41 percent said it has been "significant" or "indispensable." This could suggest that architects would have more time to focus on optimizing other business processes that impact the customer experience rather than spending time navigating the limitations of legacy offerings that simply cannot meet the demands of today's modern enterprises.
COVID-19's impact on digital transformation has been dramatic, more than doubling the pressure on digital architects. Despite these challenges, many enterprises have still pushed ahead with digital transformation (46 percent of architects indicate that they're on time with their digital plans). Rightfully so, given that this enables organizations to adapt to an increasingly online world and implement new technologies more quickly, such as edge computing, artificial intelligence, virtual reality, microservices, and more.
What factors have caused active digital projects to fail, suffer a significant delay, and/or be scaled back in the last 12 months? How has COVID-19 contributed to this?
In 2019, we found that many organizations were well on the way to executing digital transformation -- with 78 percent no longer in the planning phase but actively working to deliver digital projects. However, continuing this progress during the pandemic has been a serious challenge. From arranging remote working logistics to dealing with huge spikes in demand for online services, the pandemic has caused many enterprises to urgently revisit their digital transformation plans. Factors that have caused digital projects to fail or be delayed included limited resources to retrain staff, high ongoing and upfront technology costs, and prioritizing reacting to the COVID-19 pandemic.
According to our research, almost half of digital architects are currently under high or extremely high pressure to deliver digital projects compared to just 19 percent pre-pandemic. During a time of unpredictable demand and change, many architects may have had to delay larger digital transformation projects to ensure that new daily processes were set up to handle the influx of remote and/or hybrid working models. Despite these challenges, architects continue to face the responsibility to ensure IT has the technology and guidance to "keep the lights on" and keep wider digital progress running smoothly.
How does a failed or disrupted digital transformation project impact an enterprise's overall strategy goals?
Broadly speaking, the effects of a failed digital transformation project can be far reaching for an enterprise given that the very goal of such transformation is to enable the business to streamline its processes, better serve its customers, and position it to react better, faster, and more effectively to changing conditions. The inability to successfully achieve any of these goals can have effects that range from inconvenience for IT and developer teams to permanently impacting customers' perception of a brand.
The outbreak of COVID-19 hasn't just required a reassessment of existing digital transformation plans. It also caused problems implementing the technologies behind them. Unfortunately, the vast majority of architects (86 percent) say COVID-19 has affected their attempts to get the right technology in place for digital transformation, and 43 percent say it has made it "significantly more difficult" or even "insurmountable."
What are the potential consequences of a failure to digitally innovate?
By avoiding digital innovation, organizations harm their ability to build customer loyalty and meet their ever-evolving preferences, scale the business, and ultimately compete in the market. In addition, it makes it more difficult for architects to do their job because they will have to spend more time relying on technology that no longer supports evolving architectures. Architects will also have to spend more time troubleshooting subsequent issues.
Furthermore, by not digitally innovating, organizations find it much more challenging to implement new projects, and this problem is exacerbated by a continued overreliance on legacy databases. Specifically, our research found that 61 percent of organizations that still rely on legacy databases find it very challenging to pursue digital projects. Despite most organizations believing relational databases do not have major potential to revolutionize digital transformation, 91 percent still rely on them -- 31 percent heavily.
What's needed, and/or what steps can digital architects take, to avoid disruptions and failures and ultimately improve the end-user experience?
First is picking the right technologies to address the requirements of digital transformation projects. By learning from decisions that have resulted in success, or even failure, organizations can build a blueprint to ensure future business success.
I think the answer to what will ultimately improve the end-user experience can be discerned by looking at where architects believe the future lies. The research clearly illuminates their beliefs in this regard. Respondents said that the cloud (identified by 68 percent) and big data (59 percent) have the most potential to revolutionize digital transformation -- perhaps because both enable organizations to adapt to an increasingly online and decentralized world to better engage with customers.
However, "plug and play" technology can't deliver returns without teething problems, and it is the way in which architects navigate such challenges that will prove crucial as we head into 2022. As we close out the year, emerging stronger from COVID-19 will require careful consideration of what went wrong, what went right, and what architects truly need to continue driving digital transformation success.
Who measures the progress of these improvements, and what metrics are being used?
What ultimate success, if indeed there is such a thing, looks like and the metrics by which it will be measured will of course be unique to each enterprise. I don't believe digital transformation is so much a destination as it is a journey. Like all journeys, the successful traveler is the one who remains open and willing to reroute at a moment's notice to take advantage of, or circumvent changing traveling conditions. If there's a measure of success, then it must surely be linked to the ability of the technology being put in place to keep the business agile, competitive, and profitable.
If you look at the high-level requirements for digital transformation, organizations can define metrics around the following:
- Can users expect web and mobile applications to be available globally, 24x7, without any downtime?
- Can users expect web and mobile applications to be highly personalized and highly responsive?
- Can enterprises roll out updates to their applications quickly to meet evolving user needs?
- Can enterprises scale out the infrastructure instantly as demand spikes (e.g., the launch of a new TV drama, a new game, or black Friday sales)?
[Editor's note: Chin-Heng Hong is VP of product management at Couchbase, a modern database for enterprise applications. A senior engineering executive with more than 25 years of experience, Hong previously held roles at Hewlett Packard, Oracle, Siebel Systems, and others. You can contact him via email, on Twitter, or LinkedIn.]