By using tdwi.org website you agree to our use of cookies as described in our cookie policy. Learn More

TDWI Upside - Where Data Means Business

Changing Organizational Needs Are Driving Integration Changes

Enterprise spending on integration solutions continues to climb. What’s driving this growth and how is it impacting your organization?

When I started out in the integration industry, if you had told me that integration technology would still be a thing in 2024, I might have assumed that it would be commoditized and boring, and that I’d be doing something else.

For Further Reading:

5 Things to Look for in Your Next Data Integration Platform

Lack of Modern Data Integration Hindering Business Objectives, Survey Finds

Don't Let Data Integration Be the Downfall of Your Cloud Data Lake

The reality is, though, that despite the maturity of both the needs and solutions available, integration spending continues to project remarkable growth for the foreseeable future. Why all this growth for a set of technologies that have been in use for so long? The needs and the solutions are changing, and we are at an inflection point of how and why integration platforms are adopted.

Let’s take a closer look at the agents of change driving all this projected growth.

Extending Legacy Solutions Requires Modern Integration

Not surprisingly, many of the same dynamics that have driven the requirements for integration technologies over the years will continue. For example, one tongue-in-cheek answer is that SAP integrated with anything -- including SAP -- will need sophisticated integration. Any time an organization purchases and implements a major application suite, there's a requirement for integration technology.

Another factor is that, overwhelmingly, most of the integration technology deployed today was built for another era. Enterprise service bus (ESB) tools served the era of service-oriented architecture (SOA) well, but we left that era years ago. We need technologies that natively support cloud and microservices architectures for today’s and tomorrow’s needs. Add to this dynamic the impending flood of change coming with generative AI applications -- as my colleague Peter Kreslins writes about -- and the need gets more urgent.

An anecdote illustrates this point. No industry has ushered in more change in recent years than retail. The changes in buyer behavior and demands that have required new and rapidly evolving technology solutions are well documented. However, a technology leader from a very recognizable apparel company recently shared that their ERP system is SAP ECC running on an AS/400. It’s probably been in place for decades, but it works, so there is not a burning desire to replace it.

This leads me to believe that a continuing driver for integration will be organizations’ need to modernize their architectures and their portfolio. Whether the goal is to replace that ECC on AS/400 or to extend its usefulness, the integration will need to catch up with all the surrounding technologies of different eras. The question isn’t if but when.

As a result, I expect integration vendors will see substantially more demand than they have in recent years to integrate legacy technologies -- in place and well-functioning -- for new business outcomes.

The Changing Integration User

Another trend in our industry is the emergence of low-code tools built for citizen developers and line-of-business people to build their own integrations. This is clearly on the opposite end of the spectrum from those sophisticated use cases we discussed above.

Certainly, low-code tools are a great way to drive efficiency and productivity. They’re being used to democratize the use of integration technologies outside of IT organizations, which can be beneficial in many ways.

However, when considering all of this, the question arises: are the needs of general developers being neglected? The specialization required by designated integration developers to use legacy iPaaS platforms leads to a backlog of development projects waiting for integration work to be completed. These specialists simply can’t get to it all and front-end developers, full-stack developers, and application developers can either wait or write their own integration code.

Therefore, I expect to see the emergence of all developers using iPaaS technology to create integrations, not just the small number of integration developers or only non-technical citizen integrators. This requires solutions suitable for a community of developers that generally trust their own code over that produced by platforms. The organizations then benefit from a rapid reduction in development backlogs as well as integration infrastructures that are less reliant on code integrations and all that comes with maintaining them.

Experienced developers have immense pride of ownership in their code, and a low-code platform might not be the first tool they reach for, but this new approach introduces tools that are both robust enough to make developers want to use them and that incorporate the agility developers demand. Organizational benefits to this approach are immense, especially when proper governance is part of the operating paradigm of the development team.

Governing the New Developer Community

One immediate effect of increasing the community of developers as suggested about is the likelihood of technology sprawl, a type of technical debt that consists of unplanned versions and technologies adopted by organizations -- frequently with overlapping use cases or meant to solve the same problem. Technology sprawl is not always expected, but arises from all the quick, siloed technology decisions teams make to do the work they need to do.

Lines of business are now managing more enterprise applications than IT, and IT is struggling with managing this sprawl. These singular pieces of technology acquired to solve specific problems will provide more enterprise value when integrated with other systems. This dynamic is driving much of the growth in the application integration (iPaaS) market. However, these integrations generally lack IT oversight and technical expertise. They become burdensome and work suboptimally, forcing IT into “rescue” mode for systems they are not staffed to manage and support. Unless dealt with upfront, this problem will spiral, with AI driving even more adoption of ungoverned technologies. All of this will create security vulnerabilities as well. Governing citizen integrator–developed integrations will become a major resource commitment for IT.

About the Author

Matt Durham is the head of strategy at Digibee where he is responsible for company strategy and market awareness. You can reach the author via email or LinkedIn.


TDWI Membership

Accelerate Your Projects,
and Your Career

TDWI Members have access to exclusive research reports, publications, communities and training.

Individual, Student, and Team memberships available.