A Chaos of Connectivity is Putting Businesses At Risk

We spoke to 1,500 IT decision makers across the globe in our annual Reality Check to understand their technology challenges. What we found, in a single word: Chaos. (In three words: Chaos of Connectivity.) Companies continue to invest in new tech, but often in silos and, regrettably, without an integration plan in place.

Over the last few years, 86% of businesses have substantially grown their tech stack, but half of them have done it without a clear integration plan. This has caused a Chaos of Connectivity.  

The Software AG research found that three quarters of organizations agree that enterprise technology has become more chaotic over the last few years. This creates significant challenges—not only in terms of managing the technology itself, but in the impact it has on the business. It increases risk in a number of areas, including security, compliance, productivity, decision making and competitiveness.

While this chaos is significant, and creates a number of risk factors for businesses, the good news is that these technology challenges can be overcome. Organizations can manage the Chaos of Connectivity. But it’s important to understand where their technology chaos comes from and how it mounts up, to have an effective plan.

There was a sharp acceleration of investment in digital transformation initiatives in 2020. Our previous research showed that companies started to see more technical debt as a consequence of that investment during 2021. Technical debt, defined here as the build up of “post-go-live" recoding and technical maintenance, can leave a lot of necessary integration, security and functional development work left in a holding pattern. When that happens, chaos starts to take hold.

On top of this, proliferation of enterprise technology solutions in just the last year has been enormous, thanks in large part to Generative AI. While gen AI has the potential to drive massive business outcomes, it also adds enormous complexity to the enterprise integration landscape, raising concerns about privacy, security, resiliency, and compliance. But it is the perfect example of how technology introduces new complexity

If we look at OpenAI: GPT-1 worked from 117 million parameters. GPT-2 expanded to 1.5 billion; GPT-3 to 175 billion and now GPT-4 to a reported 1.7 trillion. Of course, not every business is running systems that are this data hungry—but all technology is expanding. When technology expands quickly and there is a lot of hype about its benefits, businesses face a choice: keep up or be left behind.

In the case of gen AI, organizations need to invest today because its potential is just too massive to wait. ServiceNow CEO Bill McDermott told Fortune that this is the “iPhone moment” for the enterprise and that he “can’t think of a bigger risk than being a ‘fast follower’ in adopting generative AI.”

Goldman Sachs estimates that AI investment growth doubled from 2020 ($40.16bn) to 2023 ($110.19bn)—with significant growth expected by 2025 ($158.41bn). But as our primary research shows, many are investing without a plan to bring it all together. Integration will be the connective tissue that unifies data sources and feeds AI. When technology has the potential to affect the whole business—as many do—they need to be integrated in a way that helps the whole business. With each new system added and with each new investment made, the risks of extending the technology chaos increase.

Take, for example, a manufacturing company that introduces new connected machinery on its factory floors. That information can be used by the machine engineers to monitor output, of course. But if that information is effectively integrated into other business processes, then it can be used by purchasing teams to monitor resource usage, by customer relationship teams to monitor delivery timelines, by operations teams to monitor machine maintenance schedules—and potentially more.

We’ve only scratched the surface of the Chaos of Connectivity in enterprises. This report will pinpoint its root causes, and the havoc they create for organizations. With this understanding, integrated enterprises can see through the chaos to become more agile, more productive and protected by stronger governance.

The vast majority (86%) of companies have seen their tech stack expand substantially in the last few years. Nearly three quarters (70%) have accrued more technical debt in the last 12 months than they have in previous years and almost half (42%) have invested heavily without a clear IT integration plan.

The acceleration of tech investment prompted by the global pandemic was a second wind for digital transformation. As companies were forced to do more in a remote model, they had to supercharge their efforts to overcome technology challenges with innovative IT strategies. For example, spinning up new ecommerce sites—perhaps a six-month project in normal circumstances—had to happen in just a matter of weeks. Many elements of that mentality still remain. Systems and applications that go live at a minimum viable product stage have the advantage of speed, but also require frequent maintenance, development and recoding later—i.e. technical debt management. In the case of an ecommerce spin-up, detailed requirements processes might be skipped over with the goal that they’re filled in later. This is not a problem when organizations have the resources to do it, but can be a cause of technology chaos because they frequently don’t.

As new systems become more widely used across the business it’s key they’re integrated effectively—whether for compliance, productivity or simply sharing valuable information. That work can be time-consuming and complex, and current Integration Platform as a Service (iPaaS) tools struggle with the new demands placed upon them. Companies that have perhaps gone live with a new ecommerce site in record time could be missing out on key connections between ordering and customer history systems, or between transactions and replenishment.

This technology challenge is ongoing and, in fact, getting greater. The majority of companies say that the number of disparate applications and systems has grown compared to two years ago—and will grow again in the next two years. Therefore, the debt and the chaos created by new technology will only subside with a very deliberate change in how these systems are managed.

A new approach for new times

The conflict between business and IT is exacerbating the problem, with nearly half (44%) of organizations saying that this divide is getting worse. This is nothing new, of course, as organizations have faced challenges with citizen developers and shadow IT for many years. However, generative AI has suddenly leveled up the technical skills of everyone in the organization overnight. Anyone with internet access has the ability to develop something new. Most employees have a greater awareness of the power of technology, if not the skills and experience. This makes relationships between line of business users and IT risk even more fractured.

Consider the pressures on all parts of a business—be it for more sales, faster product development or simply improved efficiency—and you can understand why more and more teams want to move quickly. If lines of business don’t get what they need from IT immediately, they start doing things themselves.

There are other digital transformation challenges contributing to this technology chaos. For example, nearly half of organizations (44%) have multiple vendors for their different integration needs. This means they need to integrate their integration tools. Something that gets worse in the context of application sprawl. More than two thirds (69%) of organizations have a higher number of disparate applications now compared to two years ago, with even more (71%) believing it will increase even more in the near future. There’s no slowdown in sight. Times are changing and so must technology strategy.

The reasons for the growing chaos of connectivity are myriad. While some of the environmental factors may not change, organizations can look at the consequences of these changes on their businesses and come up with ways to reverse the trend towards chaos in the future.

The technology challenges created by complexity and the chaos that can result will be felt throughout the business. However, the bigger concern is that there could be worse to come because of the risks created by the Chaos of Connectivity. If the growth of technology continues without a new approach to managing and integrating it, then there could be commercial, compliance and business problems.

Slipping into non-compliance

One of the greatest threats of the Chaos of Connectivity is non-compliance with any number of regulations or standards. Almost two thirds (65%) of companies  have governance issues that are made worse by technological complexity.

There are many possible reasons for this, but they all stem from the same problem: a disconnect between the necessary systems and data to maintain and/or demonstrate compliance. Eight in ten companies would define the majority of their systems as “legacy.” Nearly half of companies (44%) find managing new and legacy systems together increasingly complex. Highly regulated industries in particular—like governments or financial services—have significant investments and business-critical information in mainframes and other more commonly “closed” systems. While the stability and security of this setup is usually a huge positive, it’s not always true when it comes to issues such as data regulation or operational resilience. Here it can be hard to prove that data or processes are compliant without any tools that unite all systems.

Generative AI is an even bigger potential black box architecture. Adhering to data sovereignty laws when you don’t even know exactly how your data is being transmitted and to whom is only getting harder. It becomes essential to find ways of figuring out which of your business users have hopped on the AI-train on their own without any signs of implementing proper governance. To overcome this technology challenge, you need tools to help spot if critical data, strategies, and reports are being fed to unsecured AI plugins. Beyond that, it’s important to ensure that key decisions are being made on reliable information and that security is not impacted by any flaws in AI-influenced decision making.

Losing competitiveness

Every organization is under pressure to be more competitive. This often means being quick to act, whether in product development, customer service or simply in financial terms. However, 80% feel that complex technology can actually make them slower to launch new products/services, improve customer/employee experiences and increase revenue/profitability. The Chaos of Connectivity is putting key business performance at risk.

Decision-making can stall, which is ironic considering data and analytics promises to improve decision making. If businesses don’t integrate all of their data, their analysis is incomplete. This means that they either can’t act or have to make poorly informed decisions. Half of companies (47%) believe that key data is simply inaccessible when they need it. A similar number (46%) feel that decision making is slower because data is hard to move out of legacy systems.

In a perfect world, all data should be available in a few clicks. Sadly, it’s not, and organizations miss opportunities to change, to move into new markets and to win new customers. In the long run this can erode competitiveness in a way that’s hard to recover.

Losing revenue

While there’s a risk to a company’s long-term strategy and market position, there are also immediate risks to businesses in the form of disruption and stoppages that reduce revenues. Nine in ten (89%) have experienced disruption or been forced to shut down services because of technology issues. 22% say it happens often.

There is an immediate impact on sales and other transactions when they simply can’t be processed. Technology does crash from time to time. But understanding why service stoppages occur is a key part of managing the risk of further disruption. Half of companies (51%) have seen disruption caused by overloaded apps unable to respond and shutdowns resulting from compliance problems with public-facing services.

Being unproductive and wasteful

Almost half of companies (45%) experience conflict in decision making resulting from duplicated processes—caused by information and/or operational siloes. It’s easy to see how this can happen as teams throughout the business are becoming more empowered by technology. They create their own ways of working, which create their own data, and they make their own decisions. But it’s all being done in isolation because there’s no connection to the rest of the business.

This means that companies are busy doing lots of (often good) things, but when it comes to crunch time, it’s hard to take the final step. Disagreements occur late in the process—instead of coordinating at the beginning—because there isn’t a central point of visibility and management of all these systems to alert teams to the existence of others working on similar things. For example, a single, central API management tool could enable business leaders to see every department that’s tapped into certain applications or data to ensure collaboration on what that information is used for.

Productivity issues can be compounded by technology intended to solve those same issues. Just under half (44%) use multiple vendors for their different integration needs. This is likely where organizations need to integrate data, applications, B2B platforms, APIs and event streaming, but today’s iPaaS solutions cannot cover them all in one platform. And of those that do have multiple vendors, the majority have higher license and maintenance costs (58%) and find it harder to enforce standards/best practices (51%).

Security

Cyber security around digital transformation is an obvious and visceral risk that companies face. As technology infrastructures sprawl, they create more potential attack vectors in an environment where it’s harder to monitor them. This trend is not slowing down, because organizations still need to roll out new products and services. Some steps, such as centrally managing all APIs to ensure consistent standards and effective monitoring can help.

However, there are also other security elements in play, in particular around business continuity. The pace of technology change is outstripping the development of suitable skills in nearly half (46%) of organizations. Not only does this make it a challenge to keep up with the technology chaos in today’s enterprise, but it also suggests that the problem will get worse in the future unless IT teams take immediate action.

The threat doesn’t only come from failure to keep up with new technology. Some businesses are seeing a growing risk of not being able to keep up with old technology. As workers skilled in older coding languages retire from the workforce—or simply move to new roles or different organizations—businesses will find it harder to maintain key integrations and other fundamental elements of key systems.

Risky business

There are a number of problems facing businesses as a result of disconnected data, disconnected systems, unclear management tools, disparate skills and siloed systems. However, things that are disconnected can surely be connected; things that are disparate can surely be united with the right approach to integration.

Companies are exposing themselves to new and growing risks simply by continuing to invest in technology. However, the benefits of new tech outweigh—or at least should outweigh—the risks.

The Chaos of Connectivity requires organizations to look at their strategy for managing technology in a new way—with a clear integration plan in place. When they do, they will see improvement in their agility, productivity and governance—though it’s easier said than done. The solutions available to integrate the increasingly disparate and rapidly expanding technology landscapes that characterize today’s businesses simply don’t cut it. A new category of integration technology is needed.

So how and where can companies look to improve the key areas of agility, productivity and governance?

Agility

Eight in ten (80%) companies feel that the size of their tech landscape makes them less agile. They feel they cannot move quickly to adapt to the technology landscape or to market opportunities. Unfortunately, behind the scenes this is a downward spiral. On one hand, the chaos makes it harder to make sense of the tech landscape, which is slowing things down. And the only way to manage that seems to be to enforce a very specific set of standards or processes for future development. On the other hand, you see teams and departments build their own tools, launch faster and circumvent IT backlogs. The long-term outcome of these opposing trends is only more technology chaos.

This is exemplified by pockets of integration developing within a business, which happens in 86% of organizations. They end up with many different platforms, teams and lines of code living across multiple clouds, systems and geographies. This is a problem because most companies affected by a Chaos of Connectivity don’t have the tools to manage it—or often even know about it.

A unified platform across data and application integration creates a seamless flow of data so you can make decisions using your data and put them into action immediately. This means that even if there are different pockets of integration, or “siloes” of activity, they are not operating independently or creating problems at the time when action is needed.

With this in place, freedom can be given to develop integrations using any tooling that individuals or teams prefer to use. Isolated pockets of integrations become a thing of the past, but not because of tighter control. In fact, developers can be given freedom, allowing them to deploy in any cloud or region in a click.
 

Productivity

All businesses are under pressure to do more, often with less. The answer in these situations is to get more done together, with less work. Eight in ten (81%) businesses know that they have duplicate integrations because they have siloed working environments. Imagine these teams having more visibility of what the others are doing. Not only will they need to do less themselves, but the value of the work done by the one team grows because they build something for everyone to use.

A unified integration platform means that teams don’t need to learn as much, but can collaborate more than ever. They can build applications and experiences and deploy them as APIs in a single motion.

This freedom to operate and collaborate in a more efficient way leads to productivity in other ways too. It can bring business technologists into the fold a lot more productively, especially coupled with generative AI. They can automate workflows themselves and boost their own productivity. Instead of creating more technology chaos, these new integrations can be managed centrally. This helps the 82% of businesses for whom shadow IT is a problem. It also helps to resolve the conflicts between IT and Lines of Business that characterize app deployment in 80% of organizations. Ultimately it makes the whole business work more effectively.

Governance

Governance is a topic that covers a very broad range of issues and risks. Data regulations globally are a major consideration, as well as specific industry standards or pieces of legislation that must be adhered to. All of these can cause business leaders to lose sleep. And while the complexities of legislation can vary, the core of the solution is common to almost all of them: visibility and management of data and technology boosts compliance.

Eight in ten (81%) say that a major pain point is not having a clear view/management of all integrated systems. This makes it difficult, if not impossible, to get a clear idea of status. These organizations need a whole new set of tools to help with better governance.

Centralized control and distributed execution means that integrations can run anywhere but with a central point of governance. Coupled with a single pane of glass across these integrations, APIs and data pipelines, organizations suddenly have end-to-end visibility across all transactions.

A unified experience for all integrations delivers complete observability and offers compliance without chaos.

Simply put, the Chaos of Connectivity makes it hard for businesses to capitalize on the value trapped in their data. Of course, no one chooses to not share data, or refuses to collaborate or unify their technology. You only do it if you’re stuck.

You may have invested in feature-rich integration suites, but you’ve found that they’re so complicated that only your IT team or data engineers can use them. So you have high demand for your integration experts, but not the supply to meet it.

Alternatively, you may have invested in an iPaaS as a point solution for business teams who don’t have time to wait in the IT queue. You let them create the integrations and automations they need, but unregulated, this can result in an uncoordinated web of systems, the resulting complexity of it leaves you feeling like you’re “too big to innovate”. Too busy making sense of what has been created, and unable to use the new data being created to help make improvements across the business.

webMethods and StreamSets, which together form Software AG’s hybrid enterprise integration platform introduces a whole new category of integration called a Super iPaaS. It is a unified interface for all of your integration patterns—data, application, API, event streaming and B2B—across all geographies, hybrid multi-cloud hosting environments, user personas, and teams.

Instead of integration chaos you have one single integration platform that can be used by everyone across your enterprise, and managed centrally. Which means you can rapidly adapt to your technology needs; answer business requests quickly; and have exacting control over every integration across your business.

Connect your entire enterprise. Manage the chaos.