Sunday, May 02, 2021

The Biggest Challenge of Postmodern ERP - Cloud Integrations

The Biggest Challenge of Postmodern ERP - Cloud Integrations

Gartner originally coined the term “enterprise resource planning” (ERP) back in 1990 to describe a new breed of integrated software solutions. These solutions included modules for Material Requirements Planning (MRP), Manufacturer Resource Planning (MRP II), Financials, Human Resources (HR) and Customer Relationship Management (CRM).

During the 1990s and 2000s, these single monolithic ERP solutions became an essential part of most corporate IT strategies. But by the mid-2000s, such ERP solutions started to get a bad reputation. The very high price tag, the relative inflexibility of an integrated solution and the droves of ERP implementation failures that made headlines in both the tech and business world clearly showed the shortcomings of this strategy. It became hip to proclaim that “ERP is dead” or “the end is near”. 

Companies started implementing best-of-breed applications to replace individual modules within larger ERP solutions. For example, using Salesforce instead of the CRM module bundled with their ERP solution, or Workday to replace the HR module. Others ditched their ERP solution altogether and started using only best-of-breed applications and started connecting them with each other. 

Gartner coined the term “postmodern ERP” in 2014 to describe this new ERP strategy.

Postmodern ERP is a technology strategy that automates and links administrative and operational business capabilities (such as finance, HR, purchasing, manufacturing and distribution) with appropriate levels of integration that balance the benefits of vendor-delivered integration against business flexibility and agility. 

To put it another way, a traditional ERP solution is like the new car you buy every 10 years. A postmodern ERP solution is like owning the same car indefinitely, but with various components that can “easily” be changed out as needed.

Around the same time that postmodern ERP strategy became hip, we see the cloud strategy of both vendors and clients taking off. This means many of these best-of-breed applications are consumed in a SaaS model with all its advantages and disadvantages.

Whilst this all sounds like great ideas it gives us one very big challenge to solve; cloud integrations.

Why Cloud Integrations Are so Difficult

Cloud integration means integrating data used by disparate systems, between or within public or private clouds, or between cloud-based and on-premise systems. The goal is to create unified data stores that can be accessed efficiently and transparently by all relevant users and applications.

There are mature tools for data integration within public cloud providers or private cloud platforms, for example within AWS, Azure or an OpenStack data center. The main challenge begins when organizations need to integrate multiple public clouds, set up hybrid cloud environments, integrate legacy on-premise systems with cloud workloads, or lift and shift legacy workloads into the cloud.

You can increase flexibility and agility, or reduce complexity, but you can’t do both. Despite the rigid nature of traditional ERP solutions, they solved some key issues, including the need for consistency and integrity of data and processes. 

Let’s have a look at some aspects you have to deal with when implementing a postmodern ERP solution. 

Release Management
Each SaaS application that is part of your postmodern ERP solution will have a different maintenance and upgrade cycle. This means that you will have continuous change in parts of your end-to-end solution without you being able to stop this. This means you have to have a solid plan in place for regression testing. Does the end-to-end solution still work after each upgrade? Do your integrations still work? Are the data semantics still the same?

Environment Management
Tied to the previous point is environment management. In order to do development and testing you will need to have a development and test environment. This works a little differently for each SaaS application that is part of your postmodern ERP solution. And just as the production environment, they will all have different maintenance and update cycles, meaning it will be very difficult to have a stable environment for long.

Backup and Disaster Recovery
It is safe to assume that each SaaS application that is part of your postmodern ERP solution will have a solid backup and disaster recovery strategy in place. But they all will be different. Do you understand the impact of such an event on your integrations? On your data integrity? Did you test this?   

In hybrid cloud environments, when moving data between the cloud and on-premise systems, opening the firewall is a necessity that most network administrators anxiously resist. You will have to carefully evaluate the requirements for data integration flows that cross the firewall to find the solution that minimizes the exposure of enterprise systems and data.

Network Latency
Cloud environments are often preferred to on-premises because of their scalability: you can easily increase or decrease your usage of compute and storage resources in just a few minutes. But scaling your cloud environment will have limited effect if your network latency is too high, which puts a firm cap on the data integration workloads you can run.

Data Transfers
Moving data between clouds, and between cloud and on-premise systems, can be time-consuming and error-prone, or even unfeasible in some cases, depending on the data volumes and the required data transfer frequency. Cloud data integration won’t work without solid strategies to transfer data in a timely manner. There is a huge difference between 10 messages a second and a 100 -- trust me.

Data Integrity
How are you monitoring all points of integration and logging the data on the move? Depending on the method used, the process of monitoring and capturing data changes on source systems can increase the load on the source system causing slowdowns in operational systems.

Data Conversion
In traditional data integration projects, complex Extract Transform Load (ETL) workflows were set up to clean data and transform it into the precise format needed by target systems. Many SaaS applications work with unstructured data or provide a flexible data model for structured data. However, they still need data to be cleaned, treated and converted into the desired format. Integration strategies must consider how ETL can be performed without slowing down the integration or adding a lot of complexity.

Data Synchronisation
SaaS applications are often architected with scalability or performance in mind, not around data integration. For example, in a system that rapidly scales up or down, with data stored on dozens or hundreds of cloud instances, it may be challenging to synchronise with external systems.

There is no standard approach or protocol for integrating data between SaaS applications, not to mention cloud and on-premise systems. Each cloud platform, service or resource tends to have different data schemas and formats. Data connectors or adaptors need to be constantly updated, as new cloud services are introduced or as applications are updated or modified.

Security and Compliance
Consider the security requirements and industry standards applicable to each of your datasets. Both your integration platform and all connected applications need to fulfil those requirements, both for one-time data transfers and ongoing data synchronisation.

In a nutshell: You can increase flexibility and agility, or reduce complexity, but you can’t do both. 

Read more…