Lecture 1: Why We Integrate Systems

Before designing a single integration, an architect or developer must understand why integration is done at all. Integration is expensive, complex, and introduces risk. Organisations that do it well gain significant competitive and operational advantages. Organisations that do it poorly accumulate technical debt that can take years to undo. This lecture explains the business case for integration, the costs and risks involved, and where integration sits within enterprise architecture.

The Fundamental Problem

Most organisations have not grown their IT estate by design — they have grown it by accumulation. Over years and decades, systems are:

Purchased from different vendors to solve different problems
Built by different internal teams at different times
Acquired as part of mergers and acquisitions
Adopted as SaaS solutions to replace manual processes

Each system stores its own data, in its own format, using its own terminology. An employee might be a "resource" in the HR system, a "user" in the identity system, a "contact" in the CRM, and a "payee" in the payroll system. The same person, four different representations, four separate databases.

Without integration, the consequences are:

Manual re-keying of data between systems — error-prone and slow
Data inconsistency — the same fact has different values in different systems
Siloed decision-making — leaders make decisions based on incomplete information
Duplicated work — teams solve problems independently that a shared integration would solve once
Delayed processes — business workflows stall waiting for data to be transferred

Why Organisations Integrate Systems

1. Automate Business Processes

The most immediate driver. When a customer places an order, the following must happen:

Inventory levels must be checked and reserved
Payment must be authorised
A shipping label must be generated
The warehouse must be notified
The customer must receive a confirmation

Without integration, humans do these steps manually. With integration, they happen automatically, faster and without error.

2. Enable Real-Time Decision-Making

Business decisions require data. Without integration, data is locked in silos:

The marketing team cannot see which customers have open support tickets
The finance team cannot see real-time sales pipeline data
The operations team cannot see logistics and inventory together

Integration makes data available where and when decisions are being made.

3. Improve Customer Experience

Customers interact with one organisation, not with individual systems. When a customer contacts support, the agent should immediately see their account, purchase history, open orders, and recent interactions — regardless of which systems that data lives in. Integration makes this possible.

4. Support Digital Transformation

Digital transformation projects — migrating to the cloud, adopting new SaaS platforms, building mobile apps — all require integration. A new mobile app needs to read data from the ERP. A cloud data warehouse needs data from a dozen on-premises systems. Integration is the enabler of transformation, not the transformation itself.

5. Support Mergers and Acquisitions

When two companies merge, their IT estates must work together. Customers, products, employees, and financial data exist in both companies' systems. Integration bridges these estates without requiring an immediate full system replacement.

6. Regulatory and Compliance Reporting

Many industries require reporting to regulators — tax authorities, financial regulators, healthcare bodies. The data for these reports exists across multiple systems. Integration aggregates it into the required format and automates submission.

What Integration Requires

Integration is not just technical work. Successful integration requires:

Technical Requirements

Interface knowledge — understanding what interfaces (APIs, files, database connections) each system exposes
Data understanding — knowing the data models, formats, and quality characteristics of source systems
Integration platform or tooling — middleware, iPaaS, messaging infrastructure
Testing environments — staging and test systems that mirror production
Monitoring and operations — visibility into whether integrations are running correctly

Organisational Requirements

System ownership — clear ownership of each system's interface and SLA
Cross-team collaboration — integration projects always involve multiple teams; coordination is critical
Governance — standards for how integrations are built, documented, and changed
Change management — integration changes must be coordinated; changes to one system can break others
Budget — integration platforms, development, testing, and ongoing maintenance all cost money

Skills Requirements

Integration architects — to design the overall approach
Integration developers — to build and test
Data analysts — to understand and map data
Security specialists — to ensure integrations are secure
Operations engineers — to monitor and maintain live integrations

The Cost of Integration

Integration is expensive. Understanding why helps prioritise investments and set realistic expectations.

Development Cost

Building an integration involves more than writing code:

Requirements gathering and analysis
Interface investigation (reading API documentation, discovering undocumented behaviours)
Mapping and transformation design
Error handling design
Testing (unit, integration, system, performance)
Documentation

A "simple" integration between two well-documented systems can take two weeks. A complex legacy system integration can take months.

Ongoing Cost

Integrations are not build-once solutions:

Systems change — every upstream API change or data model change may break a dependent integration
Volume grows — an integration designed for 100 records per day may need to scale to 100,000
Business rules change — routing, transformation, and filtering logic must be updated
Platforms are upgraded — the integration middleware itself needs maintenance and upgrades

Cost of Poor Integration

The cost of badly designed integrations is often higher than the cost of doing it right:

Hidden failures — integrations that silently stop working, causing data to fall out of sync
Brittle connections — a change to one system breaks multiple integrations unexpectedly
Performance degradation — poorly designed integrations slow down the entire system landscape
Security incidents — unsecured integrations create attack vectors
Compliance failures — uncontrolled data flows violate data protection regulations

Integration Risks

Data Loss

If a message is not delivered and there is no error handling, data is silently lost. The downstream system never receives the update and the systems fall out of sync.

Data Corruption

Incorrect transformations or mapping errors can corrupt data in the target system. If this is not caught quickly, corrupted data can propagate across many systems.

Availability Coupling

When System A calls System B synchronously, System A's availability depends on System B's. If System B is down, System A's integrated functions fail too.

Security Exposure

Every integration is a potential attack surface. An unsecured API endpoint, an unencrypted message channel, or a service account with excessive permissions can expose sensitive data.

Compliance Violations

Integrations that move personal data across system or national boundaries may violate GDPR, HIPAA, or other regulations if not properly designed and documented.

Change Propagation Risk

A change to a system's interface can break every integration that depends on it. Without a catalogue of dependencies, organisations do not even know which integrations will be affected.

System Integration in Enterprise Architecture

Enterprise Architecture (EA) is the discipline of designing and managing the overall structure of an organisation's IT systems, processes, and data. Integration is a core concern within EA.

The Four EA Domains

Enterprise architecture frameworks (TOGAF, Zachman) typically address four domains:

| Domain | Focus | Integration Relevance | |--------|-------|----------------------| | Business Architecture | Processes, organisational structure, capabilities | Identifies which processes need data from which systems | | Information Architecture | Data models, master data, information flows | Defines what data needs to flow and in what format | | Application Architecture | Systems inventory, interfaces, dependencies | Maps which systems exist and what interfaces they expose | | Technology Architecture | Infrastructure, networks, platforms | Defines what integration platforms and protocols to use |

Integration architecture cuts across all four domains. It connects business processes to technical systems through controlled, governed data flows.

Integration and Business Strategy

Integration decisions are business decisions, not just technical ones:

Which systems are integrated determines what data is available for decision-making
How quickly data flows determines how real-time the organisation's operational picture is
How reliably integrations run determines the resilience of business processes
How governed integrations are determines the organisation's ability to change and comply with regulations

An Integration Architect must understand both the technology and the business context well enough to translate business requirements into integration designs that serve strategic goals.

Integration as a Shared Capability

The most mature organisations treat integration not as a series of one-off projects but as a shared organisational capability — a practice, platform, and set of standards that all systems use.

Signs of a mature integration capability:

A central integration platform that all systems connect through
A registry of all active integrations with ownership and SLA information
Standards for how new integrations are designed, tested, and deployed
An operations team that monitors integration health
A governance process for reviewing and approving new integration requests

Signs of an immature integration capability:

Point-to-point connections between systems, each built differently
No central record of what integrates with what
Each team builds integrations independently with no shared standards
Nobody knows what will break when a system is changed

Lecture 1 Summary

Organisations integrate systems to automate processes, enable decision-making, improve customer experience, support transformation, and comply with regulations.
Integration requires technical resources, organisational coordination, governance, and ongoing maintenance — it is never a one-time project.
Integration creates risks: data loss, corruption, availability coupling, security exposure, compliance violations, and change propagation. Each must be actively managed.
System integration sits at the heart of enterprise architecture, connecting business processes to technical systems across all four EA domains.
Mature organisations build integration as a shared capability with platforms, standards, and governance — not as a collection of independent point-to-point connections.

Next: Lecture 2 — Integration Styles and Types