Lecture 2: Integration Styles and Types

Before choosing a technology or platform, you need to choose an integration style. The style determines the fundamental mechanism of data exchange. The type describes the purpose and scope of the integration. Understanding both allows you to make principled decisions rather than defaulting to whatever is familiar or easiest.

Integration Styles

An integration style describes the mechanism by which systems exchange data. There are five fundamental styles. They are technology-agnostic — each can be implemented using many different tools and platforms.

Style 1: File Transfer

System A writes data to a file. System B reads the file. The file is the medium of exchange.

System A ──► writes ──► File (CSV, XML, JSON, EDI)
                             ▲
System B ──────────────── reads

Characteristics:

• Inherently batch — data flows in chunks, not continuously
• Asynchronous — the writer and reader do not need to be active at the same time
• Loosely coupled — systems are only connected through the agreed file format
• Simple — almost every system can write and read files

When to use:

• Bulk data loads and batch processing
• Legacy system integration where no API exists
• Data warehouse feeds (nightly extracts)
• EDI (Electronic Data Interchange) between business partners
• Regulatory reporting (generate file, submit to regulator)

Risks:

• No real-time capability — data is only as fresh as the last file transfer
• File delivery failure can go unnoticed without monitoring
• File corruption or partial writes can produce bad data
• File format mismatches between sender and receiver

Examples in practice:

• Bank transaction files sent overnight between banks
• Payroll data exported from HR system and imported into payroll system
• E-commerce orders exported as CSV for warehouse picking system

Style 2: Shared Database

Two or more systems read and write to the same database. The database is the integration point.

System A ──► reads/writes ──►
                              Shared Database
System B ──► reads/writes ──►

Characteristics:

• Tightly coupled — both systems depend on the same schema
• Synchronous — changes are immediately visible to all systems
• Removes the need for a separate data transfer mechanism

When to use:

• Systems within the same technical domain, owned by the same team
• Read replicas for reporting (one write database, multiple readers)
• Legacy integration when no other interface exists and modification is not possible

When NOT to use:

• Systems owned by different teams — schema changes must be coordinated across all teams
• Systems with different technology stacks or deployment cycles
• Any case where loose coupling is a requirement

Risks:

• Schema changes in the database break all connected systems simultaneously
• No clear ownership of the shared schema
• Performance contention — high-volume writes from System A slow reads for System B
• Creates a hidden dependency that is hard to untangle later

Style 3: Remote Procedure Call (RPC) / Request-Response

System A calls an operation exposed by System B and waits for the result.

System A ──► call: getCustomer(id=99) ──► System B
        ◄─── result: { name, email, ... } ◄───

Characteristics:

• Synchronous — the caller waits for a response
• Tightly coupled in time — both systems must be running simultaneously
• Direct — one system directly invokes another

Modern implementations: REST/HTTP, gRPC, SOAP web services.

When to use:

• The caller needs the result before it can continue (real-time query)
• The operation must succeed or fail atomically from the caller's perspective
• Low-latency, request-volume scenarios (user-facing APIs)

When NOT to use:

• The operation takes a long time (seconds to minutes)
• High availability is more important than immediacy
• Fan-out to multiple receivers is needed

Style 4: Messaging

System A sends a message to a broker. System B retrieves the message from the broker when ready.

System A ──► message ──► [Broker / Queue] ──► System B
                         (stores message)

Characteristics:

• Asynchronous — producer and consumer operate independently
• Decoupled — producer does not know the consumer's address or state
• Resilient — the broker absorbs failures and pace differences
• Supports both point-to-point (queue) and publish-subscribe (topic) topologies

When to use:

• Producer and consumer have different availability or throughput characteristics
• High reliability is needed (messages must not be lost if a system is down)
• Multiple consumers need the same message
• Workload distribution across consumer instances

Style 5: Event-Driven Architecture (EDA)

Systems emit events (facts about what happened) and other systems react. No central coordinator.

Order Service ──► event: OrderPlaced ──► Inventory Service (reacts)
                                     ──► Billing Service (reacts)
                                     ──► Analytics (reacts)

Characteristics:

• Asynchronous and loosely coupled — producers and consumers are fully independent
• Reactive — systems respond to things that happen rather than polling or waiting for calls
• Extensible — new consumers can be added without changing producers

When to use:

• Real-time notifications and triggers
• Microservice communication where services must not call each other directly
• Audit trails and data pipelines
• Fan-out — one event triggers many independent reactions

Comparing the Five Styles

| Style | Coupling | Direction | Latency | Reliability | |-------|----------|-----------|---------|-------------| | File Transfer | Low | Batch | High (minutes/hours) | Medium | | Shared Database | Very High | Any | Low | High (within DB) | | RPC / Request-Response | High (temporal) | Request → Response | Very Low | Depends on availability | | Messaging | Low | Async | Low-Medium | High (broker guarantees) | | Event-Driven | Very Low | Broadcast | Low-Medium | High (broker guarantees) |

Integration Types

Integration types describe the purpose and scope of the integration — what is being connected and why.

Data Integration

Purpose: move data from one system to another to make it available where needed.

Subtypes:

ETL (Extract, Transform, Load) — extract data from a source, transform it into the target format, load it into the destination. Classic approach for data warehouses.

Source System → Extract → Transform (cleanse, map, aggregate) → Load → Data Warehouse

ELT (Extract, Load, Transform) — load raw data first, then transform inside the destination. Common with modern cloud data platforms (Snowflake, BigQuery) where transformation happens in SQL.

Data replication — continuously copy data from one system to another in near-real-time using Change Data Capture (CDC). The source database's transaction log is monitored; every insert, update, and delete is replicated to the target.

Data synchronisation — bidirectional sync where changes in either system are reflected in the other. Requires conflict resolution strategy (which system wins when both update the same record).

Application Integration

Purpose: connect applications so they can share functionality and data, enabling end-to-end business processes to span multiple systems.

Examples:

• CRM sends a new opportunity to the ERP to create a quote
• When an invoice is approved in the finance system, the payment is triggered in the banking system
• New user registration in the web app creates accounts in the identity system, HR system, and email platform

Application integration is typically real-time or near-real-time and uses APIs or messaging.

Process Integration

Purpose: orchestrate multi-step business processes that span multiple systems and potentially involve human approvals.

Example: Purchase Order Approval Process

1. Employee creates purchase request in HR system
2. Integration sends approval request to manager (email + portal)
3. Manager approves → Integration notifies procurement system
4. Procurement system creates PO → Integration updates finance system
5. Goods received → Integration closes PO and triggers payment

Process integration typically uses a workflow engine or Business Process Management (BPM) platform that can model the sequence, handle approvals, manage exceptions, and track state.

B2B (Business-to-Business) Integration

Purpose: exchange data and trigger processes between separate organisations.

Examples:

• Retailer sends purchase orders to supplier; supplier sends invoices back
• Insurance company exchanges claims data with healthcare provider
• Financial institution sends transactions to a clearing house

B2B integration adds additional concerns:

• Standards compliance — EDI (ANSI X12, EDIFACT), cXML, RosettaNet, HL7, SWIFT
• Security — mutual authentication, non-repudiation, encrypted channels (AS2, SFTP)
• Legal agreements — SLAs, data sharing agreements, liability

B2B integration is often handled by a dedicated B2B gateway or EDI platform rather than internal middleware.

API-Led Integration

A modern integration approach, particularly associated with MuleSoft's architectural model. It organises integrations into three layers of APIs:

┌─────────────────────────────────────────────┐
│  Experience APIs (mobile, web, partner)      │
│  — tailored to each consumer's needs        │
├─────────────────────────────────────────────┤
│  Process APIs (business process logic)       │
│  — orchestrate system APIs into workflows   │
├─────────────────────────────────────────────┤
│  System APIs (unlock backend systems)        │
│  — wrap ERP, CRM, databases with APIs       │
└─────────────────────────────────────────────┘

The benefit: reusability. System APIs are built once and reused by multiple Process APIs. Process APIs are reused by multiple Experience APIs.

Choosing an Integration Style: Decision Framework

Is the integration triggered by a user action needing immediate feedback?
  Yes → RPC / Request-Response (REST, gRPC)
  No  → continue

Is the operation a one-time bulk transfer or batch job?
  Yes → File Transfer or ETL/ELT
  No  → continue

Do multiple independent systems need the same data?
  Yes → Event-Driven / Pub-Sub
  No  → continue

Must the message be reliably delivered even if the receiver is temporarily down?
  Yes → Messaging (queue-based)
  No  → continue

Are you integrating systems within the same tightly controlled team boundary?
  Yes → Shared Database (with caution)
  No  → Messaging or File Transfer

Lecture 2 Summary

• The five fundamental integration styles — File Transfer, Shared Database, RPC, Messaging, and Event-Driven — each make different coupling, latency, and reliability trade-offs.
• Shared Database is the most tightly coupled style; Event-Driven is the most loosely coupled.
• Integration types describe the purpose: Data Integration (move data), Application Integration (connect apps), Process Integration (orchestrate workflows), B2B Integration (cross-organisation exchange).
• API-led integration organises APIs into system, process, and experience layers for reusability.
• Use the decision framework: if the caller needs an immediate answer, use RPC; if you need reliable async delivery, use messaging; if multiple consumers need the same event, use EDA.

Next: Lecture 3 — What Are APIs and How Are They Used?