Module 6: Integration Testing and Deployment

Integration testing is significantly harder than application testing. You are testing flows that cross system boundaries, involve asynchronous messaging, and depend on external systems that may have their own failure modes. This module defines how to test integration solutions rigorously and deploy them reliably.

Testing Strategies for Integration Solutions

Why Integration Testing Is Different

Application unit tests are fast, isolated, and deterministic. Integration tests are none of these:

• They depend on external systems (message brokers, databases, APIs) that may be unavailable or slow
• They involve asynchronous flows where the result arrives seconds or minutes after the trigger
• They can produce side effects that affect other tests (a test that creates an order may affect inventory levels)
• They require realistic test data that exercises all code paths

This does not mean integration testing is optional — it means it must be planned and resourced explicitly.

Test Strategy Pyramid for Integration

                         ┌─────────────┐
                         │  End-to-End │  Few, slow, costly
                         │    Tests    │
                    ┌────┴─────────────┴────┐
                    │   Integration Tests   │  Moderate number
                    │  (contract, API, SIT) │
               ┌────┴───────────────────────┴────┐
               │          Unit Tests              │  Many, fast, cheap
               │  (transformations, rules, maps)  │
               └──────────────────────────────────┘

Invest heavily in the base (unit tests for transformation logic), build a solid middle layer (integration tests for each interface), and use end-to-end tests sparingly for the most critical flows.

Unit Testing, Integration Testing, and System Testing

Unit Testing Integration Logic

Integration components have testable logic: data transformation, business rules applied during routing, validation, field mapping. These should be unit tested in isolation.

What to unit test:

• Transformation functions — does the source schema map correctly to the target schema?
• Validation rules — does the validator reject invalid messages and accept valid ones?
• Routing logic — does the content-based router send messages to the correct channel?
• Error classification — does the error handler correctly classify transient vs. permanent errors?

Approach:

• Inject dependencies (message broker, HTTP client) via interfaces so they can be replaced with test doubles in unit tests
• Use table-driven tests to cover multiple input scenarios with minimal test code
• Test the unhappy path: malformed data, missing required fields, boundary values

// Example: unit test for a transformation function
[Theory]
[InlineData("John", "Doe",  "John Doe")]
[InlineData("",     "Doe",  "Doe")]
[InlineData("John", "",     "John")]
public void BuildFullName_ReturnsExpectedResult(string first, string last, string expected)
{
    var result = CustomerTransformer.BuildFullName(first, last);
    Assert.Equal(expected, result);
}

Contract Testing

Consumer-Driven Contract Testing verifies that a provider API or event schema matches what consumers actually depend on — without requiring both systems to be deployed at the same time.

How it works:

1. Each consumer writes a contract — a specification of the API calls it makes or events it expects
2. The contract is shared with the provider
3. The provider runs the contracts as tests against its implementation
4. If the provider changes in a way that breaks a consumer contract, the test fails

Tools: Pact (REST), AsyncAPI + contract testing frameworks (messaging).

Contract testing is the most efficient way to catch breaking interface changes before they reach production.

Integration Testing (Component Level)

Test each integration endpoint in isolation against real infrastructure (or close approximations):

• Spin up a real message broker (or use a containerised version)
• Test that your integration correctly sends messages and handles responses
• Test error scenarios: broker unavailable, malformed response, timeout

Test container approach: Use Docker to spin up dependencies (RabbitMQ, Kafka, SQL Server) for the duration of the test run, then tear them down.

# docker-compose.test.yml
services:
  rabbitmq:
    image: rabbitmq:3-management
    ports:
      - "5672:5672"
  sqlserver:
    image: mcr.microsoft.com/mssql/server:2022-latest
    environment:
      SA_PASSWORD: "Test@1234"

System Integration Testing (SIT)

SIT tests the complete end-to-end flow across all participating systems in an environment that closely mirrors production.

SIT scope:

• All integration paths (happy path + defined error paths)
• All data transformation scenarios
• Error handling and retry behaviour
• DLQ routing and resubmission
• Performance at expected production volumes

SIT entry criteria:

• Unit tests pass
• Individual component integration tests pass
• Test environment is configured and stable
• Test data is prepared
• All teams whose systems participate in the test are available

SIT exit criteria:

• All test cases executed
• No unresolved P1 or P2 defects
• Performance tests meet SLA targets
• Security test checklist complete

Test Data Management and Test Environment Setup

Test Data Challenges

Integration test data is harder to manage than application test data because:

• It must exist in multiple systems simultaneously and be consistent across them
• Some data (PII, PHI) cannot be used directly in non-production environments
• Test data for performance tests requires large volumes
• Test data can be corrupted by failed test runs, leaving the environment in an unpredictable state

Test Data Strategies

Synthetic data generation
Create test data programmatically. Ensure it:

• Covers all code paths (valid data, invalid data, boundary conditions, edge cases)
• Is consistent across systems (if order ORD-TEST-001 exists in the source ERP, it must also exist in the target WMS)
• Is isolated per test run (use unique IDs that prevent conflicts between parallel test runs)

Data masking / anonymisation
For environments that require realistic data volumes or formats (performance testing, UAT), create anonymised copies of production data:

• Replace PII fields (names, email addresses, phone numbers) with realistic fake values
• Preserve data structure and statistical distributions
• Maintain referential integrity across systems

Test data setup and teardown
Automate test data creation (setup) and cleanup (teardown) as part of the test suite. Tests that leave data behind create interference with subsequent test runs.

Environment Strategy

| Environment | Purpose | Data Source | Who Has Access | |---|---|---|---| | Development | Developer testing | Synthetic | Developers | | Test / SIT | Integration testing | Synthetic + anonymised | Dev + QA | | Performance | Load and stress testing | Anonymised production clone | Performance engineers | | UAT | Business acceptance | Anonymised or synthetic | Business + delivery team | | Production | Live operations | Real | Ops team only |

Never use production data in development or test environments. Never test integrations directly against production systems.

Continuous Integration and Deployment Pipelines

CI/CD for Integration Solutions

Integration solutions benefit from CI/CD just as much as application code:

• Continuous Integration (CI): every code change triggers automated tests — unit tests, contract tests, and integration tests — before it can be merged
• Continuous Deployment (CD): changes that pass all tests are automatically deployed to test and staging environments; production deployment is manual or gated

Pipeline Stages for Integration Projects

┌──────────────────────────────────────────────────────────────────┐
│ CI Pipeline                                                       │
│  1. Build & compile                                               │
│  2. Unit tests (transformation, routing, validation logic)        │
│  3. Contract tests (consumer-driven contracts against providers)  │
│  4. Component integration tests (containerised dependencies)      │
│  5. Static analysis (linting, security scanning)                  │
│  6. Package artefact                                              │
└──────────────────────────────────────────────────────────────────┘
                              ↓ (on merge to main)
┌──────────────────────────────────────────────────────────────────┐
│ CD Pipeline — Test Environment                                    │
│  1. Deploy integration to test environment                        │
│  2. Run SIT test suite                                            │
│  3. Run smoke tests                                               │
│  4. Publish test results                                          │
└──────────────────────────────────────────────────────────────────┘
                              ↓ (on approval)
┌──────────────────────────────────────────────────────────────────┐
│ CD Pipeline — Production                                          │
│  1. Deploy to staging (production mirror)                         │
│  2. Run smoke tests against staging                               │
│  3. Blue/green or canary deployment to production                 │
│  4. Post-deployment smoke tests                                   │
│  5. Monitor for 30 minutes; auto-rollback on error rate spike     │
└──────────────────────────────────────────────────────────────────┘

Integration-Specific Pipeline Considerations

Infrastructure as Code (IaC)
Message queues, topics, API management policies, and event hubs should be defined in IaC (Terraform, Bicep, CloudFormation) and provisioned by the pipeline — not created manually.

Configuration management
Integration configuration (endpoint URLs, retry policies, DLQ settings) must be environment-specific and managed via environment variables or a configuration service. Never hardcode environment-specific values.

Schema registry
For event-driven integrations, manage event schemas in a schema registry (Confluent Schema Registry, Azure Schema Registry). The pipeline should validate that message payloads conform to their registered schema before deployment.

Deployment strategies for breaking changes
If an interface change is breaking (removes a field, changes a field type), use a versioned deployment:

1. Deploy the new version alongside the old version (v1 + v2 both active)
2. Migrate consumers from v1 to v2 over the agreed transition period
3. Deprecate and remove v1 once all consumers have migrated

Managing Versioning and Change Control in Integration Projects

Interface Versioning Strategies

URI versioning (REST APIs)
Include the version in the URL path: /api/v1/orders, /api/v2/orders. Simple, visible, easy to route.

Header versioning
The version is specified in a request header: Accept: application/vnd.systemforge.order.v2+json. Keeps URLs clean but is less discoverable.

Semantic versioning for event schemas
Apply semantic versioning to event schemas:

• Patch (1.0.0 → 1.0.1): bug fix, no structural change
• Minor (1.0.0 → 1.1.0): backward-compatible addition (new optional field)
• Major (1.0.0 → 2.0.0): breaking change (removed field, changed type)

Consumers should tolerate minor version changes without modification. Major version changes require a coordinated migration.

What Constitutes a Breaking Change

| Change Type | Breaking? | |---|---| | Add optional field | No | | Add required field | Yes | | Remove field | Yes | | Change field type | Yes (e.g., string → integer) | | Rename field | Yes | | Change enum values (add) | No | | Change enum values (remove) | Yes | | Change endpoint URL | Yes | | Add new endpoint | No | | Change HTTP method | Yes |

Deprecation Policy

Establish a deprecation policy before it is needed:

• Minimum notice period before removing an interface (typically 3–6 months)
• How consumers are notified (deprecation headers, email, registry update)
• What support is available during the migration period
• What happens to consumers that have not migrated by the end date

Module Summary

• Invest in a testing pyramid: many unit tests for transformation and routing logic, contract tests for every interface, integration tests with containerised dependencies, and targeted SIT for critical end-to-end flows.
• Contract testing is the most effective way to catch breaking interface changes early and is essential for event-driven architectures where many consumers depend on a single schema.
• Test data must be synthetic or anonymised — never use production PII in non-production environments. Automate setup and teardown to keep test environments clean.
• CI/CD pipelines for integration projects must include unit tests, contract tests, component integration tests, and IaC provisioning. Production deployment should use blue/green or canary strategies with automated rollback.
• Version every interface. Define what constitutes a breaking change, enforce a deprecation policy, and use schema registries to manage event schema evolution.

Next: Module 7 — Integration Maintenance and Optimisation