System Design · Lesson 16 of 26

Service Discovery & API Gateway Patterns

In a microservices system, services come and go. Instances scale up and down. IPs change. Kubernetes assigns new IP addresses on every pod restart.

How does Service A know where to find Service B?

That's the service discovery problem.

The Problem

In a monolith, function calls don't need routing — they're in-process. In microservices, every call is a network call to an address that can change.

Static configuration (naive, breaks quickly):
  Order Service config: payment-service-url = "10.0.0.45:8080"
  
  Problem: what happens when:
  - Payment Service restarts (new IP)
  - Payment Service scales to 3 instances
  - Payment Service moves to a different node
  
  Answer: Order Service breaks, and you're manually updating configs

Service discovery solves this: services register their location in a central registry. Clients look up locations dynamically.

Client-Side Discovery

The client queries the service registry directly and picks which instance to call.

Flow:
  1. Payment Service starts → registers with registry
     {"service": "payment-service", "ip": "10.0.0.45", "port": 8080}
  
  2. Order Service wants to call Payment Service:
     a. Query registry: "where are instances of payment-service?"
     b. Registry returns: ["10.0.0.45:8080", "10.0.0.46:8080", "10.0.0.47:8080"]
     c. Order Service picks one (round robin, random, etc.)
     d. Makes HTTP call to chosen instance

┌──────────────┐  1. register   ┌──────────────┐
│  Payment Svc │─────────────→  │   Registry   │
│  (3 instances)│               │  (Consul,    │
└──────────────┘                │   Eureka)    │
                                └──────┬───────┘
┌──────────────┐  2. lookup            │
│  Order Svc   │─────────────────────→ │
│              │  3. returns addresses  │
│              │ ←──────────────────── │
│              │
│              │  4. direct call
│              │─────────────────→  Payment Svc instance
└──────────────┘

Netflix Eureka is the classic example — used in Netflix's Spring Cloud ecosystem.

Pros: Client has full control over load balancing algorithm.

Cons: Every client must implement discovery logic. Every service SDK must embed the registry client. Couples clients to the registry implementation.

Server-Side Discovery

A load balancer or proxy sits in front of service instances. Clients call the load balancer, which queries the registry and routes to the correct instance.

Flow:
  1. Payment Service registers with registry
  2. Order Service calls: http://payment-service/ (no IP, just name)
  3. Load balancer receives request
  4. Load balancer queries registry: "where are instances of payment-service?"
  5. Load balancer routes to a healthy instance

┌──────────────┐            ┌──────────────┐  query  ┌──────────────┐
│  Order Svc   │─────────→  │ Load Balancer│────────→ │   Registry   │
│              │            │              │          │   (Consul)   │
└──────────────┘            └──────┬───────┘          └──────────────┘
                                   │
                        ┌──────────┴──────────┐
                   ┌────▼───┐           ┌─────▼──┐
                   │Pay Svc1│           │Pay Svc2│
                   └────────┘           └────────┘

Pros: Client is simple — just call a service name. Load balancing logic is centralized.

Cons: Load balancer must be highly available (it's now a critical path component).

Kubernetes uses server-side discovery via kube-proxy and DNS. More on this below.

Service Registries

Consul (HashiCorp)

Consul is a service registry with health checking, key-value store, and service mesh capabilities.

Bash

# Register a service with Consul
curl -X PUT http://consul:8500/v1/agent/service/register \
  -H "Content-Type: application/json" \
  -d '{
    "Name": "payment-service",
    "ID": "payment-service-1",
    "Address": "10.0.0.45",
    "Port": 8080,
    "Check": {
      "HTTP": "http://10.0.0.45:8080/health",
      "Interval": "10s",
      "Timeout": "3s"
    }
  }'

# Discover service instances
curl http://consul:8500/v1/health/service/payment-service?passing=true

Consul removes unhealthy instances automatically via health check failures.

etcd

etcd is a distributed key-value store used by Kubernetes to store all cluster state — including service locations. Applications can watch etcd for real-time updates when service locations change.

Kubernetes Built-In Service Discovery

Kubernetes has service discovery built in. Every service gets a stable DNS name.

YAML

# Kubernetes Service — creates a stable endpoint
apiVersion: v1
kind: Service
metadata:
  name: payment-service
  namespace: ecommerce
spec:
  selector:
    app: payment-service      # routes to pods with this label
  ports:
    - port: 8080
      targetPort: 8080
  type: ClusterIP              # internal only

DNS resolution inside the cluster:
  http://payment-service/            → same namespace
  http://payment-service.ecommerce/ → cross-namespace
  http://payment-service.ecommerce.svc.cluster.local/ → fully qualified

kube-proxy handles routing to actual pod IPs transparently.
The service name never changes even when pods restart.

In Kubernetes, you don't need Consul or Eureka — the platform provides service discovery via DNS and kube-proxy.

// In .NET, simply use the Kubernetes service name
services.AddHttpClient<IPaymentClient, PaymentClient>(client =>
{
    client.BaseAddress = new Uri("http://payment-service/");
    // Kubernetes DNS resolves this to actual pod IPs
});

API Gateway — The Front Door

An API gateway is a reverse proxy that sits between clients and your services. It handles concerns that span all services.

Without API Gateway:
  Mobile App → directly calls Order Service, User Service, Product Service
  Each service implements auth, rate limiting, logging separately
  CORS headers, TLS, versioning all duplicated

With API Gateway:
  Mobile App → API Gateway → Order Service
                           → User Service
                           → Product Service
  
  API Gateway handles:
    ✓ Authentication / Authorization (JWT validation)
    ✓ Rate limiting (100 req/min per user)
    ✓ TLS termination (HTTPS at edge)
    ✓ Request routing (path-based)
    ✓ Request/response transformation
    ✓ Logging and correlation IDs
    ✓ CORS headers
    ✓ API versioning

YARP — Reverse Proxy for .NET

YARP (Yet Another Reverse Proxy) is Microsoft's official .NET reverse proxy library. It's what powers Azure App Service routing internally.

// Program.cs
var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

var app = builder.Build();
app.MapReverseProxy();
app.Run();

JSON

// appsettings.json
{
  "ReverseProxy": {
    "Routes": {
      "orders-route": {
        "ClusterId": "order-cluster",
        "Match": { "Path": "/api/orders/{**catch-all}" },
        "Transforms": [
          { "PathPattern": "/api/orders/{**catch-all}" }
        ]
      },
      "users-route": {
        "ClusterId": "user-cluster",
        "Match": { "Path": "/api/users/{**catch-all}" }
      }
    },
    "Clusters": {
      "order-cluster": {
        "Destinations": {
          "order-svc-1": { "Address": "http://order-service:8080/" }
        }
      },
      "user-cluster": {
        "Destinations": {
          "user-svc-1": { "Address": "http://user-service:8080/" }
        }
      }
    }
  }
}

Adding JWT validation middleware to YARP:

builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =>
    {
        options.Authority = "https://your-identity-server/";
        options.Audience = "api";
    });

app.UseAuthentication();
app.UseAuthorization();
app.MapReverseProxy();

Kong — API Gateway for Larger Setups

Kong is an open-source API gateway built on NGINX. Used by enterprise teams that need plugin ecosystems.

YAML

# Kong declarative config (deck)
services:
  - name: order-service
    url: http://order-service:8080
    routes:
      - name: orders-route
        paths: ["/api/orders"]
    plugins:
      - name: jwt
      - name: rate-limiting
        config:
          minute: 100
          policy: local
      - name: correlation-id
        config:
          header_name: X-Correlation-ID
          generator: uuid

AWS API Gateway

For AWS workloads, API Gateway is the managed solution. Integrates natively with Lambda, ALB, and ECS.

Features:
  - Route to Lambda functions, HTTP backends, or ALB
  - JWT/Cognito authorization out of the box
  - Built-in throttling and quotas
  - WebSocket support
  - Usage plans per API key
  - CloudWatch logging and X-Ray tracing

The tradeoff: AWS API Gateway adds ~10-50ms latency per request. For low-latency services, consider ALB + Lambda authorizer instead.

The BFF Pattern — Backend for Frontend

A Backend for Frontend (BFF) is a specialized API gateway layer built for a specific client type.

Problem:
  Mobile app needs: minimal data (limited bandwidth)
  Web app needs: rich data (multiple joined resources)
  Partner API needs: raw data (full access)
  
  One API tries to serve all three → compromises everywhere

Solution — BFF pattern:
  Mobile App → Mobile BFF → Services (returns minimal payload)
  Web App    → Web BFF    → Services (returns rich/composed payload)
  Partners   → Partner API Gateway → Services (full access with API keys)

// Web BFF — composes data for the web client's needs
[ApiController]
[Route("web/orders")]
public class WebOrderController : ControllerBase
{
    // Returns full order with product details and user info
    // (web client shows a rich order details page)
    [HttpGet("{id}")]
    public async Task<OrderDetailViewModel> GetOrderDetail(Guid id)
    {
        var order = await _orderClient.GetAsync(id);
        var user = await _userClient.GetAsync(order.UserId);
        var products = await Task.WhenAll(
            order.Items.Select(i => _productClient.GetAsync(i.ProductId)));
        
        return new OrderDetailViewModel(order, user, products);
    }
}

// Mobile BFF — same data, different shape
[ApiController]
[Route("mobile/orders")]
public class MobileOrderController : ControllerBase
{
    // Returns compact summary (mobile has limited bandwidth)
    [HttpGet("{id}")]
    public async Task<OrderSummaryViewModel> GetOrderSummary(Guid id)
    {
        var order = await _orderClient.GetAsync(id);
        return new OrderSummaryViewModel
        {
            Status = order.Status,
            Total = order.Total,
            ItemCount = order.Items.Count
        };
    }
}

API Gateway vs Service Mesh

These are often confused because both handle inter-service communication.

| | API Gateway | Service Mesh (Istio, Linkerd) | |---|---|---| | Position | North-south (client → services) | East-west (service → service) | | Manages | External traffic | Internal traffic | | Auth | User authentication (JWT, OAuth) | Service-to-service mTLS | | Features | Routing, rate limiting, transformation | Retries, circuit breaking, observability | | Complexity | Low-moderate | High | | Examples | YARP, Kong, AWS API Gateway | Istio, Linkerd, Consul Connect |

Use both when: You have an API gateway for external traffic AND want consistent retry/mTLS for all internal service calls.

Use only an API gateway when: Small system, internal services trust each other, not enough complexity to justify a service mesh.

Use only a service mesh when: All clients are internal (no public API) and you want consistent observability/security for internal calls.

Most teams don't need a service mesh. Add it when you have >10 services and are hitting real problems with retry logic, mutual TLS, or observability — not as a default.

Key Takeaways

Service discovery solves the "where is this service?" problem as instances come and go.
Client-side discovery (Eureka): client queries registry and picks an instance.
Server-side discovery (Kubernetes): platform routes transparently via stable DNS names.
Kubernetes DNS (http://service-name/) is the default discovery mechanism in Kubernetes — no Consul needed.
API Gateway handles: auth validation, rate limiting, routing, TLS termination, correlation IDs, CORS.
YARP is the .NET-native reverse proxy for building API gateways. Kong for enterprise plugin needs. AWS API Gateway for Lambda/AWS workloads.
BFF pattern: build separate API gateway layers for different client types (mobile, web, partners).
API Gateway = north-south (external traffic). Service Mesh = east-west (internal traffic).
Most teams don't need a service mesh — add one when you have >10 services and real complexity.

Data Management in Microservices — No Shared DB

Next Lesson

Resilience Patterns — Circuit Breaker, Retry & Bulkhead