Terraform Modules & Best Practices

Why Modules?

Imagine you deploy to three environments: dev, staging, and prod. Without modules you'd copy-paste the same Lambda + API Gateway + DynamoDB Terraform three times. When you need to change a security setting, you change it in three places and hope you don't miss one.

Modules solve this — define infrastructure once, instantiate it many times with different inputs.

modules/
└── serverless-api/      ← define once
    ├── main.tf
    ├── variables.tf
    └── outputs.tf

environments/
├── dev/
│   └── main.tf          ← call module with dev inputs
├── staging/
│   └── main.tf          ← call module with staging inputs
└── prod/
    └── main.tf          ← call module with prod inputs

Writing Your First Module

A Terraform module is just a directory of .tf files. No special declaration needed — any directory is a module.

Module: `modules/serverless-api/`

HCL

# modules/serverless-api/variables.tf

variable "environment" {
  description = "Deployment environment (dev/staging/prod)"
  type        = string
}

variable "project_name" {
  description = "Project name prefix"
  type        = string
}

variable "lambda_handler" {
  description = "Lambda handler in format file.function_name"
  type        = string
  default     = "handler.lambda_handler"
}

variable "lambda_runtime" {
  description = "Lambda runtime identifier"
  type        = string
  default     = "python3.12"
}

variable "lambda_memory_mb" {
  type    = number
  default = 256
}

variable "lambda_timeout_seconds" {
  type    = number
  default = 30
}

variable "lambda_source_dir" {
  description = "Path to Lambda source code directory"
  type        = string
}

variable "environment_variables" {
  description = "Environment variables to pass to Lambda"
  type        = map(string)
  default     = {}
}

variable "enable_point_in_time_recovery" {
  description = "Enable DynamoDB PITR"
  type        = bool
  default     = false
}

variable "log_retention_days" {
  type    = number
  default = 14
}

variable "tags" {
  description = "Additional tags to merge"
  type        = map(string)
  default     = {}
}

HCL

# modules/serverless-api/main.tf

locals {
  name_prefix = "${var.project_name}-${var.environment}"
  
  common_tags = merge(var.tags, {
    Environment = var.environment
    Project     = var.project_name
    ManagedBy   = "terraform"
  })
}

# DynamoDB
resource "aws_dynamodb_table" "main" {
  name         = "${local.name_prefix}-items"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "pk"
  range_key    = "sk"

  attribute { name = "pk"; type = "S" }
  attribute { name = "sk"; type = "S" }

  point_in_time_recovery {
    enabled = var.enable_point_in_time_recovery
  }

  server_side_encryption { enabled = true }
  
  deletion_protection_enabled = var.environment == "prod"

  tags = local.common_tags
}

# IAM Role
resource "aws_iam_role" "lambda" {
  name = "${local.name_prefix}-lambda-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Action    = "sts:AssumeRole"
      Principal = { Service = "lambda.amazonaws.com" }
    }]
  })

  tags = local.common_tags
}

resource "aws_iam_role_policy_attachment" "basic_execution" {
  role       = aws_iam_role.lambda.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "dynamodb" {
  name = "dynamodb-access"
  role = aws_iam_role.lambda.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "dynamodb:GetItem", "dynamodb:PutItem", "dynamodb:UpdateItem",
        "dynamodb:DeleteItem", "dynamodb:Query", "dynamodb:Scan"
      ]
      Resource = [
        aws_dynamodb_table.main.arn,
        "${aws_dynamodb_table.main.arn}/index/*"
      ]
    }]
  })
}

# Lambda
data "archive_file" "lambda" {
  type        = "zip"
  source_dir  = var.lambda_source_dir
  output_path = "${path.module}/.lambda_${var.environment}.zip"
}

resource "aws_cloudwatch_log_group" "lambda" {
  name              = "/aws/lambda/${local.name_prefix}-api"
  retention_in_days = var.log_retention_days
  tags              = local.common_tags
}

resource "aws_lambda_function" "api" {
  function_name    = "${local.name_prefix}-api"
  role             = aws_iam_role.lambda.arn
  runtime          = var.lambda_runtime
  handler          = var.lambda_handler
  filename         = data.archive_file.lambda.output_path
  source_code_hash = data.archive_file.lambda.output_base64sha256
  memory_size      = var.lambda_memory_mb
  timeout          = var.lambda_timeout_seconds

  environment {
    variables = merge(var.environment_variables, {
      DYNAMODB_TABLE = aws_dynamodb_table.main.name
      ENVIRONMENT    = var.environment
    })
  }

  depends_on = [aws_cloudwatch_log_group.lambda]
  tags       = local.common_tags
}

# API Gateway
resource "aws_apigatewayv2_api" "main" {
  name          = "${local.name_prefix}-api"
  protocol_type = "HTTP"
  tags          = local.common_tags
}

resource "aws_apigatewayv2_integration" "lambda" {
  api_id                 = aws_apigatewayv2_api.main.id
  integration_type       = "AWS_PROXY"
  integration_uri        = aws_lambda_function.api.invoke_arn
  integration_method     = "POST"
  payload_format_version = "2.0"
}

resource "aws_apigatewayv2_route" "proxy" {
  api_id    = aws_apigatewayv2_api.main.id
  route_key = "$default"
  target    = "integrations/${aws_apigatewayv2_integration.lambda.id}"
}

resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.main.id
  name        = "$default"
  auto_deploy = true
}

resource "aws_lambda_permission" "api_gateway" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.api.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.main.execution_arn}/*/*"
}

HCL

# modules/serverless-api/outputs.tf

output "api_endpoint" {
  description = "API Gateway invoke URL"
  value       = aws_apigatewayv2_stage.default.invoke_url
}

output "lambda_function_name" {
  value = aws_lambda_function.api.function_name
}

output "lambda_function_arn" {
  value = aws_lambda_function.api.arn
}

output "dynamodb_table_name" {
  value = aws_dynamodb_table.main.name
}

output "dynamodb_table_arn" {
  value = aws_dynamodb_table.main.arn
}

Calling the Module

HCL

# environments/dev/main.tf

terraform {
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.0" }
  }
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "dev/terraform.tfstate"
    region = "us-east-1"
  }
}

provider "aws" { region = "us-east-1" }

module "api" {
  source = "../../modules/serverless-api"

  environment     = "dev"
  project_name    = "learnixo"
  lambda_source_dir = "${path.module}/../../lambda_src"

  lambda_memory_mb       = 256
  lambda_timeout_seconds = 30
  log_retention_days     = 7

  environment_variables = {
    LOG_LEVEL = "DEBUG"
  }

  tags = {
    CostCenter = "engineering"
  }
}

output "dev_api_url" {
  value = module.api.api_endpoint
}

HCL

# environments/prod/main.tf

module "api" {
  source = "../../modules/serverless-api"

  environment     = "prod"
  project_name    = "learnixo"
  lambda_source_dir = "${path.module}/../../lambda_src"

  lambda_memory_mb                = 512
  lambda_timeout_seconds          = 60
  enable_point_in_time_recovery   = true
  log_retention_days              = 90

  environment_variables = {
    LOG_LEVEL = "WARNING"
  }
}

Prod gets more memory, longer retention, and PITR enabled — without duplicating all the infrastructure code.

Using the Public Registry

HashiCorp maintains a public Terraform registry with thousands of community and verified modules. Use battle-tested modules instead of writing everything from scratch.

HCL

# Use the official AWS VPC module
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "my-vpc"
  cidr = "10.0.0.0/16"

  azs             = ["us-east-1a", "us-east-1b", "us-east-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24", "10.0.103.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true

  tags = { Environment = "prod" }
}

# Use the official RDS module
module "rds" {
  source  = "terraform-aws-modules/rds/aws"
  version = "~> 6.0"

  identifier     = "my-postgres"
  engine         = "postgres"
  engine_version = "16"
  instance_class = "db.t3.micro"

  allocated_storage = 20
  db_name           = "appdb"
  username          = "admin"
  port              = 5432

  vpc_security_group_ids = [module.vpc.default_security_group_id]
  subnet_ids             = module.vpc.private_subnets
  create_db_subnet_group = true

  family               = "postgres16"
  major_engine_version = "16"

  deletion_protection = true
}

Always pin module versions with version = "~> x.y". Never use unpinned modules in production — they can introduce breaking changes when the registry updates.

Module Sources

Modules can be sourced from multiple locations:

HCL

# Local path
module "api" {
  source = "./modules/serverless-api"
}

# Public registry
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"
}

# GitHub (public repo)
module "custom" {
  source = "github.com/my-org/terraform-modules//serverless-api?ref=v1.2.0"
}

# GitHub (private repo via SSH)
module "private" {
  source = "git::ssh://git@github.com/my-org/private-modules.git//module?ref=v1.0.0"
}

# Terraform Cloud private registry
module "shared" {
  source  = "app.terraform.io/my-org/serverless-api/aws"
  version = "~> 1.0"
}

Best Practices

1. Pin Provider and Module Versions

HCL

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"   # Minor updates OK, major updates require explicit change
    }
  }
}

Run terraform init -upgrade to update within constraints. Commit the generated .terraform.lock.hcl file — it pins exact provider versions like package-lock.json.

2. Use `for_each` Over `count` for Named Resources

HCL

# BAD: count creates aws_subnet.private[0], aws_subnet.private[1]
# Inserting in the middle renumbers everything
resource "aws_subnet" "private" {
  count = 3
  # ...
}

# GOOD: for_each creates aws_subnet.private["us-east-1a"], etc.
# Stable keys — adding/removing one doesn't affect others
resource "aws_subnet" "private" {
  for_each = toset(["us-east-1a", "us-east-1b", "us-east-1c"])

  availability_zone = each.value
  cidr_block        = cidrsubnet("10.0.0.0/16", 8, index(tolist(toset(["us-east-1a", "us-east-1b", "us-east-1c"])), each.value))
  vpc_id            = aws_vpc.main.id
}

3. Separate State Per Environment

state/
├── dev/terraform.tfstate      → s3://state-bucket/dev/terraform.tfstate
├── staging/terraform.tfstate  → s3://state-bucket/staging/terraform.tfstate
└── prod/terraform.tfstate     → s3://state-bucket/prod/terraform.tfstate

Never share state between environments. A failed terraform apply on dev should never affect prod.

4. Use `moved` Blocks Instead of Destroying

When you rename a resource or refactor into a module, Terraform would destroy and recreate by default. Use moved blocks to preserve state:

HCL

# Refactoring: resource moved into a module
moved {
  from = aws_lambda_function.api
  to   = module.api.aws_lambda_function.api
}

5. `lifecycle` Rules for Production Safety

HCL

resource "aws_rds_cluster" "main" {
  # ...

  lifecycle {
    # Prevent accidental deletion
    prevent_destroy = true

    # Ignore changes to password (managed outside Terraform)
    ignore_changes = [master_password]

    # Create new before destroying old (zero-downtime replacement)
    create_before_destroy = true
  }
}

6. Consistent Tagging Strategy

HCL

locals {
  required_tags = {
    Project     = var.project_name
    Environment = var.environment
    ManagedBy   = "terraform"
    Repository  = "github.com/my-org/infra"
    CostCenter  = var.cost_center
  }
}

# Use AWS provider default_tags to apply to all resources
provider "aws" {
  region = var.aws_region
  default_tags {
    tags = local.required_tags
  }
}

7. Validate Inputs with `validation` Blocks

HCL

variable "environment" {
  type = string
  validation {
    condition     = contains(["dev", "staging", "prod"], var.environment)
    error_message = "Environment must be dev, staging, or prod."
  }
}

variable "lambda_memory_mb" {
  type = number
  validation {
    condition     = var.lambda_memory_mb >= 128 && var.lambda_memory_mb <= 10240
    error_message = "Lambda memory must be between 128 MB and 10,240 MB."
  }
}

8. Document Modules with README

Every shared module should have a README.md generated from the code:

Bash

# Install terraform-docs
brew install terraform-docs

# Generate README from HCL comments and variable descriptions
terraform-docs markdown table . > README.md

Directory Structure: Large Team Pattern

infra/
├── modules/
│   ├── serverless-api/     # Reusable Lambda + API GW + DynamoDB
│   ├── vpc/                # VPC with public/private subnets
│   ├── rds-postgres/       # Encrypted PostgreSQL with parameter groups
│   └── ecs-service/        # ECS Fargate service with ALB
│
├── environments/
│   ├── _shared/            # Shared data (account IDs, region lists)
│   ├── dev/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   ├── staging/
│   │   ├── main.tf
│   │   ├── terraform.tfvars
│   │   └── backend.tf
│   └── prod/
│       ├── main.tf
│       ├── terraform.tfvars
│       └── backend.tf
│
├── bootstrap/              # One-time: S3 state bucket + DynamoDB lock table
│   └── main.tf
│
└── .terraform.lock.hcl     # Provider version lock file (commit this)

Testing Modules

`terraform validate` — Syntax check

Bash

cd modules/serverless-api
terraform init
terraform validate

`terraform plan` — Dry run in real environment

Bash

cd environments/dev
terraform plan -out=plan.tfplan

Terratest — Integration testing

// test/serverless_api_test.go
package test

import (
    "testing"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
)

func TestServerlessApiModule(t *testing.T) {
    opts := &terraform.Options{
        TerraformDir: "../modules/serverless-api",
        Vars: map[string]interface{}{
            "environment":      "test",
            "project_name":     "learnixo-test",
            "lambda_source_dir": "./fixtures/lambda",
        },
    }

    defer terraform.Destroy(t, opts)
    terraform.InitAndApply(t, opts)

    endpoint := terraform.Output(t, opts, "api_endpoint")
    assert.NotEmpty(t, endpoint)
}

Common Anti-Patterns to Avoid

Anti-pattern: Hardcoded ARNs and IDs Never hardcode resource ARNs or account IDs. Use data sources or outputs. Your staging and prod accounts have different IDs.

HCL

# BAD
role_arn = "arn:aws:iam::123456789:role/my-role"

# GOOD
data "aws_iam_role" "my_role" { name = "my-role" }
role_arn = data.aws_iam_role.my_role.arn

Anti-pattern: Giant main.tf Files with 1,000+ lines of mixed resources are hard to navigate. Split by concern: lambda.tf, iam.tf, dynamodb.tf, api_gateway.tf.

Anti-pattern: No .terraform.lock.hcl This file ensures everyone on the team (and CI) uses identical provider versions. Commit it to git.

Anti-pattern: terraform apply from your laptop in production Production applies should only happen through CI/CD. Use Terraform Cloud, Atlantis, or GitHub Actions. Local applies are fine for dev.

Summary

| Practice | Why | |----------|-----| | Modules for each stack component | DRY, tested, reusable | | Pin all versions | Reproducible builds | | for_each over count | Stable resource keys | | Separate state per environment | Isolation, safety | | moved blocks for refactors | No destroy/recreate | | lifecycle.prevent_destroy | Protect production data | | default_tags on provider | Consistent billing tags | | validation on variables | Fail early with clear messages |

Next up: Multi-Environment Strategy — structuring Terraform for dev, staging, and prod with shared modules and per-environment variable files.

Why Modules?

Writing Your First Module

Module: modules/serverless-api/

Calling the Module

Using the Public Registry

Module Sources

Best Practices

1. Pin Provider and Module Versions

2. Use for_each Over count for Named Resources

3. Separate State Per Environment

4. Use moved Blocks Instead of Destroying

5. lifecycle Rules for Production Safety

6. Consistent Tagging Strategy

7. Validate Inputs with validation Blocks

8. Document Modules with README

Directory Structure: Large Team Pattern

Testing Modules

terraform validate — Syntax check

terraform plan — Dry run in real environment

Terratest — Integration testing

Common Anti-Patterns to Avoid

Summary

Module: `modules/serverless-api/`

2. Use `for_each` Over `count` for Named Resources

4. Use `moved` Blocks Instead of Destroying

5. `lifecycle` Rules for Production Safety

7. Validate Inputs with `validation` Blocks

`terraform validate` — Syntax check

`terraform plan` — Dry run in real environment