Mutation Testing in .NET with Stryker — Measure Real Test Quality

Code coverage tells you which lines your tests execute. Mutation testing tells you whether your tests would actually catch a bug. Stryker.NET introduces small bugs (mutations) into your code and checks whether your test suite fails as a result. Tests that don't catch a mutation are not testing anything useful.

What you'll learn:

• Why coverage is insufficient and what mutation score measures
• Installing and running Stryker.NET
• Understanding the mutation report
• Common mutators and what they catch
• Configuring Stryker for realistic CI use
• Interpreting and acting on results

Coverage vs Mutation Score

public decimal CalculateDiscount(decimal total, string tier)
{
    if (tier == "gold")
        return total * 0.20m;
    return total * 0.10m;
}

// This test gives 100% line coverage
[Fact]
public void CalculateDiscount_Gold_Returns_Twenty_Percent()
{
    var result = _service.CalculateDiscount(100m, "gold");
    // Missing: Assert.Equal(20m, result);
    // Just calling the method — no assertion!
}

Line coverage: 100%. The test executes every line. But if Stryker changes 0.20m to 0.10m, the test still passes — it asserts nothing. Mutation score: 0%. The test is worthless.

Mutation testing finds assertion-free tests, insufficient boundary checks, and logic that could be wrong without any test failing.

Installation

# Install Stryker as a .NET tool (global or local)
dotnet tool install --global dotnet-stryker

# Or as a local tool (recommended for CI)
dotnet new tool-manifest  # creates .config/dotnet-tools.json
dotnet tool install dotnet-stryker

Running Stryker

# From the solution root — tests all projects
dotnet stryker

# From a specific test project directory
cd tests/MyApp.Tests
dotnet stryker

# Target a specific source project
dotnet stryker --project MyApp.Application

Stryker discovers your test projects automatically via csproj references. It runs your existing tests after each mutation and reports which mutations were "killed" (test failed — good) vs "survived" (test passed — bad).

Reading the Report

Stryker generates an HTML report in StrykerOutput/reports/. Open it in a browser:

Mutation score: 73%

✓ Killed:    146   (mutation caught by tests — good)
✗ Survived:   54   (mutation not caught — test gap)
  No coverage:  8   (mutation in uncovered code)
  Timeout:      3   (test hung — investigate)

Mutation example in the report

// Original
if (order.Total > 1000)
    ApplyBulkDiscount(order);

// Mutation 1 (Survived — ✗ BAD): changed > to >=
if (order.Total >= 1000)
    ApplyBulkDiscount(order);

// Mutation 2 (Killed — ✓ GOOD): changed > to <
if (order.Total < 1000)
    ApplyBulkDiscount(order);

Mutation 1 survived because your tests never test the boundary condition at exactly 1000. Add a test:

[Theory]
[InlineData(999.99, false)]
[InlineData(1000.00, true)]   // boundary — killed the >= mutation
[InlineData(1001.00, true)]
public void BulkDiscount_Applied_Only_Above_1000(decimal total, bool shouldApply)
{
    var order = new Order { Total = total };
    _service.ProcessOrder(order);
    Assert.Equal(shouldApply, order.HasBulkDiscount);
}

Common Mutators

Stryker applies these mutations by default:

Arithmetic operators: + → -, * → /, etc.

return a + b;  // mutated to: return a - b;

Boolean literals: true → false

return isActive && hasPermission;  // mutated to: return false && hasPermission;

Conditional operators: > → >=, == → !=

if (count > 0)  // mutated to: if (count >= 0), if (count < 0), etc.

String mutations: empty string substitution

return "error";  // mutated to: return "";

Logical operators: && → ||

if (isValid && isActive)  // mutated to: if (isValid || isActive)

Statement deletion: removes a line entirely

_auditLog.Record(action);  // deleted — tests should catch missing side effect

Statement deletion is often the most revealing — it catches tests that don't verify that a method was called (missing mock verification).

Configuration

Create stryker-config.json in your solution root:

{
  "stryker-config": {
    "project": "MyApp.Application/MyApp.Application.csproj",
    "test-projects": [
      "tests/MyApp.Tests/MyApp.Tests.csproj"
    ],
    "mutation-level": "Standard",
    "reporters": ["html", "json", "dashboard"],
    "threshold-high": 80,
    "threshold-low": 70,
    "threshold-break": 60,
    "coverage-analysis": "perTest",
    "since": {
      "enabled": true,
      "git-diff-target": "main"
    },
    "mutate": [
      "src/**/*.cs",
      "!src/**/*Generated*.cs",
      "!src/**/*Migrations*/**"
    ],
    "ignore-mutations": [
      "string"
    ]
  }
}

Key options:

threshold-break: Stryker exits with code 1 (fails CI) if the score drops below this. Set it at a level your current codebase can achieve — don't start at 80% on a legacy codebase.

coverage-analysis: "perTest": Stryker analyses which tests cover which code and only runs relevant tests per mutation — 10-50x faster than running all tests for every mutation.

since: { enabled: true, git-diff-target: "main" }: Only mutate code changed since main. This makes PR checks fast — only the touched code is mutation-tested.

ignore-mutations: ["string"]: String mutations often generate false positives (error message strings, log messages). Ignore them unless your tests check message content.

CI Integration

# .github/workflows/ci.yml
- name: Restore Stryker tool
  run: dotnet tool restore

- name: Run mutation tests
  run: dotnet stryker --config-file stryker-config.json
  # Stryker exits with code 1 if score < threshold-break
  # GitHub Actions treats non-zero exit as failure

Practical CI thresholds

Start permissively, tighten over time:

| Phase | threshold-break | Note | |---|---|---| | Initial adoption | 50% | Just get it running | | 3 months in | 65% | Teams start writing better tests | | Steady state | 75–80% | High confidence in test suite |

Don't target 100%. Mutation testing has diminishing returns above 85% — surviving mutations are often equivalent mutations (two different expressions with identical behaviour) or mutations in genuinely untestable code paths (infrastructure boundaries, generated code).

Acting on Surviving Mutations

Not every surviving mutation requires action. Categorise them:

Fix the test: The mutation reveals a genuine gap. Write a test that kills it.

// Survived: statement deletion of _cache.Invalidate(key)
// Fix: verify the call in the test
mockCache.Verify(c => c.Invalidate(orderId), Times.Once);

Mark as excluded: The mutation is in code that genuinely shouldn't be tested (logging, telemetry, third-party calls).

// stryker-config.json — exclude specific files
"mutate": [
  "src/**/*.cs",
  "!src/**/Telemetry*.cs",
  "!src/**/LoggingExtensions.cs"
]

Equivalent mutation: The mutated code behaves identically to the original. This is rare but real — if a mutation survives and you can't find a test that would behave differently, document it and move on.

What Mutation Testing Is Not

It is not a replacement for code review. Mutations test that your tests verify behaviour, not that the behaviour is correct.

It does not tell you what to test. It tells you what your existing tests don't test. You still need to reason about requirements.

It is not fast. Even with perTest coverage analysis, mutation testing is 3–10x slower than a normal test run. Run it on changed code in CI (using --since), not on the full codebase on every commit.

It does not replace integration tests. Mutation testing works on unit-tested code. Integration tests cover scenarios that mutations don't model: network failures, database constraint violations, external service responses.