Definition & Overview

The mutation score is the primary metric produced by mutation testing, calculated as the ratio of killed mutants to total decisive mutants (killed + survived), expressed as a percentage. A mutation score of 85% means that 85 out of every 100 non-equivalent mutants were detected by the test suite. The remaining 15% represent test gaps where the suite fails to distinguish between correct and incorrect code behavior.

Mutation score is considered the gold standard for test suite quality measurement because it directly assesses behavioral verification rather than mere code execution. A test suite can achieve 100% line coverage while having a mutation score well below 50%, because coverage only measures which code was run, not whether the tests actually checked the results. Mutation score answers the more fundamental question: if a bug were introduced at this location, would the tests catch it?

In the Prismatic Platform, mutation scores are tracked as part of the Quality DNA system. The NO MERCY doctrine mandates specific minimum mutation scores based on module criticality: 100% for security-critical paths (Perimeter scoring, authentication, NABLA confidence), 90% for core business logic (DD pipeline, OSINT execution), and 80% for standard application code. Scores are computed during CI and tracked over time to detect quality trends.

Technical Deep Dive

The mutation score formula is: MS = (killed / (killed + survived)) * 100. Equivalent mutants (those that produce identical behavior to the original) are excluded from both numerator and denominator because they are impossible to kill and would unfairly penalize the score. Timed-out mutants are typically counted as killed (since the original does not timeout, the behavioral difference is detectable).

Interpreting mutation scores requires context. A score of 80% does not mean "20% of the code is untested." It means that 20% of possible single-fault mutations are undetected. Some of these may be in trivial code (logging, formatting) where the business impact of a missed bug is minimal. Others may be in critical paths where even one missed mutation indicates a serious test gap. The Prismatic Platform addresses this by computing per-module mutation scores and weighting them by module criticality.

defmodule PrismaticQuality.MutationScore do
  @moduledoc """
  Mutation score computation, tracking, and compliance checking.
  Integrates with Quality DNA for cross-session score monitoring.
  """

  @type score_result :: %{
    module: String.t(),
    score: float(),
    killed: non_neg_integer(),
    survived: non_neg_integer(),
    equivalent: non_neg_integer(),
    total: non_neg_integer(),
    compliance: :compliant | :warning | :violation,
    threshold: float()
  }

  @criticality_thresholds %{
    critical: 100.0,
    core: 90.0,
    standard: 80.0,
    utility: 70.0
  }

  @spec compute([PrismaticQuality.Mutation.t()]) :: float()
  def compute(mutations) do
    killed = Enum.count(mutations, &(&1.status == :killed))
    survived = Enum.count(mutations, &(&1.status == :survived))
    timed_out = Enum.count(mutations, &(&1.status == :timeout))

    decisive = killed + timed_out + survived

    if decisive > 0 do
      (killed + timed_out) / decisive * 100
      |> Float.round(2)
    else
      100.0
    end
  end

  @spec evaluate(String.t(), [PrismaticQuality.Mutation.t()], atom()) :: score_result()
  def evaluate(module_path, mutations, criticality \\ :standard) do
    score = compute(mutations)
    threshold = Map.fetch!(@criticality_thresholds, criticality)

    compliance =
      cond do
        score >= threshold -> :compliant
        score >= threshold - 5.0 -> :warning
        true -> :violation
      end

    %{
      module: module_path,
      score: score,
      killed: Enum.count(mutations, &(&1.status in [:killed, :timeout])),
      survived: Enum.count(mutations, &(&1.status == :survived)),
      equivalent: Enum.count(mutations, &(&1.status == :equivalent)),
      total: length(mutations),
      compliance: compliance,
      threshold: threshold
    }
  end

  @spec trend([{DateTime.t(), float()}]) :: %{
    direction: :improving | :stable | :declining,
    change_rate: float(),
    current: float(),
    average: float()
  }
  def trend(historical_scores) when length(historical_scores) >= 2 do
    scores = Enum.map(historical_scores, &elem(&1, 1))
    recent = Enum.take(scores, -5)
    older = Enum.take(scores, 5)

    recent_avg = Enum.sum(recent) / length(recent)
    older_avg = Enum.sum(older) / length(older)
    change_rate = recent_avg - older_avg

    direction =
      cond do
        change_rate > 1.0 -> :improving
        change_rate < -1.0 -> :declining
        true -> :stable
      end

    %{
      direction: direction,
      change_rate: Float.round(change_rate, 2),
      current: List.last(scores),
      average: Float.round(Enum.sum(scores) / length(scores), 2)
    }
  end

  def trend(_), do: %{direction: :stable, change_rate: 0.0, current: 0.0, average: 0.0}
end

Higher mutation scores have diminishing returns in terms of development effort. Moving from 80% to 90% typically requires adding targeted tests for edge cases. Moving from 90% to 95% requires testing subtle behavioral differences. Moving from 95% to 100% often involves equivalent mutant analysis and highly specific assertions. The platform's tiered thresholds reflect this effort curve, reserving the 100% requirement for code where the cost of missed bugs is highest.

Architecture & Implementation

The mutation score system integrates with three platform subsystems. The Quality DNA persists historical scores in .claude/quality-dna/current-state.json, enabling trend analysis across sessions. The pre-commit hooks can optionally enforce minimum mutation scores for changed files, preventing quality degradation at the commit level. The CI pipeline computes scores for all critical modules and publishes results to the quality dashboard.

Score decomposition by mutation operator provides actionable insights. If a module has a high overall score but low scores for relational operator mutations, it indicates that boundary conditions are undertested. If arithmetic mutations survive, mathematical calculations lack assertions. This operator-level analysis guides test improvement efforts toward the most impactful areas.

The platform also computes a weighted aggregate mutation score across all critical modules, using module criticality as the weight. This single number provides a platform-level quality indicator that executives and project managers can track without understanding mutation testing details. A platform mutation score above 90% is a strong indicator of comprehensive test coverage.

Interactive Mutation Score Analysis

Platform Mutation Scores by Module

Security-Critical

100%

Required threshold

Core Business Logic

90%

Required threshold

Standard Application

80%

Required threshold

Live Data: Last updated:

Cache: % hit rate

Connecting to live mutation score data...

⚠️ Connection Error:

Usage in Prismatic Platform

Integrated mutation score tracking in the quality pipeline:

defmodule PrismaticQuality.MutationScoreGate do
  @moduledoc """
  Quality gate enforcement for mutation scores.
  Blocks merges when critical modules fall below thresholds.
  """

  alias PrismaticQuality.MutationScore

  @critical_modules %{
    "apps/prismatic_perimeter/lib/prismatic_perimeter/scoring.ex" => :critical,
    "apps/prismatic_perimeter/lib/prismatic_perimeter/compliance/nis2.ex" => :critical,
    "apps/prismatic_dd/lib/prismatic_dd/loader.ex" => :core,
    "apps/prismatic_osint_core/lib/prismatic_osint_core/tool_registry.ex" => :core,
    "apps/prismatic_academy/lib/prismatic_academy/topic_registry.ex" => :standard
  }

  @spec check_compliance() :: {:ok, :all_compliant} | {:error, [MutationScore.score_result()]}
  def check_compliance do
    results =
      @critical_modules
      |> Enum.map(fn {module_path, criticality} ->
        mutations = run_mutations_for(module_path)
        MutationScore.evaluate(module_path, mutations, criticality)
      end)

    violations = Enum.filter(results, &(&1.compliance == :violation))

    case violations do
      [] -> {:ok, :all_compliant}
      violations -> {:error, violations}
    end
  end

  @spec format_report([MutationScore.score_result()]) :: String.t()
  def format_report(results) do
    results
    |> Enum.map(fn r ->
      status = if r.compliance == :compliant, do: "PASS", else: "FAIL"
      "  [#{status}] #{r.module}: #{r.score}% (threshold: #{r.threshold}%)"
    end)
    |> Enum.join("\n")
    |> then(&"Mutation Score Report:\n#{&1}")
  end

  defp run_mutations_for(_module_path) do
    # Delegates to MutantGenerator + MutationTesting
    []
  end
end

The mutation score gate ensures that test quality is maintained at the level required by each module's criticality, preventing quality erosion that could compromise the platform's security and reliability guarantees.

Cross-References

Mutation Testing - The methodology producing mutation scores
Mutant - Individual code change contributing to the score
Killed/Survived - Binary outcomes that determine the score
Test Coverage - Complementary but weaker quality metric
Quality DNA - Cross-session quality tracking including mutation scores

Created by Tomas Korcak (korczis) | GitHub

Mutation Score

Definition & Overview

Technical Deep Dive

Architecture & Implementation

Interactive Mutation Score Analysis

Usage in Prismatic Platform

Cross-References

Glossary

Related Terms