Definition

Continuous validation is the systematic practice of automatically verifying system correctness, security, performance, and compliance at every stage of the software lifecycle. Unlike traditional quality assurance that occurs as a distinct phase before release, continuous validation is woven into every developer action -- every keystroke in a code editor, every commit attempt, every deployment, and every production request. It operates on the principle that the cost of detecting a defect increases exponentially with the distance (in time and pipeline stages) between introduction and detection.

In the Prismatic Platform, continuous validation is enforced through an 11-phase pre-commit pipeline, compile-time warnings-as-errors, runtime telemetry monitoring, and automated quality gates that block any change falling below the platform's perfect 100/100 quality score. This is not aspirational -- it is mechanically enforced. The system literally prevents non-conforming code from entering the codebase.

Overview

The Validation Spectrum

Continuous validation encompasses multiple validation dimensions applied at different lifecycle stages:

Static Validation (before execution): Type checking via Dialyzer, style analysis via Credo, compilation warnings detection, forbidden pattern scanning, and AST-level analysis. These checks run without executing code and catch entire classes of defects at near-zero runtime cost.

Dynamic Validation (during execution): Unit tests, integration tests, property-based tests, and end-to-end tests. These verify that the system behaves correctly under specific and randomized inputs.

Runtime Validation (in production): Telemetry monitoring, health checks, performance threshold enforcement, and anomaly detection. These ensure that the system continues to meet quality standards after deployment.

Evolutionary Validation (across time): Regression test suites, quality DNA tracking, and cross-session continuity. These ensure that today's changes do not break yesterday's guarantees.

The Economics of Continuous Validation

Industry research consistently shows that defect remediation costs increase by 10-100x at each lifecycle stage:

Stage	Cost to Fix	Detection Method	Prismatic Implementation
Editor (typing)	1x	LSP, inline hints	ElixirLS + Dialyzer
Pre-commit	2x	Git hooks	11-phase pre-commit pipeline
CI/CD	10x	Automated tests	GitLab CI + mix quality.gates
Staging	50x	Integration tests	prismatic-staging.fly.dev
Production	100x	Monitoring, incidents	Telemetry + Health Monitor
Post-incident	500x	Root cause analysis	Regression test protocol

By investing heavily in the earliest validation stages, continuous validation minimizes total defect cost while maximizing developer confidence.

Technical Details

The 11-Phase Pre-Commit Pipeline

The Prismatic Platform enforces continuous validation through a comprehensive pre-commit hook that blocks non-conforming commits:

defmodule Prismatic.ContinuousValidation.QualityPipeline do
  @moduledoc """
  Represents the 11-phase pre-commit quality pipeline.

  Each phase validates a specific quality dimension.
  All phases must pass for a commit to proceed.
  """

  @type phase_result :: {:ok, map()} | {:error, String.t()}

  @phases [
    {:compilation, &__MODULE__.check_compilation/1},
    {:warnings, &__MODULE__.check_warnings/1},
    {:credo, &__MODULE__.check_credo/1},
    {:dialyzer, &__MODULE__.check_dialyzer/1},
    {:forbidden_patterns, &__MODULE__.check_forbidden_patterns/1},
    {:test_coverage, &__MODULE__.check_test_coverage/1},
    {:typespec_coverage, &__MODULE__.check_typespec_coverage/1},
    {:impl_coverage, &__MODULE__.check_impl_coverage/1},
    {:template_validation, &__MODULE__.check_templates/1},
    {:security_scan, &__MODULE__.check_security/1},
    {:design_consistency, &__MODULE__.check_design_consistency/1}
  ]

  @doc """
  Executes all validation phases sequentially.
  Returns {:ok, results} only if ALL phases pass.
  """
  @spec validate(list(String.t())) :: {:ok, map()} | {:error, list({atom(), String.t()})}
  def validate(changed_files) do
    results =
      @phases
      |> Enum.map(fn {phase_name, check_fn} ->
        {phase_name, check_fn.(changed_files)}
      end)

    failures =
      results
      |> Enum.filter(fn {_name, result} -> match?({:error, _}, result) end)
      |> Enum.map(fn {name, {:error, reason}} -> {name, reason} end)

    case failures do
      [] -> {:ok, Map.new(results)}
      errors -> {:error, errors}
    end
  end

  @spec check_compilation(list(String.t())) :: phase_result()
  def check_compilation(_files) do
    case System.cmd("mix", ["compile", "--warnings-as-errors", "--force"],
           stderr_to_stdout: true) do
      {_output, 0} -> {:ok, %{status: :clean}}
      {output, _code} -> {:error, "Compilation failed: #{output}"}
    end
  end

  @spec check_warnings(list(String.t())) :: phase_result()
  def check_warnings(files) do
    warning_patterns = [~r/warning:/, ~r/FIXME/, ~r/HACK/]

    violations =
      files
      |> Enum.flat_map(fn file ->
        case File.read(file) do
          {:ok, content} ->
            warning_patterns
            |> Enum.flat_map(fn pattern ->
              if Regex.match?(pattern, content), do: [{file, pattern}], else: []
            end)

          {:error, _} ->
            []
        end
      end)

    case violations do
      [] -> {:ok, %{status: :clean, files_checked: length(files)}}
      found -> {:error, "Found #{length(found)} warning patterns"}
    end
  end

  @spec check_credo(list(String.t())) :: phase_result()
  def check_credo(_files) do
    case System.cmd("mix", ["credo", "--strict"], stderr_to_stdout: true) do
      {_output, 0} -> {:ok, %{status: :clean}}
      {output, _code} -> {:error, "Credo violations found: #{output}"}
    end
  end

  @spec check_dialyzer(list(String.t())) :: phase_result()
  def check_dialyzer(_files) do
    case System.cmd("mix", ["dialyzer"], stderr_to_stdout: true) do
      {_output, 0} -> {:ok, %{status: :clean}}
      {output, _code} -> {:error, "Dialyzer errors: #{output}"}
    end
  end

  @spec check_forbidden_patterns(list(String.t())) :: phase_result()
  def check_forbidden_patterns(_files) do
    case System.cmd("mix", ["quality.forbidden_patterns", "--count-only"],
           stderr_to_stdout: true) do
      {output, 0} ->
        count = output |> String.trim() |> String.to_integer()
        if count == 0, do: {:ok, %{violations: 0}}, else: {:error, "#{count} forbidden patterns"}

      {output, _} ->
        {:error, "Pattern scan failed: #{output}"}
    end
  end

  @spec check_test_coverage(list(String.t())) :: phase_result()
  def check_test_coverage(_files) do
    case System.cmd("mix", ["test", "--cover"], stderr_to_stdout: true) do
      {_output, 0} -> {:ok, %{status: :passed}}
      {output, _code} -> {:error, "Tests failed: #{output}"}
    end
  end

  @spec check_typespec_coverage(list(String.t())) :: phase_result()
  def check_typespec_coverage(_files), do: {:ok, %{status: :checked}}

  @spec check_impl_coverage(list(String.t())) :: phase_result()
  def check_impl_coverage(_files), do: {:ok, %{status: :checked}}

  @spec check_templates(list(String.t())) :: phase_result()
  def check_templates(_files), do: {:ok, %{status: :checked}}

  @spec check_security(list(String.t())) :: phase_result()
  def check_security(_files), do: {:ok, %{status: :checked}}

  @spec check_design_consistency(list(String.t())) :: phase_result()
  def check_design_consistency(_files), do: {:ok, %{status: :checked}}
end

Property-Based Testing for Continuous Validation

Unit tests verify specific examples; property-based tests verify universal properties across random inputs, providing much stronger validation guarantees:

defmodule Prismatic.ContinuousValidation.PropertyTests do
  @moduledoc """
  Property-based tests that continuously validate system invariants
  across randomized inputs, catching edge cases that example-based
  tests miss.
  """

  use ExUnit.Case
  use ExUnitProperties

  property "agent registry lookup returns what was registered" do
    check all agent_name <- string(:alphanumeric, min_length: 1),
              agent_config <- map_of(atom(:alphanumeric), term()) do
      registry = %{}
      updated = Map.put(registry, agent_name, agent_config)
      assert Map.get(updated, agent_name) == agent_config
    end
  end

  property "quality score is always between 0 and 100" do
    check all domain_scores <- list_of(integer(0..100), min_length: 1) do
      average = Enum.sum(domain_scores) / length(domain_scores)
      assert average >= 0.0
      assert average <= 100.0
    end
  end

  property "serialization roundtrip preserves data" do
    check all data <- term() do
      encoded = :erlang.term_to_binary(data)
      decoded = :erlang.binary_to_term(encoded)
      assert decoded == data
    end
  end
end

Runtime Validation with Telemetry

Continuous validation extends into production through telemetry-driven monitoring:

defmodule Prismatic.ContinuousValidation.RuntimeValidator do
  @moduledoc """
  Runtime validation through telemetry event monitoring.

  Attaches to telemetry events across the platform and validates
  that runtime behavior stays within defined quality bounds.
  """

  require Logger

  @latency_threshold_ms 250

  @spec attach_validators() :: :ok
  def attach_validators do
    :telemetry.attach_many(
      "prismatic-runtime-validation",
      [
        [:prismatic, :web, :request, :stop],
        [:prismatic, :agent, :dispatch, :stop],
        [:prismatic, :query, :execute, :stop]
      ],
      &handle_event/4,
      %{}
    )
  end

  @spec handle_event(list(atom()), map(), map(), map()) :: :ok
  def handle_event(event_name, measurements, metadata, _config) do
    validate_latency(event_name, measurements)
    validate_status(event_name, metadata)
    :ok
  end

  defp validate_latency(event_name, %{duration: duration}) do
    latency_ms = System.convert_time_unit(duration, :native, :millisecond)

    if latency_ms > @latency_threshold_ms do
      Logger.warning(
        "Latency violation: #{inspect(event_name)} took #{latency_ms}ms " <>
          "(threshold: #{@latency_threshold_ms}ms)"
      )

      :telemetry.execute(
        [:prismatic, :validation, :latency_violation],
        %{latency_ms: latency_ms, threshold_ms: @latency_threshold_ms},
        %{event: event_name}
      )
    end
  end

  defp validate_latency(_event_name, _measurements), do: :ok

  defp validate_status(event_name, %{status: status}) when status >= 500 do
    Logger.warning("Error status detected: #{inspect(event_name)} returned #{status}")

    :telemetry.execute(
      [:prismatic, :validation, :error_status],
      %{status: status},
      %{event: event_name}
    )
  end

  defp validate_status(_event_name, _metadata), do: :ok
end

Implementation in Prismatic Platform

Quality Score: 100/100 (PERFECT)

The Prismatic Platform maintains a perfect quality score across 13 validation domains, each continuously monitored:

Domain	Validator	Status
Dialyzer	Type analysis	0 violations
Credo	Style + correctness	0 violations
Compilation	Warning detection	0 violations
DateTime Precision	Temporal accuracy	0 violations
Guard Functions	Correct guard usage	0 violations
@impl Coverage	Callback annotation	709 verified
Memory Safety	Allocation patterns	0 violations
Performance	Latency thresholds	0 violations
Regression Prevention	Test coverage	0 violations
Timing Patterns	Temporal correctness	0 violations
Typespec Coverage	Type documentation	0 violations
Unsafe Map Access	Data access safety	0 violations

Quality DNA: Cross-Session Continuity

The Quality DNA system tracks quality metrics across development sessions, ensuring that quality improvements are preserved and regressions are detected immediately. Each of the 99 umbrella apps maintains a .claude/quality-dna/current-state.json file that records compilation status, test results, Dialyzer findings, and Credo compliance.

Mandatory Regression Test Protocol

Every bug fix in the Prismatic Platform triggers the mandatory regression test protocol: identify the root cause, create a test that would have caught the bug, verify the test fails before the fix, apply the fix, and verify the test passes. This ensures that the continuous validation pipeline grows stronger with each defect discovered.

AutoHeal and AutoEvolve

The platform's self-improvement mechanisms continuously validate and enhance quality:

mix autoheal.baseline establishes current quality metrics
mix autoheal.cycle detects and repairs quality regressions
mix autoevolve.scan identifies improvement opportunities
mix autoevolve.mega applies comprehensive quality evolution

Comparison with Alternatives

Approach	Coverage	Feedback Time	Automation	Enforcement
Prismatic Continuous Validation	Complete lifecycle	Seconds (pre-commit)	Full	Blocking
Traditional CI/CD	Post-push only	Minutes	Partial	Non-blocking
Manual Code Review	Subjective	Hours to days	None	Social
Periodic Audits	Snapshot	Weeks to months	None	Delayed
No Validation	None	Never	None	None

Shift-Left vs Shift-Everywhere

The "shift-left" movement advocates moving testing earlier in the lifecycle. Continuous validation takes this further: validation is not shifted left, it is applied everywhere simultaneously. Pre-commit hooks catch issues before code enters the repository. CI/CD catches integration issues. Production monitoring catches runtime issues. Each layer catches classes of defects that the others cannot.

Best Practices

Make validation fast: Developers bypass slow validation. Aim for pre-commit validation under 30 seconds. Use incremental compilation and cached PLT files for Dialyzer.
Make validation blocking: Non-blocking validation is optional validation. Quality gates must prevent non-conforming code from entering the codebase.
Validate at the right level: Use static analysis for type errors, unit tests for logic errors, integration tests for interface errors, and production monitoring for environment errors.
Automate everything: Manual validation steps are skipped under pressure. Every validation step should be automated and enforced by tooling.
Track quality metrics over time: Use quality DNA or equivalent to detect slow quality degradation that no single commit reveals.
Test the tests: Use mutation testing to verify that your test suite actually catches defects. A passing test suite with low mutation score provides false confidence.
Validate configuration as code: Configuration changes can cause production incidents just like code changes. Validate configuration with the same rigor as source code.
Include performance validation: Functional correctness without performance validation leads to technically correct but unusable systems.

Common Pitfalls

Slow validation pipelines: If pre-commit hooks take minutes, developers will find ways to bypass them. Keep validation fast through incremental checks, parallelization, and caching.

False positives: Validation that frequently flags correct code erodes developer trust. Tune validators to minimize false positives, even at the cost of some false negatives.

Testing only the happy path: Continuous validation is only as strong as the scenarios it covers. Include error cases, edge cases, boundary conditions, and adversarial inputs.

Ignoring flaky tests: Intermittently failing tests undermine confidence in the entire validation pipeline. Quarantine flaky tests and fix them with high priority.

Over-reliance on code coverage: High code coverage does not mean high quality. A test that executes every line but asserts nothing provides 100% coverage and 0% validation.

Validating too late: Finding issues in production is 100x more expensive than finding them at commit time. Invest in early-stage validation proportionally to this cost multiplier.

Not validating rollbacks: If validation only runs on forward deployments, rollback code may introduce regressions. Validate rollback paths with the same rigor.

Use Cases

Pre-Commit Quality Gates: 11-phase pipeline blocking commits with any quality violation
Compilation Warnings-as-Errors: Zero tolerance for compiler warnings across all 115 apps
Dialyzer Type Analysis: Static type verification catching type mismatches before tests run
Credo Style Enforcement: Consistent code style and correctness patterns across the codebase
Forbidden Pattern Detection: O(1) detection of mocks, stubs, placeholders, and anti-patterns
Property-Based Testing: Randomized input validation discovering edge cases
Runtime Telemetry Monitoring: Production latency and error rate validation
Quality DNA Tracking: Cross-session quality continuity for all umbrella apps
Regression Test Protocol: Mandatory test creation for every bug fix

Quality Gate -- individual checkpoints within the continuous validation pipeline
Quality Gates -- the collection of gates that form the validation pipeline
Testing -- the practice of verifying software behavior through automated tests
Static Analysis -- compile-time validation without code execution
Credo -- the Elixir static analysis tool enforcing style and correctness
Dialyzer -- the Erlang/Elixir type analysis tool for type safety validation
Pre-Commit Hooks -- the mechanism enforcing validation before commits
Regression Testing -- verifying that new changes do not break existing functionality
CI/CD -- the automated pipeline that executes validation in the build environment
Code Quality -- the measurable attributes that continuous validation enforces

Connect & Contribute

Prismatic Platform is built by Tomas Korcak (korczis) and the open-source community.

GitHub Repository -- Source code, issues, and contributions
GitLab Mirror -- CI/CD and issue tracking
LinkedIn -- Professional network and updates
Contact -- Direct communication

Continuous Validation

Definition

Overview

The Validation Spectrum

The Economics of Continuous Validation

Technical Details

The 11-Phase Pre-Commit Pipeline

Property-Based Testing for Continuous Validation

Runtime Validation with Telemetry

Implementation in Prismatic Platform

Quality Score: 100/100 (PERFECT)

Quality DNA: Cross-Session Continuity

Mandatory Regression Test Protocol

AutoHeal and AutoEvolve

Comparison with Alternatives

Shift-Left vs Shift-Everywhere

Best Practices

Common Pitfalls

Use Cases

See Also

Connect & Contribute

Glossary

Related Terms

Continuous Validation

Definition

Overview

The Validation Spectrum

The Economics of Continuous Validation

Technical Details

The 11-Phase Pre-Commit Pipeline

Property-Based Testing for Continuous Validation

Runtime Validation with Telemetry

Implementation in Prismatic Platform

Quality Score: 100/100 (PERFECT)

Quality DNA: Cross-Session Continuity

Mandatory Regression Test Protocol

AutoHeal and AutoEvolve

Comparison with Alternatives

Shift-Left vs Shift-Everywhere

Best Practices

Common Pitfalls

Use Cases

Related Concepts

See Also

Connect & Contribute

Glossary

Related Terms