Quality Systems

architecture

Quality Systems

Definition and Overview

Quality Systems refers to the integrated architecture of interconnected subsystems that collectively enforce, monitor, evolve, and persist software quality across the entire lifecycle of a software platform. In the Prismatic Platform, the quality systems architecture comprises a coordinated network of specialized components -- the Quality Standard, Quality Gates, Quality Monitoring, Quality DNA, Quality Floor Guardian, Quality Debt tracker, AutoHeal, AutoEvolve, and SEADF -- each responsible for a distinct quality function, yet operating together as a unified whole through shared telemetry, common data formats, and OTP-based supervision hierarchies.

The fundamental insight behind quality systems architecture is that no single tool, check, or process can ensure quality at scale. Compilation checks catch syntax and type errors but miss style violations. Credo catches style issues but misses runtime behavior. Tests verify behavior but miss performance regressions. Coverage tools measure test extent but not test quality. Only by composing these individual capabilities into a coherent system -- where each component's output feeds into other components' inputs, where gaps in one component's coverage are compensated by another's strengths, and where the system as a whole is monitored for its own health -- can quality be maintained across a platform of 115 applications and approximately 2.8 million lines of code.

The Prismatic Quality Systems architecture is distinguished by three properties that set it apart from conventional quality toolchains: autonomy (the system detects and responds to quality issues without human intervention), persistence (quality state survives session boundaries and accumulates knowledge over time), and evolution (the system improves its own capabilities across platform generations). These properties emerge from the interaction of the constituent subsystems rather than being implemented in any single component.

System-of-Systems Architecture

Component Topology

The quality systems architecture follows a layered topology where each layer depends on the layers below it:

EVOLUTION LAYER
  AutoEvolve | SEADF | AutoHeal | Trend Engine

PERSISTENCE LAYER
  Quality DNA | Session Context | Metric Store

ENFORCEMENT LAYER
  Quality Gates | Pre-Commit | CI Pipeline | Prod

MONITORING LAYER
  Telemetry | Quality Floor Guardian | Alert System

STANDARDS LAYER
  Quality Standard | Credo Rules | Forbidden Patterns

FOUNDATION LAYER
  Dialyzer | Compiler | ExUnit | Mix Tasks

OTP Supervision Tree

The quality systems are organized under an OTP supervision tree that ensures fault tolerance and ordered startup:

defmodule Prismatic.Quality.Supervisor do
  @moduledoc """
  Top-level supervisor for the quality systems architecture.
  Manages all quality subsystems with appropriate restart strategies.
  """

  use Supervisor

  @impl true
  def start_link(opts) do
    Supervisor.start_link(__MODULE__, opts, name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    children = [
      # Foundation: metric storage (must start first)
      {Prismatic.Quality.MetricStore, []},

      # Standards: configuration and rule definitions
      {Prismatic.Quality.Standard.Registry, []},

      # Monitoring: telemetry handlers and collectors
      {Prismatic.Quality.Monitor.Supervisor, []},

      # Enforcement: gate evaluation and blocking
      {Prismatic.Quality.GateKeeper, []},

      # Persistence: DNA loading and saving
      {Prismatic.Quality.DNA.Manager, []},

      # Guardian: autonomous quality floor protection
      {Prismatic.Quality.FloorGuardian, []},

      # Evolution: trend analysis and improvement recommendations
      {Prismatic.Quality.EvolutionEngine, []},

      # Alert: escalation and notification routing
      {Prismatic.Quality.AlertRouter, []}
    ]

    Supervisor.init(children, strategy: :rest_for_one)
  end
end

The rest_for_one strategy ensures that if a lower-level subsystem crashes, all subsystems that depend on it are also restarted in the correct order. For example, if the MetricStore crashes, the Monitor, GateKeeper, Guardian, and Evolution subsystems are all restarted because they depend on metric data.

Subsystem Deep Dive

Standards Subsystem

The standards subsystem defines what quality means. It maintains the 8-dimension scoring framework, custom credo rules, forbidden pattern definitions, and domain-specific quality policies:

defmodule Prismatic.Quality.Standard.Registry do
  @moduledoc """
  Registry of all quality standards, rules, and policies.
  Provides a unified interface for querying quality requirements.
  """

  use GenServer

  @type standard :: %{
    name: atom(),
    dimensions: [dimension()],
    thresholds: %{atom() => number()},
    policies: [policy()]
  }

  @type dimension :: %{
    name: atom(),
    max_score: float(),
    checker: (String.t() -> boolean()),
    auto_fixable: boolean()
  }

  @type policy :: %{
    name: atom(),
    severity: :info | :warning | :critical,
    patterns: [Regex.t()],
    whitelist: [String.t()]
  }

  @impl true
  def init(_opts) do
    state = %{
      standards: load_standards(),
      policies: load_policies(),
      forbidden_patterns: load_forbidden_patterns(),
      custom_credo_rules: load_credo_rules()
    }

    {:ok, state}
  end

  @spec get_standard(atom()) :: {:ok, standard()} | {:error, :not_found}
  def get_standard(name) do
    GenServer.call(__MODULE__, {:get_standard, name})
  end

  @spec all_requirements(atom()) :: [map()]
  def all_requirements(app_name) do
    GenServer.call(__MODULE__, {:all_requirements, app_name})
  end
end

Monitoring Subsystem

The monitoring subsystem observes what is happening. It collects telemetry from every quality-related operation, aggregates metrics, detects anomalies, and feeds data to downstream subsystems:

defmodule Prismatic.Quality.Monitor.Supervisor do
  @moduledoc """
  Supervisor for all monitoring components.
  Manages collectors, aggregators, and anomaly detectors.
  """

  use Supervisor

  @impl true
  def init(_opts) do
    children = [
      # Domain-specific collectors
      {Prismatic.Quality.Monitor.CompilationCollector, []},
      {Prismatic.Quality.Monitor.CredoCollector, []},
      {Prismatic.Quality.Monitor.DialyzerCollector, []},
      {Prismatic.Quality.Monitor.TestCollector, []},
      {Prismatic.Quality.Monitor.CoverageCollector, []},
      {Prismatic.Quality.Monitor.PerformanceCollector, []},

      # Cross-domain aggregation
      {Prismatic.Quality.Monitor.Aggregator, []},

      # Statistical anomaly detection
      {Prismatic.Quality.Monitor.AnomalyDetector, []}
    ]

    Supervisor.init(children, strategy: :one_for_one)
  end
end

Enforcement Subsystem

The enforcement subsystem acts on what monitoring observes. It evaluates quality gates, blocks non-compliant operations, and ensures that no code enters the platform without meeting established standards:

defmodule Prismatic.Quality.GateKeeper do
  @moduledoc """
  Central enforcement point for quality gate evaluation.
  Receives monitoring data and makes pass/fail decisions.
  """

  use GenServer

  @type gate_result :: :pass | :warn | :block | :halt

  @impl true
  def init(_opts) do
    state = %{
      active_blocks: %{},
      gate_history: [],
      enforcement_mode: :strict
    }

    {:ok, state}
  end

  @spec evaluate_gate(atom(), map()) :: gate_result()
  def evaluate_gate(gate_name, context) do
    GenServer.call(__MODULE__, {:evaluate, gate_name, context})
  end

  @impl true
  def handle_call({:evaluate, gate_name, context}, _from, state) do
    result = do_evaluate(gate_name, context, state)

    :telemetry.execute(
      [:prismatic, :quality, :gate, :evaluated],
      %{},
      %{gate: gate_name, result: result}
    )

    updated_state = record_evaluation(state, gate_name, result)
    {:reply, result, updated_state}
  end

  defp do_evaluate(gate_name, context, state) do
    standards = Prismatic.Quality.Standard.Registry.get_gate_requirements(gate_name)

    violations =
      standards
      |> Enum.filter(fn requirement ->
        not requirement_met?(requirement, context)
      end)

    classify_violations(violations, state.enforcement_mode)
  end

  defp classify_violations([], _mode), do: :pass
  defp classify_violations(violations, :strict) do
    if Enum.any?(violations, &(&1.severity == :critical)), do: :halt, else: :block
  end
end

Persistence Subsystem

The persistence subsystem remembers what has happened. Through Quality DNA and session context, it maintains cross-session quality state that enables trend analysis, regression detection, and continuous improvement:

defmodule Prismatic.Quality.DNA.Manager do
  @moduledoc """
  Manages Quality DNA lifecycle: loading, updating, and persisting.
  Ensures quality state survives across development sessions.
  """

  use GenServer

  @impl true
  def init(_opts) do
    dna_map = load_all_dna()

    :telemetry.execute(
      [:prismatic, :quality, :dna, :loaded],
      %{app_count: map_size(dna_map)},
      %{}
    )

    schedule_periodic_save()

    {:ok, %{dna_map: dna_map, dirty: MapSet.new()}}
  end

  @spec update_domain(atom(), atom(), map()) :: :ok
  def update_domain(app, domain, metrics) do
    GenServer.cast(__MODULE__, {:update_domain, app, domain, metrics})
  end

  @impl true
  def handle_cast({:update_domain, app, domain, metrics}, state) do
    updated_dna = update_app_dna(state.dna_map, app, domain, metrics)
    dirty = MapSet.put(state.dirty, app)

    {:noreply, %{state | dna_map: updated_dna, dirty: dirty}}
  end

  @impl true
  def handle_info(:periodic_save, state) do
    save_dirty_dna(state.dna_map, state.dirty)
    schedule_periodic_save()

    {:noreply, %{state | dirty: MapSet.new()}}
  end

  defp schedule_periodic_save do
    Process.send_after(self(), :periodic_save, :timer.minutes(5))
  end
end

Evolution Subsystem

The evolution subsystem improves the quality systems themselves. It analyzes trends, identifies improvement opportunities, and recommends or autonomously implements quality enhancements:

defmodule Prismatic.Quality.EvolutionEngine do
  @moduledoc """
  Drives autonomous quality evolution across platform generations.
  Analyzes DNA trends, identifies opportunities, and orchestrates improvements.
  """

  use GenServer

  @type evolution_action :: %{
    type: :recommendation | :auto_fix | :investigation,
    target: {atom(), atom()},
    priority: float(),
    description: String.t(),
    estimated_impact: float()
  }

  @impl true
  def init(_opts) do
    state = %{
      pending_actions: [],
      completed_actions: [],
      generation: current_generation(),
      fitness: current_fitness()
    }

    {:ok, state}
  end

  @spec analyze_and_recommend() :: [evolution_action()]
  def analyze_and_recommend do
    GenServer.call(__MODULE__, :analyze_and_recommend)
  end

  @impl true
  def handle_call(:analyze_and_recommend, _from, state) do
    dna_map = Prismatic.Quality.DNA.Manager.get_all()

    actions =
      dna_map
      |> analyze_regressions()
      |> Kernel.++(analyze_improvement_opportunities(dna_map))
      |> Kernel.++(analyze_cross_domain_patterns(dna_map))
      |> prioritize_actions()
      |> Enum.take(20)

    {:reply, actions, %{state | pending_actions: actions}}
  end

  defp analyze_regressions(dna_map) do
    dna_map
    |> Enum.flat_map(fn {app, dna} ->
      dna.domains
      |> Enum.filter(fn {_domain, record} -> record.trend == :regressing end)
      |> Enum.map(fn {domain, record} ->
        %{
          type: :investigation,
          target: {app, domain},
          priority: 10.0 - record.score,
          description: "Regression detected in #{app}/#{domain}",
          estimated_impact: record.score * 0.1
        }
      end)
    end)
  end
end

Cross-System Data Flow

The Quality Feedback Loop

The quality systems form a continuous feedback loop where each subsystem's output feeds into other subsystems' inputs:

Standards --define--> Gates --evaluate--> Monitoring --observe--> DNA --persist-->
    ^                                                                             |
    |                                                                             |
    +---------- Evolution --analyze--> Recommendations --improve------------------+

This loop operates at multiple timescales:

TimescaleLoopSubsystems Involved
MillisecondsFile-save feedbackMonitor to Local checks
SecondsPre-commit gateStandards to Gates to Monitor
MinutesCI pipelineAll subsystems
HoursSession lifecycleDNA to Evolution to Standards
DaysTrend analysisDNA history to Evolution to Standards update
GenerationsPlatform evolutionSEADF to All subsystems

Telemetry Event Bus

All quality subsystems communicate through a shared telemetry event bus. This decoupled architecture allows subsystems to evolve independently while maintaining system-wide visibility:

defmodule Prismatic.Quality.EventBus do
  @moduledoc """
  Quality-specific event bus built on :telemetry.
  Provides typed event definitions and routing for quality subsystems.
  """

  @event_catalog %{
    standard_evaluated: [:prismatic, :quality, :standard, :evaluated],
    gate_passed: [:prismatic, :quality, :gate, :passed],
    gate_blocked: [:prismatic, :quality, :gate, :blocked],
    metric_collected: [:prismatic, :quality, :metric, :collected],
    anomaly_detected: [:prismatic, :quality, :anomaly, :detected],
    dna_updated: [:prismatic, :quality, :dna, :updated],
    regression_detected: [:prismatic, :quality, :regression, :detected],
    evolution_recommended: [:prismatic, :quality, :evolution, :recommended],
    floor_violation: [:prismatic, :quality, :floor, :violation],
    debt_eliminated: [:prismatic, :quality, :debt, :eliminated]
  }

  @spec emit(atom(), map(), map()) :: :ok
  def emit(event_name, measurements, metadata) do
    event_path = Map.fetch!(@event_catalog, event_name)
    :telemetry.execute(event_path, measurements, metadata)
  end

  @spec subscribe(atom(), (list(), map(), map(), any() -> any())) :: :ok
  def subscribe(event_name, handler) do
    event_path = Map.fetch!(@event_catalog, event_name)
    :telemetry.attach(
      "quality-#{event_name}-#{System.unique_integer()}",
      event_path,
      handler,
      %{}
    )
  end
end

Quality Systems Metrics

System Health Dashboard

The quality systems themselves are monitored for health, creating a meta-quality layer:

System MetricTargetCurrentStatus
Overall quality score100/100100/100OPTIMAL
Quality domains passing13/1313/13OPTIMAL
Compilation warnings00OPTIMAL
Quality debt points00OPTIMAL (eliminated)
QDP elimination rate100%100%COMPLETE
App DNA coverage100%100%COMPLETE
Generation1919Current
Platform fitness0.9990.9995Exceeding

Quality Floor Guardian Enforcement Levels

The guardian operates at graduated enforcement levels based on the aggregate quality score:

Score RangeLevelResponse
100-99%OPTIMALMonitor only
98-99%WARNINGAlert + investigation
95-98%CRITICALAuto-evolution trigger
Below 95%EMERGENCYBlock commits + escalate

Historical Evolution

Generational Progress

The quality systems architecture did not emerge fully formed. It evolved across 19 generations of the Prismatic Platform, with each generation adding capabilities and refinements:

GenerationQuality Systems Milestone
Gen 1-3Basic compilation checks, manual Credo runs
Gen 4-6Pre-commit hooks, initial quality gates
Gen 7-9Quality DNA introduced, cross-session persistence
Gen 10-12AutoHeal, automated quality debt elimination
Gen 13-15Quality Floor Guardian, anomaly detection
Gen 16-17SEADF integration, ecosystem evolution
Gen 18-19100/100 score achieved, 0 QDP, full automation

This evolutionary trajectory demonstrates a key principle: quality systems that are designed as a monolithic whole are fragile. Quality systems that evolve incrementally, adding one capability at a time and validating each addition against the existing system, are robust. The OTP supervision tree model supports this incremental evolution naturally, as new child processes can be added to existing supervisors without restructuring the tree.

Lessons from 905 QDP Elimination

The systematic elimination of 905 quality debt points across the platform's history produced valuable lessons for quality systems design. The most significant was that quality debt elimination is not a one-time event but a continuous process requiring dedicated subsystems. The AutoHeal system was created specifically to automate the detection and resolution of quality regressions, preventing the re-accumulation of debt that manual processes had allowed.

Usage in Prismatic Platform

Commands and Workflows

# Full quality systems check
mix quality.gates

# Quick systems health check
mix quality.gates.check --fast

# Quality standard compliance
mix quality.enforce_standard

# Forbidden patterns scan
mix quality.forbidden_patterns

# Auto-heal quality issues
mix autoheal.cycle

# Evolution status
mix autoevolve status --brief

# Full evolution scan
mix autoevolve.mega

# SEADF ecosystem status
mix seadf status --verbose

Operational Runbook

When a quality system issue is detected:

  1. Identify the affected subsystem using mix quality.gates output
  2. Check Quality DNA for trend data showing when the regression began
  3. Review recent changes using git log for the affected application
  4. Run targeted checks on the specific domain (mix credo --strict, mix dialyzer, etc.)
  5. Apply fix and verify the quality gate passes
  6. Commit with regression test per mandatory regression test protocol
  7. Verify DNA update shows the fix reflected in quality state

Integration with Platform Doctrine

The quality systems architecture is deeply integrated with the platform's philosophical and operational doctrines. The No Mercy, No Doubts doctrine mandates that quality enforcement is non-negotiable and non-bypassable. The NABLA Infinity epistemic framework requires that quality claims be evidence-based and independently verifiable. The Trinity Gate demands structural, logical, and formal consistency before any quality assertion is accepted. These doctrines are not external constraints imposed on the quality systems -- they are the philosophical foundations from which the quality systems were designed.

The interaction between quality systems and the doctrine framework creates a closed loop of accountability. Doctrines define the principles. Quality standards translate principles into measurable criteria. Quality gates enforce criteria automatically. Quality DNA tracks compliance over time. The evolution engine identifies opportunities to strengthen enforcement. And the doctrines themselves are subject to the same evidence-based evaluation that they mandate for everything else.

Best Practices

  1. Treat the quality system as a product. The quality infrastructure deserves the same engineering rigor as the features it protects. Test the quality tools, monitor the monitors, and evolve the evolution engine.

  2. Maintain subsystem independence. Each quality subsystem should be independently deployable, testable, and recoverable. Tight coupling between subsystems creates fragility that undermines the reliability the system is meant to provide.

  3. Use the OTP supervision tree. Quality subsystems should be supervised OTP processes with appropriate restart strategies. A crash in the anomaly detector should not take down the gate enforcement system.

  4. Prioritize feedback speed. The value of quality systems is proportional to how quickly they provide feedback. Optimize the critical path (file save to feedback) ruthlessly.

  5. Document quality system architecture. The meta-architecture of the quality system itself must be documented as thoroughly as the application architecture it protects.

  6. Evolve the system incrementally. Quality systems that change too rapidly destabilize the platform. Apply the same rigor to quality system changes as to application changes: test, review, gate, deploy.

Common Pitfalls

  • Quality system sprawl: Adding new quality checks without considering their interaction with existing checks creates redundancy, conflicting requirements, and alert fatigue. Each new check should be evaluated for its unique contribution to the system.

  • Neglecting system performance: Quality checks that slow development workflows will be circumvented. Monitor the performance of quality systems and optimize aggressively to maintain fast feedback loops.

  • Over-centralization: A single monolithic quality system is fragile. Distribute quality responsibilities across independent subsystems that can fail and recover independently.

  • Under-instrumentation: Quality systems without their own telemetry are blind to their own failures. Instrument every subsystem and monitor the quality system's health alongside application health.

  • Static standards: Quality standards that never evolve become irrelevant as the platform changes. Schedule periodic standard reviews and evolve requirements based on observed failure patterns and industry best practices.

  • Ignoring cross-system interactions: Quality subsystems that are tested in isolation but never validated together miss integration issues. The telemetry event bus should be tested end-to-end to verify that events flow correctly between subsystems.

See Also

  • Architecture -- Platform architecture overview
  • Technologies -- Technology stack details
  • Apps -- Application directory with quality system integration

Connect & Contribute

Created by Tomas Korcak (korczis) | Open Source under GHL

Back to Glossary