Quality and Transparency

quality

Quality and Transparency

Definition

Quality and transparency is the architectural principle and operational practice of making every quality metric, enforcement decision, and measurement outcome fully visible, auditable, and traceable across all layers of a software platform. In the context of the Prismatic Platform, quality transparency means that no quality gate operates in secret, no metric is hidden from stakeholders, and every enforcement action leaves an immutable audit trail. This principle stands in direct opposition to "quality theater" -- the practice of running checks that nobody inspects, generating reports that nobody reads, and enforcing standards that nobody can verify.

Quality transparency encompasses three interconnected dimensions: metric visibility (all measurements are accessible to every team member), decision auditability (every pass/fail decision can be traced to specific evidence), and reporting openness (quality reports are generated automatically and shared without gatekeeping). Together, these dimensions ensure that quality is not merely claimed but demonstrably proven.

The Prismatic Platform achieves quality transparency through a layered architecture that instruments every compilation, test run, static analysis pass, and deployment gate with structured telemetry events. These events flow into centralized reporting pipelines that produce dashboards, session reports, and trend analyses -- all visible to every contributor.

Historical Context and Motivation

The need for quality transparency emerged from repeated failures in the software industry where quality was treated as a checkbox rather than a continuous, observable property. Projects would pass CI pipelines while accumulating hidden technical debt, achieve "green builds" while ignoring compilation warnings, and report high test coverage while leaving critical paths untested. The root cause was always the same: quality metrics were collected but not made transparent.

The academic literature on software quality, particularly the work on GQM (Goal-Question-Metric) frameworks by Basili and Weiss, established that metrics without transparency create perverse incentives. Teams optimize for the metric rather than the underlying quality property. The Prismatic Platform addresses this by making the entire quality enforcement chain -- from raw measurement to final decision -- visible and auditable.

In the Elixir ecosystem, the BEAM VM's built-in observability primitives (process introspection, :telemetry events, ETS-based registries) provide a natural foundation for quality transparency. The Prismatic Platform extends these primitives with domain-specific quality instrumentation that covers 13 quality domains, each with its own transparency pipeline.

Core Principles

Metric Provenance

Every quality metric in the Prismatic Platform carries provenance metadata: when it was measured, by which tool, against which version of the codebase, and under what configuration. This provenance chain ensures that no metric exists without traceable origin, satisfying the NABLA Infinity axiom of Provenance Mandatory.

Decision Transparency

When a quality gate blocks a commit or deployment, the decision is recorded with full context: which checks failed, what thresholds were violated, what the measured values were, and what remediation is required. Developers never face opaque "quality check failed" messages -- they see exactly what failed and why.

Stakeholder Accessibility

Quality reports are not locked behind specialized tooling or restricted dashboards. The Prismatic Platform generates quality reports in multiple formats (terminal output, JSON, HTML dashboards) and distributes them through multiple channels (CI logs, session context files, LiveView dashboards). Every stakeholder -- from individual developers to platform architects -- can access quality data at the appropriate level of detail.

Continuous Reporting

Quality transparency is not a periodic activity. The platform generates quality reports continuously: on every commit (via pre-commit hooks), on every CI pipeline run, on every deployment, and on every LLM session start. This continuous reporting ensures that quality drift is detected immediately, not during quarterly reviews.

Platform Architecture

The Prismatic Platform implements quality transparency through a multi-layer architecture that spans the entire development lifecycle.

Layer 1: Measurement Instrumentation

At the lowest layer, individual quality tools (compiler, Credo, Dialyzer, custom analyzers) produce raw measurements. Each tool emits structured telemetry events via Elixir's :telemetry library:

defmodule Prismatic.Quality.Instrumentation do
  @moduledoc """
  Telemetry instrumentation for quality measurement events.

  Emits structured events for every quality check execution,
  enabling downstream transparency pipelines to collect, aggregate,
  and report quality metrics without polling or manual intervention.
  """

  @spec emit_measurement(atom(), map(), map()) :: :ok
  def emit_measurement(domain, measurements, metadata) do
    :telemetry.execute(
      [:prismatic, :quality, :measurement],
      measurements,
      Map.merge(metadata, %{
        domain: domain,
        timestamp: DateTime.utc_now(),
        source_version: git_sha(),
        tool_version: tool_version(domain)
      })
    )
  end

  @spec emit_gate_decision(atom(), boolean(), map()) :: :ok
  def emit_gate_decision(gate_name, passed?, evidence) do
    :telemetry.execute(
      [:prismatic, :quality, :gate_decision],
      %{passed: passed?, duration_ms: evidence.duration_ms},
      %{
        gate: gate_name,
        evidence: evidence,
        timestamp: DateTime.utc_now(),
        decision_provenance: build_provenance(gate_name, evidence)
      }
    )
  end

  defp git_sha do
    case System.cmd("git", ["rev-parse", "--short", "HEAD"]) do
      {sha, 0} -> String.trim(sha)
      _ -> "unknown"
    end
  end

  defp tool_version(:credo), do: "1.7.x"
  defp tool_version(:dialyzer), do: "1.4.x"
  defp tool_version(_), do: "unknown"

  defp build_provenance(gate_name, evidence) do
    %{
      gate: gate_name,
      checks_executed: Map.keys(evidence.results),
      threshold_config: evidence.thresholds,
      measured_at: DateTime.utc_now()
    }
  end
end

Layer 2: Aggregation and Storage

Telemetry events flow into aggregation handlers that compute domain-level and platform-level quality scores. These aggregated metrics are stored in the Quality DNA system, which maintains a persistent, cross-session record of quality state:

defmodule Prismatic.Quality.Aggregator do
  @moduledoc """
  Aggregates raw quality measurements into domain-level and
  platform-level quality scores with full transparency metadata.
  """

  @type domain_score :: %{
    domain: atom(),
    score: non_neg_integer(),
    max_score: non_neg_integer(),
    violations: non_neg_integer(),
    checks_passed: non_neg_integer(),
    checks_total: non_neg_integer(),
    evidence: [map()]
  }

  @spec aggregate_domain(atom(), [map()]) :: domain_score()
  def aggregate_domain(domain, measurements) do
    results = Enum.map(measurements, &evaluate_check/1)

    %{
      domain: domain,
      score: compute_score(results),
      max_score: length(results) * 100,
      violations: Enum.count(results, &(&1.status == :violation)),
      checks_passed: Enum.count(results, &(&1.status == :passed)),
      checks_total: length(results),
      evidence: Enum.map(results, &build_evidence/1)
    }
  end

  @spec platform_score([domain_score()]) :: map()
  def platform_score(domain_scores) do
    total = Enum.sum(Enum.map(domain_scores, & &1.score))
    max = Enum.sum(Enum.map(domain_scores, & &1.max_score))
    percentage = if max > 0, do: round(total / max * 100), else: 0

    %{
      score: percentage,
      domains_perfect: Enum.count(domain_scores, &(&1.violations == 0)),
      domains_total: length(domain_scores),
      computed_at: DateTime.utc_now()
    }
  end

  defp evaluate_check(measurement) do
    %{
      check: measurement.check_name,
      status: if(measurement.value <= measurement.threshold, do: :passed, else: :violation),
      measured: measurement.value,
      threshold: measurement.threshold
    }
  end

  defp build_evidence(result) do
    Map.put(result, :provenance, %{evaluated_at: DateTime.utc_now()})
  end

  defp compute_score(results) do
    Enum.count(results, &(&1.status == :passed)) * 100
  end
end

Layer 3: Reporting and Visualization

Aggregated scores flow into reporting modules that produce human-readable reports, machine-readable JSON, and LiveView dashboard updates. The reporting layer ensures that quality data reaches every stakeholder through their preferred channel.

Layer 4: Audit Trail

Every quality measurement, aggregation, and gate decision is recorded in an append-only audit trail. This trail enables retrospective analysis ("When did this quality domain first show violations?"), compliance reporting ("Show me all gate decisions for the last 30 days"), and trend analysis ("How has typespec coverage changed over the last 6 months?").

The 13 Quality Domains

The Prismatic Platform organizes quality transparency around 13 distinct domains, each with its own measurement pipeline, thresholds, and reporting:

DomainMeasurementCurrent Status
DialyzerType consistency violations0 violations
CredoStatic analysis findings0 violations
CompilationCompiler warnings0 violations
DateTime PrecisionTemporal accuracy0 violations
Guard FunctionsDefensive programming0 violations
@impl CoverageCallback documentation0 violations (709 callbacks)
Memory SafetyResource management0 violations
PerformanceRuntime efficiency0 violations
Regression PreventionChange safety0 violations
Timing PatternsTemporal correctness0 violations
TODO ManagementTechnical debt tracking0 violations
Typespec CoverageType documentation0 violations
Unsafe Map AccessData safety0 violations

Each domain's metrics are independently visible, independently auditable, and independently reportable. This granularity ensures that a regression in one domain does not hide behind aggregate scores -- transparency extends to every individual measurement.

Quality Floor Guardian

The Quality Floor Guardian is the autonomous agent that enforces quality transparency in real time. It continuously monitors all 13 domains and escalates when transparency requirements are violated:

Score RangeEnforcement LevelAction
100-99%OPTIMALMonitor and report
98-99%WARNINGAlert and investigate
95-98%CRITICALAuto-evolution trigger
Below 95%EMERGENCYBlock commits and escalate

The Guardian's decisions are themselves transparent -- every escalation includes the full evidence chain that triggered it, ensuring that enforcement actions can be reviewed and validated.

Integration with Quality Gates

Quality gates are the enforcement points where transparency becomes actionable. The Prismatic Platform's quality gates operate with full transparency:

defmodule Prismatic.Quality.TransparentGate do
  @moduledoc """
  A quality gate implementation that enforces checks while
  maintaining full transparency of its decision process.

  Every gate execution produces an audit record containing
  the checks performed, values measured, thresholds applied,
  and the final pass/fail decision with justification.
  """

  @type gate_result :: %{
    gate: atom(),
    passed: boolean(),
    checks: [check_result()],
    duration_ms: non_neg_integer(),
    audit_record: map()
  }

  @type check_result :: %{
    name: atom(),
    passed: boolean(),
    measured: term(),
    threshold: term(),
    message: String.t()
  }

  @spec execute(atom(), keyword()) :: gate_result()
  def execute(gate_name, opts \\ []) do
    start_time = System.monotonic_time(:millisecond)
    checks = load_checks(gate_name)
    results = Enum.map(checks, &run_check/1)
    passed = Enum.all?(results, & &1.passed)
    duration = System.monotonic_time(:millisecond) - start_time

    result = %{
      gate: gate_name,
      passed: passed,
      checks: results,
      duration_ms: duration,
      audit_record: %{
        executed_at: DateTime.utc_now(),
        gate_version: gate_version(gate_name),
        config: Keyword.get(opts, :config, %{}),
        environment: Mix.env()
      }
    }

    Prismatic.Quality.Instrumentation.emit_gate_decision(
      gate_name,
      passed,
      result
    )

    result
  end

  defp load_checks(gate_name) do
    Application.get_env(:prismatic, :quality_gates, %{})
    |> Map.get(gate_name, [])
  end

  defp run_check(check_spec) do
    {module, function, args} = check_spec.mfa
    result = apply(module, function, args)

    %{
      name: check_spec.name,
      passed: result <= check_spec.threshold,
      measured: result,
      threshold: check_spec.threshold,
      message: format_message(check_spec, result)
    }
  end

  defp format_message(spec, result) do
    if result <= spec.threshold do
      "#{spec.name}: #{result} <= #{spec.threshold} (PASSED)"
    else
      "#{spec.name}: #{result} > #{spec.threshold} (FAILED - #{spec.remediation})"
    end
  end

  defp gate_version(_gate_name), do: "2.0.0"
end

Quality Reporting Pipelines

Quality transparency requires not just data collection but active reporting. The Prismatic Platform implements multiple reporting pipelines.

Session Reports

Every Claude Code session generates a quality report at start and end, capturing the quality state transition during the session. These reports are saved to .claude/session-context/ and include domain-by-domain breakdowns.

CI/CD Reports

The GitLab CI pipeline generates quality reports as pipeline artifacts, making them accessible from the merge request interface. Failed quality gates produce structured reports that explain exactly what failed and how to fix it.

LiveView Dashboards

The PrismaticWeb application provides real-time quality dashboards via Phoenix LiveView. These dashboards display current quality scores, historical trends, and active violations with drill-down capability to individual measurements.

JSON Machine-Readable Reports

For integration with external tools and automated systems, quality reports are available in JSON format via mix quality.gates --format=json. This enables programmatic consumption of quality data by monitoring systems, alerting frameworks, and trend analysis tools.

Relationship to Open Source

Quality transparency is particularly important in open-source contexts, where external contributors need visibility into project health without access to internal tooling. The Prismatic Platform's open-source strategy includes publishing quality metrics publicly, ensuring that potential contributors can assess project health before investing their time.

The platform's 4 published OSS packages (SDK, Plugin Kit, Security, UI) each carry quality badges derived from the same transparency pipelines that govern the internal codebase. This consistency ensures that external quality claims are backed by the same rigorous measurement infrastructure used internally.

Anti-Patterns and Quality Theater

Quality transparency explicitly rejects several common anti-patterns.

Hidden Dashboards

Quality dashboards that are only accessible to leads or managers violate the accessibility principle. In the Prismatic Platform, every contributor can see every quality metric.

Aggregate-Only Reporting

Reporting only aggregate scores (e.g., "quality: 95%") without domain-level breakdown enables hiding regressions behind overall averages. The platform always reports at domain granularity.

Silent Gate Failures

Quality gates that silently pass (or are silently bypassed) when checks fail undermine the entire quality system. The Prismatic Platform's --no-verify flag is explicitly forbidden in the session discipline protocol.

Metric Without Context

A metric like "0 Credo violations" is meaningless without context: which rules were enabled, what configuration was used, when was it last run. The platform's provenance system ensures every metric carries its full context.

Comparison with Industry Approaches

ApproachTransparency LevelPrismatic Advantage
SonarQubeDashboard-centric, often restrictedAll metrics accessible to all contributors
GitHub ActionsLog-based, ephemeralPersistent audit trail with provenance
Manual Code ReviewSubjective, undocumentedObjective, automated, fully documented
Badge-only (shields.io)Surface-level indicatorsDeep drill-down to individual measurements

Best Practices for Implementation

  1. Instrument before you enforce -- Collect metrics transparently before adding gates, so teams can see what will be enforced.
  2. Report at every granularity -- Platform-level, domain-level, and individual-check-level reports serve different audiences.
  3. Make audit trails append-only -- Never delete or modify historical quality data; it provides essential trend context.
  4. Automate distribution -- Quality reports should reach stakeholders without manual action.
  5. Include remediation guidance -- Every violation report should include specific steps to resolve the issue.

Connection to NABLA Infinity

Quality transparency aligns directly with the Trinity Gate requirements of the NABLA Infinity framework. The Provenance Mandatory axiom demands that all beliefs (including quality claims) carry traceable provenance. The Signal Plurality axiom requires that quality assessments be based on multiple independent measurements, not single metrics. Quality transparency provides the infrastructure to satisfy both axioms.

Performance Considerations

Quality transparency adds instrumentation overhead, but the Prismatic Platform keeps this minimal through several techniques: batched telemetry emission (events are buffered and flushed periodically), ETS-based aggregation (in-memory computation with O(1) lookups), and lazy report generation (reports are computed only when requested, not on every event). The measured overhead is less than 5ms per quality gate execution.

Future Directions

Quality transparency in the Prismatic Platform continues to evolve. Planned enhancements include predictive quality modeling (using historical trends to forecast future quality states), cross-repository transparency (surfacing quality metrics from dependent projects), and natural language quality summaries (using LLM capabilities to generate human-readable quality narratives from raw metrics).

Connect & Contribute

Created by Tomas Korcak (korczis) | Open Source under GHL

Back to Glossary