Autonomous Self-Healing

DOCTRINE

Autonomous Self-Healing

Overview

Autonomous Self-Healing enables the Prismatic Platform to detect, diagnose, and resolve quality degradation and system issues without human intervention. The system maintains a quality floor of 100/100 through continuous monitoring and automated correction. This capability spans five escalation levels, from instant pattern fixes to architectural improvements, creating a platform that actively resists entropy and quality decay.

The self-healing infrastructure builds on OTP's foundational "let it crash" philosophy but extends it far beyond process restart. Where OTP supervisors restart failed processes, the Prismatic self-healing system detects quality degradation patterns, diagnoses root causes, generates corrective patches, applies them with full test verification, and reports outcomes -- all without human intervention. This is the difference between recovering from failure and preventing degradation.

Self-healing is not an optional enhancement. In a platform with 99 umbrella applications, 6,652 Elixir source files, and 400+ autonomous agents, manual quality maintenance is infeasible. The volume of code, the frequency of changes, and the interconnected nature of the component ecosystem demand automated quality preservation. Every session, every commit, every deployment triggers self-healing processes that verify, correct, and evolve the platform's quality posture.

Core Architecture

The self-healing architecture consists of four interconnected subsystems: detection, diagnosis, remediation, and knowledge persistence. Each subsystem is implemented as an OTP application with its own supervision tree, ensuring that the healing infrastructure itself is fault-tolerant.

defmodule PrismaticSafety.SelfHealing.Supervisor do
  @moduledoc """
  Root supervisor for the self-healing subsystem.
  Manages detection, diagnosis, remediation, and knowledge workers.
  """
  use Supervisor

  def start_link(opts) do
    Supervisor.start_link(__MODULE__, opts, name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    children = [
      {PrismaticSafety.QualityFloorGuardian, []},
      {PrismaticSafety.AutoHeal.Detector, []},
      {PrismaticSafety.AutoHeal.Diagnostician, []},
      {PrismaticSafety.AutoHeal.Remediator, []},
      {PrismaticSafety.QualityDNA.Persistence, []},
      {DynamicSupervisor, name: PrismaticSafety.HealingWorkers, strategy: :one_for_one}
    ]

    Supervisor.init(children, strategy: :rest_for_one)
  end
end

The :rest_for_one strategy ensures that if the detector crashes, the diagnostician and remediator (which depend on detection output) are also restarted, maintaining a consistent healing pipeline state.

Healing Levels

The five-level hierarchy ensures proportionate response to issues of varying scope and complexity. Each level has defined authority, response time expectations, and verification requirements.

LevelScopeResponse TimeTriggerAuthorityExample
L1 - Pattern FixIndividual code patternsImmediate (< 1s)Known anti-pattern detectedAutomaticlength(list) > 0 replaced with list != []
L2 - Module HealModule-level issuesMinutesModule quality gate failureAutomaticMissing typespec, Credo violation
L3 - App RecoveryApplication-level problemsMinutes to hoursApp compilation failure, test regressionSemi-automaticDependency conflict, breaking API change
L4 - System HealCross-app systemic issuesHoursPlatform-wide quality score dropCoordinatedShared dependency upgrade cascade
L5 - ArchitectureArchitectural improvementsPlanned cyclesStructural optimization opportunityReview requiredModule extraction, supervision tree redesign

Level Escalation Protocol

Escalation follows a strict protocol: if L1 cannot resolve an issue within its scope, it escalates to L2. If L2 fails, L3 takes over. At each level, the full diagnostic context is preserved and passed to the next level, ensuring no information is lost during escalation.

L1 (Pattern) --> L2 (Module) --> L3 (App) --> L4 (System) --> L5 (Architecture)
     |                |              |             |                |
  Auto-fix        Auto-fix       Diagnose      Coordinate      Plan + Review
  + Verify        + Test         + Plan         + Execute       + Approve
  (instant)       (minutes)      (hours)        (hours)         (planned)

The escalation model also supports de-escalation. When an L3 investigation reveals that the root cause is actually a simple pattern issue, the fix is delegated back to L1 for immediate application. This bidirectional flow ensures that fixes are applied at the most efficient level.

Quality Floor Guardian

The Quality Floor Guardian is the central monitoring process that continuously evaluates platform quality and triggers healing when scores deviate from the floor.

Score RangeStatusGuardian ResponseHealing Level
100%OPTIMALMonitor onlyNone
99%WARNINGAlert + investigationL1-L2 auto-fix
98%CAUTIONActive healing triggeredL2-L3 diagnosis
95-97%CRITICALAuto-evolution cycleL3-L4 coordinated healing
< 95%EMERGENCYBlock commits + escalateL4-L5 full intervention
defmodule PrismaticSafety.QualityFloorGuardian do
  @moduledoc """
  Continuous quality monitoring with automated healing triggers.
  Maintains platform quality floor at 100/100 through proactive
  detection and proportionate response.
  """
  use GenServer

  @check_interval :timer.seconds(30)
  @quality_floor 100

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  @impl true
  def init(_opts) do
    schedule_check()
    {:ok, %{status: :optimal, last_score: @quality_floor, healing_active: false}}
  end

  @impl true
  def handle_info(:check_quality, state) do
    score = PrismaticSafety.QualityScorer.current_score()
    :telemetry.execute(
      [:prismatic_safety, :quality_floor, :check],
      %{score: score},
      %{previous: state.last_score}
    )

    new_state =
      case score do
        @quality_floor ->
          %{state | status: :optimal, healing_active: false}

        99 ->
          trigger_investigation(state, score)

        s when s >= 95 ->
          trigger_autoheal(state, score)

        _ ->
          trigger_emergency(state, score)
      end

    schedule_check()
    {:noreply, %{new_state | last_score: score}}
  end

  defp trigger_investigation(state, score) do
    PrismaticSafety.AutoHeal.Detector.investigate(score)
    %{state | status: :warning, healing_active: true}
  end

  defp trigger_autoheal(state, score) do
    PrismaticSafety.AutoHeal.cycle(%{trigger: :quality_drop, score: score})
    %{state | status: :critical, healing_active: true}
  end

  defp trigger_emergency(state, score) do
    PrismaticSafety.AutoHeal.emergency(%{score: score, block_commits: true})
    %{state | status: :emergency, healing_active: true}
  end

  defp schedule_check, do: Process.send_after(self(), :check_quality, @check_interval)
end

OTP Supervision Integration

Self-healing deeply integrates with OTP supervision trees for process-level fault tolerance. The platform's supervision tree spans all 99 umbrella applications with domain-specific supervisors coordinating cross-app recovery.

Supervision StrategyPlatform ApplicationRecovery Pattern
:one_for_oneIndependent agent workersRestart failed agent only
:one_for_allCoordinated agent teamsRestart entire team on failure
:rest_for_onePipeline stage workersRestart failed stage and downstream
DynamicSupervisorOn-demand investigation workersSpawn/terminate as needed
PrismaticSupervisor (root)
  +-- StorageDomain.Supervisor
  |     +-- ETS.Supervisor
  |     +-- Ecto.Supervisor
  |     +-- Redis.Supervisor
  +-- IntelligenceDomain.Supervisor
  |     +-- OSINT.Supervisor
  |     +-- Analysis.Supervisor
  +-- SecurityDomain.Supervisor
  |     +-- Perimeter.Supervisor
  |     +-- ColorTeam.Supervisor
  +-- QualityDomain.Supervisor
        +-- QualityFloorGuardian
        +-- AutoHeal.Supervisor
        +-- AutoEvolve.Supervisor

AutoHeal Cycle

The AutoHeal cycle is the primary automated healing mechanism, executing a full diagnostic-and-repair loop through seven phases.

Monitor --> Detect --> Diagnose --> Plan --> Execute --> Verify --> Report
   |           |          |          |         |          |          |
 Quality    Issue      Root       Fix      Apply     Tests       Quality
 Metrics   Found     Cause     Strategy   Patch     Pass         DNA

Each phase is instrumented with telemetry events and produces structured output that feeds the Quality DNA for cross-session continuity.

PhaseOutputPersistenceTelemetry Event
MonitorQuality score snapshotSession log[:autoheal, :monitor, :score]
DetectIssue description, affected files, severitySession log[:autoheal, :detect, :issue]
DiagnoseRoot cause analysis, dependency chainQuality DNA[:autoheal, :diagnose, :cause]
PlanFix strategy, affected modules, test requirementsQuality DNA[:autoheal, :plan, :strategy]
ExecuteApplied patches, modified filesGit commit[:autoheal, :execute, :patch]
VerifyTest results, quality score deltaQuality DNA + report[:autoheal, :verify, :result]
ReportSummary with before/after metricsSession context[:autoheal, :report, :complete]

CASCADE Pattern Elimination

The CASCADE system automates fixing of known anti-patterns with pre-verified transformations. Each CASCADE pattern has been validated across hundreds of instances to ensure zero false positives.

CASCADE PatternDetectionFixVerificationInstances Fixed
Type MismatchDialyzer warning analysisType annotation correctionDialyzer re-check200+
Dead CodeUnreachable code detectionSafe removal with dependency checkCompilation + tests150+
Empty Checklength(list) > 0 patternReplace with list != []Semantic equivalence test180+
Timer ReplacementProcess.sleep detectionReplace with :timer-based patternsTiming behavior test75+
Nuclear CacheStale build artifact detectionrm -rf _build/dev/lib/*/ebinFull recompilation300+
defmodule PrismaticSafety.Cascade.EmptyCheck do
  @moduledoc """
  CASCADE pattern: Replace length(list) > 0 with list != []
  Performance improvement: O(n) -> O(1) comparison.
  """

  @pattern ~r/length\((\w+)\)\s*>\s*0/

  @spec detect(String.t()) :: [{non_neg_integer(), String.t()}]
  def detect(source_code) do
    source_code
    |> String.split("\n")
    |> Enum.with_index(1)
    |> Enum.filter(fn {line, _idx} -> Regex.match?(@pattern, line) end)
    |> Enum.map(fn {line, idx} -> {idx, line} end)
  end

  @spec fix(String.t()) :: String.t()
  def fix(source_code) do
    Regex.replace(@pattern, source_code, "\\1 != []")
  end
end

SEADF Integration

The Self-Evolving Autonomous Development Framework (SEADF) provides the higher-order evolution capability that complements AutoHeal.

SEADF SubsystemSelf-Healing RoleInteraction
ScannerDiscovers quality issues and improvement opportunitiesFeeds AutoHeal detection
PipelineOrchestrates multi-stage healing workflowsCoordinates L3+ healing
Quality GuardianMaintains quality floor enforcementTriggers healing cycles
Knowledge SyncPreserves healing patterns across sessionsQuality DNA persistence
Cross-Domain InnovatorApplies patterns from one domain to othersL5 architectural patterns
Autonomous ReporterGenerates healing reports and metricsCompliance documentation
Enhanced Healing5-level healing execution engineCore healing implementation

Pre-Commit Quality Protection

The .githooks/pre-commit hook implements predictive quality protection, running checks before code enters the repository. This is the first line of defense that prevents quality degradation from reaching the codebase.

# Pre-commit quality protection phases
Phase 1: Syntax validation
Phase 2: Compilation (--warnings-as-errors)
Phase 3: Credo strict analysis
Phase 4: Dialyzer type checking
Phase 5: Affected test execution
Phase 6: Quality score verification
Phase 7: CASCADE pattern scan
Phase 8: Template validation
Phase 9: Security scan
Phase 10: Design consistency

Each phase has a defined timeout, failure policy, and escalation path. A failure at any phase blocks the commit and provides actionable diagnostic output describing the violation and suggested fix.

Quality DNA Cross-Session Continuity

Quality DNA enables self-healing to persist knowledge across sessions, preventing the same issues from recurring. The DNA is stored as structured JSON in .claude/quality-dna/current-state.json and is loaded at the start of every session.

DNA ComponentContentPurpose
Current StateQuality scores per domain, known patternsBaseline for detection
Fix HistoryApplied fixes with outcomesPattern library for future healing
Risk PatternsKnown anti-patterns with detection rulesProactive prevention
Evolution LogArchitectural improvements appliedL5 decision history
Session ContextLatest session's healing actionsCross-session continuity

Performance and Metrics

MetricCurrent ValueTarget
Quality Score100/100Maintain 100/100
QDP (Quality Debt Points)0Maintain 0
Mean Time to Detect< 10s< 5s
Mean Time to Heal (L1)< 1s< 500ms
Mean Time to Heal (L2)< 5 min< 2 min
CASCADE Patterns Fixed905+Ongoing
False Positive Rate< 1%< 0.5%
Quality Domains at 100%13/1313/13
Compilation Warnings00

Configuration

Self-healing behavior is configurable through the platform's configuration system, allowing teams to adjust thresholds, intervals, and automation levels.

# config/config.exs
config :prismatic_safety, PrismaticSafety.QualityFloorGuardian,
  check_interval: :timer.seconds(30),
  quality_floor: 100,
  emergency_threshold: 95,
  auto_heal_enabled: true,
  cascade_enabled: true,
  max_concurrent_heals: 5

config :prismatic_safety, PrismaticSafety.AutoHeal,
  l1_auto_apply: true,
  l2_auto_apply: true,
  l3_requires_review: false,
  l4_requires_review: true,
  l5_requires_approval: true,
  heal_timeout: :timer.minutes(10)

Integration


Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL