Lab

Overview

The Prismatic Lab is the platform's dedicated research and development environment, a controlled sandbox where new agent workflows, intelligence pipelines, architectural patterns, and epistemic pipeline validation strategies are prototyped, stress-tested, and validated before they are considered for production deployment. Built on Elixir/OTP primitives with ETS-backed state isolation and dedicated supervision trees, the Lab provides a structured, repeatable, and auditable framework for turning hypotheses into proven capabilities -- far beyond ad-hoc experimentation in feature branches or developer sandboxes.

The Lab occupies a unique position within the Prismatic ecosystem. It bridges the gap between theoretical design and production reality by providing the infrastructure to run experiments at meaningful scale without risking the stability of the live platform. Every experiment operates within strict resource boundaries, uses only synthetic data, and produces artifacts that can be independently reviewed and reproduced by any team member. The NABLA Infinity framework's epistemic axioms govern how experimental claims are validated, while the Trinity Gate ensures that no conclusion passes without structural, logical, and formal verification.

What distinguishes the Prismatic Lab from conventional R&D environments is its deep integration with the platform's quality and evolution systems. Experiments are not simply run and forgotten. They follow a rigorous lifecycle governed by the No Mercy, No Doubts doctrine that demands clear hypotheses, measurable success criteria, peer review, and formal graduation gates. The Quality DNA system tracks experimental quality metrics across sessions, while autoevolve scanners identify improvement opportunities in graduated code. An experiment that proves its value earns the right to be promoted to production through the API gateway. An experiment that fails provides equally valuable data that is captured, analyzed, and fed back into the Academy as lessons learned.

The Lab currently maintains 15 registered experiments, 12 of which are actively producing results. The graduation rate of 78% reflects the platform's commitment to promoting only thoroughly validated patterns while preserving the freedom to explore ideas that may not succeed on first attempt. Research spans agent prototyping and Color Team security simulations through formal verification studies and EASM discovery experiments.

Experiment Lifecycle

Every experiment in the Lab follows a structured five-phase lifecycle designed to maximize scientific rigor while minimizing overhead. This lifecycle is not merely a suggestion; it is enforced by the Lab's infrastructure through automated checks at each phase transition.

Phase 1: Hypothesis

Every experiment begins with a clearly articulated hypothesis, grounded in the NABLA Infinity principle that claims require evidence and provenance. This is not a vague aspiration like "improve agent performance" but a falsifiable statement with measurable boundaries. A well-formed hypothesis follows the template: "Given [preconditions], when [action], then [expected outcome] within [constraints]."

%Experiment.Hypothesis{
  id: "EXP-2026-042",
  statement: "A pipeline with parallel NABLA axiom validation will process
              epistemic claims 3x faster than sequential validation while
              maintaining identical correctness guarantees",
  preconditions: ["Minimum 1000 claims per batch", "All 7 axioms active"],
  success_metric: {:throughput_ratio, :gte, 3.0},
  correctness_invariant: {:axiom_violations, :eq, 0},
  max_duration: {:days, 14},
  author: "epistemic-research-team",
  review_status: :approved
}

The hypothesis phase requires sign-off from at least one domain expert before the experiment can proceed. This prevents resource waste on poorly defined investigations and ensures that the experiment's success criteria are unambiguous.

Phase 2: Design

During the design phase, the experiment's architecture is specified in detail. This includes the agents involved, the data sources (always synthetic), the pipeline topology, the monitoring hooks, and the resource limits. The design document serves as both a blueprint and a contract: it defines exactly what the experiment will do and what it will not do.

Design artifacts include a supervision tree diagram, a data flow specification, resource allocation requests, and a rollback plan. The Lab's infrastructure validates the design against platform constraints before allowing execution to begin.

Phase 3: Execute

Execution occurs within the Lab's sandboxed environment. Each experiment receives its own OTP application context with dedicated supervision trees, isolated ETS tables, and independent telemetry streams. The sandbox enforces strict boundaries: no network access to external systems, no access to production data, and hard limits on memory and CPU consumption.

During execution, the Lab's monitoring system captures fine-grained telemetry data including throughput metrics, latency distributions, error rates, resource consumption, and any NABLA axiom interactions. This data is persisted in a structured format that supports both real-time dashboards and post-hoc analysis.

Phase 4: Analyze

Analysis transforms raw execution data into actionable conclusions. The Lab provides automated analysis tools that compare observed metrics against the hypothesis's success criteria, generate statistical summaries, identify anomalies, and produce visualization-ready datasets.

Analysis is not purely automated. Every experiment requires a human-authored analysis report that interprets the results, identifies confounding factors, acknowledges limitations, and recommends next steps. This report undergoes peer review before the experiment can proceed to the graduation decision.

Phase 5: Graduate

Graduation is the formal transition from experiment to production candidate. Not every experiment graduates, and that is by design. The graduation decision considers not only whether the hypothesis was confirmed but also whether the implementation meets the platform's quality standards, whether the pattern is generalizable, and whether the maintenance burden is acceptable.

Experiments that graduate are packaged as standard platform modules with full test suites, documentation, and AIAD agent specifications. They enter the production codebase through the normal quality gate process and become discoverable through the API gateway's auto-introspection mechanism.

Research Domains

The Lab organizes its research activities across five primary domains, each addressing a distinct aspect of the platform's capability development.

Agent Prototyping

Agent prototyping experiments explore new patterns for autonomous agent design, including novel coordination strategies, decision-making algorithms, and self-healing behaviors. This domain has produced several production agents, including improvements to the orchestration layer and new specialist agent archetypes. Agent experiments typically involve designing a new AIAD-compliant agent specification, implementing its core behavior in an isolated GenServer process, and validating its interactions with synthetic workloads.

Pipeline Architecture

Pipeline experiments investigate data flow patterns, transformation strategies, and processing topologies. The platform's multi-stage processing pipelines handle everything from OSINT data collection to epistemic claim validation, and the Lab provides a safe space to explore optimizations, alternative topologies, and new pipeline stages. Recent work has focused on parallel axiom validation pipelines and streaming analytics integration.

Epistemic Validation

The NABLA Infinity framework's seven axioms -- including Signal Plurality, Contradiction Preservation, and Provenance Mandatory -- and the Trinity Gate form the epistemic backbone of the platform. Lab experiments in this domain test edge cases in axiom enforcement, explore relaxation strategies for exploratory contexts, benchmark validation performance, and prototype new epistemic tools using property-based testing and Lean4 proofs. This is one of the most active research areas, reflecting the platform's commitment to evidence-based decision making.

Security Simulation

Security experiments leverage the Color Team framework to test new attack simulation scenarios, defensive strategies, and threat models in a completely isolated environment. Adversarial agents from the Red Team probe epistemic vulnerabilities while Blue Team agents construct evidence-based defenses, with Purple Team coordination ensuring findings reach closure. The Lab's security sandbox provides even stricter isolation than the standard sandbox, with additional controls to prevent any security research artifacts from escaping the experimental context. All security experiments use synthetic threat models and simulated attack surfaces.

Storage Benchmarks

Storage experiments evaluate adapter pattern performance, caching strategies, query optimization patterns, and data model alternatives across the platform's diverse storage backends including PostgreSQL, ETS, KuzuDB, and Meilisearch. These experiments generate the benchmark data that informs production storage decisions and capacity planning.

Sandbox Architecture

The Lab's sandbox architecture provides hermetic isolation for experiments while maintaining enough platform fidelity to produce meaningful results. The isolation model operates at multiple levels.

Process Isolation

Each experiment runs within its own OTP application boundary with a dedicated supervision tree. The experiment's root supervisor is started as a child of the Lab's master supervisor, which monitors resource consumption and enforces limits. If an experiment exceeds its allocated resources or crashes repeatedly, the master supervisor terminates it cleanly and logs the failure.

defmodule PrismaticLab.ExperimentSupervisor do
  use Supervisor

  def start_experiment(experiment_id, config) do
    child_spec = %{
      id: experiment_id,
      start: {PrismaticLab.ExperimentRunner, :start_link, [config]},
      restart: :temporary,
      shutdown: :timer.seconds(30),
      type: :supervisor
    }

    Supervisor.start_child(__MODULE__, child_spec)
  end
end

Data Isolation

Experiments have no access to production data. The Lab provides a synthetic data generation framework that can produce realistic datasets matching the statistical properties of production data without containing any real information. Each experiment receives its own ETS tables and, when needed, its own isolated PostgreSQL schema. Data generated during an experiment is tagged with the experiment ID and retained for the configured retention period before automatic cleanup.

Resource Limits

Every experiment declares its resource requirements during the design phase. The Lab enforces these limits through OTP process monitoring and periodic resource audits. Limits cover memory consumption, CPU time, ETS table size, disk I/O, and experiment duration. Hard limits trigger automatic termination; soft limits trigger warnings that are surfaced in the experiment's dashboard.

Monitoring

The Lab integrates with the platform's telemetry system to provide real-time visibility into experiment execution. Each experiment emits structured telemetry events under the [:prismatic_lab, :experiment, *] namespace. These events feed into both the Lab's internal dashboards and the platform's general observability infrastructure. Experiment authors can define custom telemetry events for domain-specific metrics.

Graduation Criteria

An experiment must satisfy a comprehensive set of criteria before it can graduate to production. These criteria ensure that graduated experiments meet the platform's quality, security, and maintainability standards.

Functional Criteria: The experiment's hypothesis must be confirmed with statistical significance. Success metrics must meet or exceed their defined thresholds. The implementation must handle edge cases identified during execution. Error handling must follow the {:ok, _} / {:error, _} pattern consistently.

Quality Criteria: The graduated module must compile with zero warnings under --warnings-as-errors per the Zero Warning Policy. Credo strict mode must report zero violations. Dialyzer must produce zero warnings. Test coverage must reach 100% for business logic and 90% overall. All typespecs must be present and accurate.

Documentation Criteria: The module must include complete @moduledoc and @doc annotations. An AIAD agent specification must be authored if the experiment involves a new agent. Usage examples must be included in the documentation. A post-mortem report summarizing the experiment's journey from hypothesis to graduation must be filed.

Security Criteria: The graduated code must pass the platform's security review checklist. No hardcoded credentials, no unsafe external calls, no unbounded resource allocation. If the experiment involves security-sensitive functionality, a Color Team review is required.

Integration Points

The Lab does not exist in isolation. It maintains active integration pathways with other platform components that amplify its value.

Lab to Academy Feedback Loop

Every graduated experiment produces learning materials for the Academy. The experiment's hypothesis, methodology, results, and lessons learned are distilled into tutorial content that teaches the underlying concepts. Failed experiments are equally valuable for the Academy, as they document what does not work and why, preventing others from repeating unsuccessful approaches.

The Academy's curriculum is directly influenced by Lab activity. When the Lab identifies a new pattern that proves successful, the Academy team creates corresponding learning materials. When Academy students encounter questions that existing materials cannot answer, those questions may seed new Lab experiments.

Lab to API Promotion

When an experiment graduates, its public functions are implemented in the appropriate Prismatic facade module. The API gateway's auto-introspection scanner discovers these new functions at boot time and exposes them as documented REST endpoints. This means that a successful Lab experiment can go from validated prototype to publicly accessible API endpoint through a well-defined, automated pathway.

Lab to Architecture Influence

Lab experiments frequently inform architectural decisions documented in the architecture section. When an experiment reveals that a particular approach scales better, handles failures more gracefully, or integrates more cleanly with existing systems, those findings influence the platform's architectural evolution.

Active Research Areas

The Lab maintains several active research threads that reflect the platform's current strategic priorities.

Parallel Epistemic Validation: Investigating concurrent evaluation of NABLA Infinity axioms to improve throughput in high-volume claim processing scenarios. Early results suggest a 2.5x to 4x speedup is achievable without compromising validation correctness.

Adaptive Agent Coordination: Exploring dynamic agent topology adjustment based on workload characteristics. Rather than static orchestration graphs, these experiments test agents that can negotiate coordination strategies at runtime.

Graph-Based Threat Modeling: Leveraging KuzuDB's knowledge graph capabilities to represent attack surface relationships and compute risk propagation paths. This research supports the Prismatic Perimeter EASM module's security rating calculations.

Streaming Pipeline Optimization: Benchmarking GenStage and Flow-based pipeline architectures for continuous data pipeline workloads, with particular focus on backpressure handling and graceful degradation under load.

Cross-Domain Knowledge Transfer: Investigating mechanisms for agents to share learned patterns across domain boundaries while respecting the NABLA Infinity source independence axiom and maintaining belief graph integrity.

Methodology Standards

The Lab enforces rigorous methodology standards to ensure that experimental results are trustworthy, reproducible, and comparable.

Reproducibility

Every experiment must be fully reproducible from its specification. This means that all configuration, synthetic data generation seeds, agent specifications, and environmental parameters are versioned and stored alongside the experiment's results. Any team member should be able to re-execute a past experiment and obtain statistically equivalent results.

Documentation Requirements

Experiments must maintain living documentation throughout their lifecycle. The hypothesis document is written before execution begins. Execution logs are captured automatically. Analysis reports are authored by the experiment team and reviewed by peers. Graduation proposals include a complete narrative of the experiment's journey.

Peer Review

All experiments undergo peer review at two critical junctures: before execution begins (design review) and before graduation is approved (results review). Design reviews verify that the hypothesis is well-formed, the methodology is sound, and the resource requests are reasonable. Results reviews verify that the analysis is accurate, the conclusions are supported by data, and the graduation criteria are genuinely met.

Statistical Rigor

Quantitative claims must be supported by appropriate statistical analysis. The Lab provides standard analysis templates for common experimental patterns including A/B comparisons, throughput benchmarks, and latency distribution analysis. Experiments that make performance claims must report confidence intervals, not just point estimates.

Infrastructure

The Lab's infrastructure is built on Elixir/OTP primitives, leveraging the platform's supervision and process isolation capabilities to provide robust experiment management.

OTP Process Isolation

Each experiment runs as an independent OTP application with its own supervision tree. The PrismaticLab.MasterSupervisor manages all active experiments and enforces platform-level resource policies following the let-it-crash philosophy. Experiment processes communicate with the rest of the platform only through well-defined message-passing interfaces, preventing unintended coupling.

ETS Sandboxes

Experiments that require fast in-memory state use dedicated ETS tables created by the Lab's table manager. These tables are owned by the experiment's supervisor and are automatically cleaned up when the experiment terminates. The table manager enforces size limits and provides access control to prevent cross-experiment data leakage.

Telemetry Integration

The Lab emits structured telemetry events for every significant lifecycle transition: experiment creation, phase transitions, resource limit warnings, and termination. These events integrate with the platform's standard observability infrastructure, enabling Lab activity to be monitored through the same dashboards used for production systems.

:telemetry.execute(
  [:prismatic_lab, :experiment, :phase_transition],
  %{duration_ms: elapsed},
  %{
    experiment_id: experiment.id,
    from_phase: :execute,
    to_phase: :analyze,
    metrics_collected: length(experiment.metrics)
  }
)

Persistent Storage

Experiment artifacts, including configuration, results, analysis reports, and telemetry data, are persisted in a structured format that supports long-term retention and cross-experiment analysis. The Lab's storage layer uses PostgreSQL for relational data and the filesystem for large artifacts, with a configurable retention policy that defaults to 90 days for completed experiments.

Getting Started

Proposing and running a Lab experiment involves a straightforward process designed to minimize bureaucratic overhead while maintaining methodological rigor.

Step 1: Draft a Hypothesis. Write a clear, falsifiable statement of what you want to investigate. Include preconditions, expected outcomes, and measurable success criteria. Use the hypothesis template provided in the Lab's documentation.

Step 2: Design the Experiment. Specify the agents, pipelines, data sources, and monitoring hooks your experiment requires. Declare resource limits. Identify potential risks and define a rollback plan. Submit the design for peer review.

Step 3: Request Approval. Once the design review is complete, submit your experiment for approval. The Lab coordinator reviews resource availability and scheduling constraints. Most experiments are approved within one business day.

Step 4: Execute and Monitor. Launch your experiment using the Lab CLI. Monitor progress through the Lab dashboard. The infrastructure handles process management, data collection, and resource enforcement automatically.

# Submit a new experiment proposal
mix lab.propose --hypothesis experiments/EXP-2026-042.md

# Start an approved experiment
mix lab.start EXP-2026-042

# Check experiment status
mix lab.status EXP-2026-042

# View experiment dashboard
mix lab.dashboard

Step 5: Analyze and Report. When execution completes, use the Lab's analysis tools to process your results. Author an analysis report interpreting the findings. Submit the report for peer review.

Step 6: Graduate or Archive. Based on the analysis, propose graduation to production or archive the experiment with lessons learned. Either outcome contributes to the platform's knowledge base and feeds into Academy learning materials.

The Lab welcomes experiments of all scales, from quick validation tests that run in minutes to extended research campaigns that span weeks. The structured lifecycle ensures that every experiment, regardless of scale, produces value for the platform.

Connect & Contribute

Created by Tomas Korcak (korczis) | Open Source under GHL

• GitHub | GitLab | LinkedIn | Contact
• Developer Portal | Architecture | Meet the Creator