Prismatic Labs

Overview

Prismatic Labs is the platform's experimentation sandbox within the Prismatic Platform umbrella architecture. It houses prototype features, experimental algorithms, and research implementations that are being evaluated for promotion to production modules. The platform's NO MERCY NO DOUBTS doctrine enforces extremely high quality standards on production code, which is exactly right for deployed systems but can stifle exploration. Labs provides a controlled environment where new ideas can be tested, measured, and iterated upon without production-level constraints, while still maintaining safety isolation.

Labs manages experiments through a structured lifecycle: proposal, implementation, evaluation, and either promotion to a production module or retirement with documented learnings. Each experiment has defined success metrics, a time budget, and resource limits. This prevents the common R&D failure mode where experiments run indefinitely without clear evaluation criteria. When an experiment succeeds, Labs provides a promotion pathway that includes writing production-quality code, adding comprehensive tests, and integrating with the platform's quality gates before the feature leaves the sandbox.

The module currently hosts active experiments in areas including novel graph algorithms for entity resolution, alternative risk scoring models, advanced NLP techniques for intelligence extraction, and new visualization approaches for LiveView dashboards. Each experiment runs in isolation with its own data and cannot access production databases.

Architecture

Labs is built on an OTP supervision tree designed for isolated, fault-tolerant experiment execution.

PrismaticLabs.Application
└── PrismaticLabs.Supervisor (:one_for_one)
    ├── PrismaticLabs.ExperimentRegistry (GenServer)
    │   └── ETS: :labs_experiments (experiment metadata, status, metrics)
    ├── PrismaticLabs.SandboxSupervisor (DynamicSupervisor)
    │   ├── Experiment.Worker (per-experiment GenServer)
    │   ├── Experiment.Worker ...
    │   └── Experiment.Worker ...
    ├── PrismaticLabs.MetricsCollector (GenServer)
    │   └── Telemetry event aggregation per experiment
    └── PrismaticLabs.PromotionPipeline (GenServer)
        └── Checklist enforcement: tests, docs, quality gates

The Experiment Registry is a GenServer that tracks all experiments with their metadata, status, resource allocations, and metrics. The Sandbox Runtime provides isolated execution environments using separate ETS tables and dedicated database schemas, with resource limits enforced via Process.flag(:max_heap_size, ...) and Task timeouts. The Metrics Collector aggregates performance and quality measurements per experiment using Telemetry, producing structured reports for evaluation decisions.

Experiments are tagged with categories (algorithm, model, visualization, integration) and maturity levels (prototype, candidate, promoted, retired) for filtering and reporting.

Key Modules

Configuration

config :prismatic_labs,
  # Experiment limits
  max_concurrent_experiments: 20,
  max_memory_per_experiment_mb: 512,
  max_processes_per_experiment: 10,

  # Lifecycle
  default_experiment_duration: :days_30,
  auto_retire_after_days: 90,

  # Promotion pipeline
  promotion_checklist: [:tests, :docs, :quality_gates, :code_review],

  # Metrics collection
  metrics_flush_interval_ms: 30_000,
  telemetry_prefix: [:prismatic_labs, :experiment]

API Reference

# Create a new experiment with success criteria
{:ok, experiment} = PrismaticLabs.create_experiment(%{
  name: "graph_entity_resolution_v2",
  description: "Test new graph traversal algorithm for entity deduplication",
  category: :algorithm,
  duration: :days_30,
  success_criteria: %{precision: 0.95, recall: 0.90, latency_ms: 50},
  resource_limits: %{max_memory_mb: 512, max_processes: 10}
})

# Run experiment with test data
{:ok, result} = PrismaticLabs.run(experiment.id, test_dataset,
  iterations: 1000,
  collect_metrics: [:precision, :recall, :latency])

# Compare experiment results against baseline
{:ok, comparison} = PrismaticLabs.compare(
  experiment: experiment.id,
  baseline: :production_entity_resolution,
  metrics: [:precision, :recall, :f1_score])

# Promote successful experiment to production candidate
{:ok, promotion} = PrismaticLabs.promote(experiment.id,
  target_app: :prismatic_agents,
  reviewer: "lead@prismatic.io")

# List all active experiments with status
{:ok, experiments} = PrismaticLabs.list(status: :active)

# Retire an experiment with documented learnings
:ok = PrismaticLabs.retire(experiment.id, reason: "Superseded by v3 approach")

Testing

# Run all Labs tests
cd apps/prismatic_labs && mix test

# Run with coverage reporting
mix test --cover

# Run specific experiment lifecycle tests
mix test test/prismatic_labs/experiment_registry_test.exs

# Run sandbox isolation tests
mix test test/prismatic_labs/sandbox_test.exs

Testing covers experiment lifecycle management, sandbox isolation boundaries (verifying experiments cannot access production data), metrics collection accuracy, and promotion pipeline checklist enforcement. Property-based tests validate that resource limits are enforced under concurrent experiment execution.

Integration Points

NABLA Compliance

Labs experiments are subject to relaxed NABLA requirements during the exploration phase, but full compliance is required for promotion.

The promotion pipeline verifies that experimental code meets full NABLA axiom compliance before transitioning to production status. Experiments failing this gate remain in Labs until modified to comply.

Performance

Related Resources

• Prismatic Deduction -- Target for promoted reasoning algorithm experiments
• Prismatic Credo -- Experimental quality checks prototyped in Labs
• Prismatic Tidewave -- Code generation templates tested in Labs sandbox
• Prismatic Perimeter -- Security scoring model experiments
• Evolution Orchestrator Supreme -- Orchestrates autonomous evolution cycles that promote successful experiments
• Cross-Pollination Specialist -- Identifies cross-domain experiment opportunities and pattern transfers
• Quality Gates -- Enforces quality thresholds before experiments can be promoted to production
• Multi-Paradigm Solving -- Enables diverse experimental approaches across reasoning paradigms

Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL

• GitHub | GitLab | LinkedIn | Contact
• Developer Portal | Architecture | Meet the Creator