Prismatic Ollama

Overview

Prismatic Ollama provides the platform's local AI inference capability through integration with Ollama. It enables sovereign, on-premise language model usage without sending data to external cloud providers. The module manages model lifecycle, request routing, response processing, and quality gate enforcement for AI-generated outputs.

Data sovereignty is a fundamental requirement for intelligence platforms handling sensitive OSINT, legal, and financial data. Prismatic Ollama ensures that no query or context data leaves the organization's infrastructure during AI-assisted analysis. Combined with the platform's NABLA epistemic framework, it provides AI capabilities where outputs are treated as signals requiring the same confidence scoring, provenance tracking, and multi-source validation as any other intelligence source.

Model Management

Coordinator Agent

The Ollama Coordinator Agent (AIAD-compliant) manages the full model lifecycle:

Model Discovery --> Health Check --> Load/Unload --> Request Routing --> Quality Gate
      |                |                |                 |                  |
   ollama list     Memory check     Memory-aware      Best model for     Pass/fail
   API call        + latency test   scheduling         task type          + fallback

Model Registry

# Check available models and their status
{:ok, models} = PrismaticOllama.list_models()

[
  %{name: "qwen3-coder", size: "7B", status: :loaded, memory_mb: 7200,
    capabilities: [:code_generation, :code_review, :refactoring]},
  %{name: "gpt-oss:20b", size: "20B", status: :available, memory_mb: 15800,
    capabilities: [:general_intelligence, :analysis, :summarization]},
  %{name: "deepseek-coder", size: "6.7B", status: :loaded, memory_mb: 6900,
    capabilities: [:code_analysis, :bug_detection, :documentation]}
]

Supported Models

Cloud Fallback Architecture

When local models are unavailable or insufficient for a task, the coordinator transparently routes to cloud providers:

Request --> Local Model Available? --> YES --> Local Inference --> Quality Gate --> Result
                    |
                    NO (or quality gate fails)
                    |
            Cloud Fallback --> Cloud API --> Quality Gate --> Result
                    |
            Circuit Breaker monitors failure rate

Cloud Provider Configuration

# Cloud fallback configuration
config :prismatic_ollama, :cloud_fallback,
  enabled: true,
  providers: [
    %{name: :anthropic, model: "claude-sonnet-4-20250514", priority: 1},
    %{name: :openai, model: "gpt-4o", priority: 2}
  ],
  circuit_breaker: %{
    failure_threshold: 3,
    reset_timeout_ms: 60_000
  }

Inference Pipeline

Request Processing

# Structured prompt with variable injection
{:ok, code} = PrismaticOllama.generate("Write an Elixir GenServer for rate limiting",
  model: "qwen3-coder",
  template: :code_generation,
  variables: %{language: "Elixir", pattern: "GenServer"},
  max_tokens: 2000,
  temperature: 0.3
)

# Code analysis with quality scoring
{:ok, analysis} = PrismaticOllama.analyze(source_code,
  model: "deepseek-coder",
  task: :quality_review,
  criteria: [:credo_compliance, :otp_patterns, :typespec_coverage]
)

# Structured entity extraction
{:ok, entities} = PrismaticOllama.extract(text,
  model: "gpt-oss:20b",
  schema: %{
    entities: [%{name: :string, type: :enum, confidence: :float}],
    relationships: [%{source: :string, target: :string, type: :string}]
  }
)

Pipeline Stages

Quality Gate Enforcement

AI-generated outputs are treated as untrusted signals requiring validation:

NABLA Epistemic Integration

AI outputs are integrated into the belief graph as signals with explicit provenance:

# AI output registered as NABLA signal
signal = %{
  source: :ollama,
  model: "qwen3-coder",
  model_version: "2024.11",
  confidence: 0.82,  # Model-assessed confidence
  provenance: %{
    prompt_hash: "sha256:...",
    model_id: "qwen3-coder:latest",
    inference_time_ms: 2340,
    temperature: 0.3
  }
}

Performance Metrics

Architecture

Request --> Prompt Template --> Model Selector --> Ollama API --> Response Parser --> Quality Gate --> Result
              |                    |                  |               |                 |
         Variable           Capability +          HTTP/Stream     Structured       Pass/Fail
         Injection          Load Balance          via Finch       Extraction       + Fallback

Integration Points

Related Components

• Prismatic MCP - Model Context Protocol server
• Prismatic Claude - Claude AI integration (cloud)
• Prismatic Agents - Agent runtime with AI capabilities
• Prismatic Web - LiveView dashboards
• Prismatic API - REST API gateway

Related Agents

• Evolution Orchestrator Supreme -- Drives model selection and quality gate evolution for AI inference
• Cross-Pollination Specialist -- Transfers AI capabilities across code generation, analysis, and extraction domains
• ChatGPT Analyze -- AI-assisted analysis patterns adapted for local Ollama inference

Related Capabilities

• NABLA Axioms -- AI outputs treated as signals with provenance tracking and confidence scoring
• Multi-Paradigm Solving -- Multiple model capabilities (code, analysis, extraction) unified in single pipeline
• Autonomous Self-Healing -- Automatic cloud fallback when local models are unavailable or fail quality gates

Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL

• GitHub | GitLab | LinkedIn | Contact
• Developer Portal | Architecture | Meet the Creator