Blackboard

architecture

Blackboard

{% import "macros/flowbite.html" as fb %}

Definition

The Blackboard pattern is a software architecture for collaborative problem solving in which multiple independent knowledge sources (agents, solvers, or subsystems) cooperate by reading from and writing to a shared data structure known as the blackboard. Unlike direct message-passing architectures where agents must know about each other, the blackboard decouples producers from consumers: agents observe the current state of shared knowledge, contribute partial solutions when their expertise is relevant, and trigger further reasoning by other agents who recognize new opportunities in the updated state.

The pattern originated in speech understanding research at Carnegie Mellon University in the 1970s (the Hearsay-II system) and has since been applied to signal interpretation, planning, diagnosis, and multi-agent intelligence systems. The classical blackboard architecture consists of three components: the blackboard data structure (shared working memory), knowledge sources (independent specialist modules), and a control component (scheduling mechanism that determines which knowledge source to activate next).

The blackboard's enduring appeal lies in its ability to coordinate heterogeneous agents that possess different representations, algorithms, and domains of expertise without requiring them to understand each other's internal workings. Each agent interacts only with the shared knowledge structure, contributing what it can and consuming what it needs. This makes the pattern particularly well-suited for complex analytical tasks -- such as intelligence fusion, security assessment, and epistemic reasoning -- where no single agent possesses sufficient knowledge to solve the problem alone.

Historical Context and Evolution

{{ fb::p5_interactive_dashboard(title="Blackboard Knowledge Sharing Simulation", sketch_type="particles", data_source="blackboard.knowledge_sharing", controls=true) }}

{{ fb::divider(label="Architecture Visualization") }}

Interactive Demo: The visualization above demonstrates the blackboard pattern in action within the Prismatic Platform. Watch as multiple knowledge sources (represented as colored agents around the perimeter) contribute insights to the shared blackboard space (center). Each particle represents a knowledge contribution flowing from agents to the blackboard and triggering responses from other agents. Observe how the system resolves conflicts when contradictory knowledge is contributed and how new knowledge propagates through the agent network. The animation shows the three classical components: knowledge sources (agents), the blackboard data structure (shared central space), and the control mechanism (coordination logic).

The blackboard pattern has evolved significantly from its origins in AI research to its modern incarnation in distributed systems and multi-agent platforms.

EraSystemContribution
1971-1976Hearsay-II (CMU)First blackboard system for speech understanding
1980sBB1, GBBGeneralized blackboard frameworks with explicit control
1986Nii's "Blackboard Systems" surveyCanonical formalization of the pattern
1990sCLIPS/JESSRule-based systems with blackboard-like working memory
2000sMulti-agent systems (JADE, JACK)Blackboard as agent coordination mechanism
2005-2020GARDEN legacy (Prismatic)Blackboard for OSINT intelligence fusion
2025+Prismatic PlatformModernized blackboard with ETS, KuzuDB, 552 agents

The GARDEN legacy knowledge base represents a particularly significant chapter in this evolution. Over 20+ years and 22 repositories, the GARDEN ecosystem developed a Blackboard system for OSINT intelligence gathering that coordinated multiple data sources, enrichment providers, and analysis modules. This battle-tested architecture directly informs the Prismatic Platform's modern implementation.

{{ fb::p5_grid_2x2( title1="Knowledge Sources Activity", type1="agents", data1="blackboard.knowledge_sources", title2="Shared Memory Structure", type2="network", data2="blackboard.memory_structure", title3="Control Flow Coordination", type3="generative", data3="blackboard.control_flow", title4="Conflict Resolution", type4="particles", data4="blackboard.conflict_resolution" ) }}

Classical Blackboard Architecture

The canonical blackboard architecture consists of three interacting components, each serving a distinct role in the collaborative problem-solving process.

Knowledge Sources

Knowledge sources (KS) are independent specialist modules, each possessing expertise in a specific domain or analysis technique. A knowledge source monitors the blackboard for conditions that match its area of expertise, and when triggered, reads relevant data, performs its analysis, and writes results back to the blackboard. Knowledge sources do not communicate with each other directly.

In the Prismatic Platform, the 434 agents across 14 domains serve as knowledge sources. Each agent specializes in a particular capability -- OSINT data extraction, security rating calculation, epistemic verification, quality assessment -- and contributes its findings to the shared knowledge structure.

The Blackboard Data Structure

The blackboard itself is a structured shared workspace organized into levels of abstraction or problem-solving stages. Data on the blackboard is typically organized hierarchically, with raw observations at lower levels and increasingly refined interpretations at higher levels.

Level 5: Conclusions     [Security Rating: B (780)]
Level 4: Interpretations [3 critical vulnerabilities, 12 informational]
Level 3: Hypotheses      [Possible misconfigured TLS, outdated cipher suites]
Level 2: Features        [Open ports: 80, 443, 8080; TLS version: 1.2]
Level 1: Raw Data        [DNS records, certificate chain, port scan results]

Control Component

The control component determines which knowledge source to activate next, based on the current state of the blackboard and a scheduling strategy. Control strategies range from simple (round-robin, priority-based) to sophisticated (opportunistic, focus-of-attention, meta-level reasoning about which KS activation would most advance the solution).

GARDEN Legacy and Modernization

The GARDEN legacy knowledge base (22 repositories, 3,050+ files, 55+ patterns, 20+ years) is the direct ancestor of Prismatic's blackboard implementation. The GARDEN ecosystem's Blackboard system, originating from the prismatic-legacy repository (1,302 files, Tier 4 archive), demonstrated several key principles that carry forward.

GARDEN PatternModern Prismatic Implementation
Shared working memory (in-process)ETS tables with microsecond access
Knowledge source registryAgent Registry (552 agents)
Hierarchical blackboard levelsMulti-backend storage tiers (ETS, PostgreSQL, KuzuDB)
Rule-based controlAIAD agent activation policies and priority scheduling
Serialized knowledgeStructured Elixir terms with provenance metadata
Single-process coordinationDistributed coordination via PubSub and message passing

The modernization effort preserved GARDEN's core insight -- that heterogeneous knowledge sources produce better analytical results when coordinated through shared state rather than point-to-point communication -- while replacing the implementation with OTP-native technologies capable of operating across distributed BEAM nodes.

Prismatic's Blackboard Implementation

The Prismatic Platform implements the blackboard pattern using a multi-backend storage architecture that separates concerns by access pattern and query complexity.

Storage Backend Selection

BackendRole in BlackboardAccess PatternLatency
ETSHot working memoryKey-value lookup, concurrent readsMicroseconds
PostgreSQLPersistent fact storeRelational queries, transactionsMilliseconds
KuzuDBRelationship graphPath queries, pattern matchingMilliseconds
MeilisearchFull-text searchKeyword and semantic searchMilliseconds
RedisCross-node shared statePub/sub, distributed countersSub-millisecond

Agent Interaction Model

Agents interact with the blackboard through a standardized interface that enforces provenance tracking and NABLA Infinity axiom compliance.

# Agent contributing findings to the blackboard
defmodule PrismaticAgents.SecurityScanner do
  @behaviour PrismaticAgents.KnowledgeSource

  @impl true
  def trigger_condition(blackboard) do
    # Activate when new domains are discovered
    Blackboard.has_new_entries?(blackboard, :discovered_domains)
  end

  @impl true
  def contribute(blackboard, context) do
    domains = Blackboard.read(blackboard, :discovered_domains)

    findings =
      domains
      |> Enum.map(&scan_domain/1)
      |> Enum.flat_map(&extract_vulnerabilities/1)

    Blackboard.write(blackboard, :security_findings, findings,
      source: __MODULE__,
      confidence: 0.85,
      timestamp: DateTime.utc_now()
    )
  end
end

Color Team Signal Flow

The Color Teams (Gray, Red, Blue, Purple, White, Black) use the blackboard as their primary coordination mechanism, following the signal flow architecture.

Gray Team writes  --> :boundary_seeds      (edge cases, specification gaps)
Red Team reads    <-- :boundary_seeds
Red Team writes   --> :adversarial_findings (simulated attack results)
Purple reads      <-- :adversarial_findings + :defensive_evidence
Purple writes     --> :synthesis_results    (Red-Blue loop closure)
Blue Team reads   <-- :synthesis_results
Blue Team writes  --> :defensive_posture    (updated defense status)
White Team reads  <-- :synthesis_results
White Team writes --> :verification_proofs  (formal verification results)

Collaborative Problem Solving

The blackboard pattern excels at problems that require multiple forms of expertise applied in a flexible, data-driven order. The Prismatic Platform exploits this for several key workflows.

OSINT Intelligence Fusion: Multiple data source agents (Shodan, Censys, GreyNoise, DNS, certificate transparency) independently contribute raw observations to the blackboard. Entity resolution agents identify connections between observations. Analysis agents synthesize findings into coherent intelligence assessments. No single agent orchestrates this process; the blackboard's state drives activation.

Security Rating Calculation: The EASM pipeline calculates security ratings (A-F, 300-900 score) by combining findings from diverse assessment agents. Vulnerability scanners, configuration analyzers, compliance checkers, and reputation services each contribute their assessments independently. The rating engine reads the accumulated evidence and computes a composite score.

Epistemic Verification: The Trinity Gate verification process uses the blackboard to coordinate structural consistency checks, logical consistency validation, and formal proof construction. Each verification layer writes its results to the blackboard, and the gate controller reads the combined evidence to make accept/reject decisions.

Control Strategies

The Prismatic Platform implements several control strategies for blackboard agent activation, selected based on the problem domain and urgency.

StrategyDescriptionUse Case
Priority-basedAgents activated by priority level (L1-L4)Normal operations
OpportunisticActivate agent whose preconditions are most satisfiedIntelligence fusion
Focus-of-attentionDirect activation toward specific blackboard regionsIncident response
Event-drivenPubSub notifications trigger agent activationReal-time monitoring
Meta-levelControl agent reasons about which KS to activateComplex multi-step analysis

The event-driven strategy, implemented via Phoenix PubSub, is the most common in practice. When an agent writes to the blackboard, a PubSub notification is broadcast to all subscribed agents, which then evaluate their trigger conditions and self-activate if appropriate. This eliminates polling overhead while maintaining the blackboard's decoupled architecture.

Comparison with Alternative Coordination Patterns

PatternCouplingState ModelScalabilityBest For
BlackboardLow (shared state)Persistent, inspectableModerateComplex analytical tasks
Message passingLow (async messages)Transient (mailbox)HighRequest-response, streaming
PubSubLow (topic-based)Transient (events)HighEvent notification
OrchestratorHigh (central control)CentralizedLowSequential workflows
ChoreographyModerate (protocol)DistributedModerateMulti-step transactions

The Prismatic Platform uses all of these patterns in combination, with the blackboard serving as the coordination backbone for complex analytical workflows and message passing handling the high-throughput operational communication.

Performance and Scalability

The multi-backend blackboard architecture is designed to handle the platform's scale: 552 agents, 92 umbrella applications, and millions of intelligence records.

MetricTargetImplementation
Read latency (hot data)< 10 microsecondsETS concurrent reads
Write latency (facts)< 1 millisecondETS write-through to PostgreSQL
Relationship queries< 50 millisecondsKuzuDB graph traversal
Full-text search< 100 millisecondsMeilisearch indexed search
Agent activation latency< 5 millisecondsPubSub event-driven triggers
Concurrent readersUnlimitedETS lock-free concurrent reads

ETS provides the performance-critical hot path, with write-through replication to PostgreSQL for durability and KuzuDB for relationship-rich queries. This tiered approach ensures that agent activation is never bottlenecked by storage latency while maintaining full persistence and queryability of the accumulated knowledge.

  • Knowledge Graph - Graph-structured knowledge representation complementing blackboard state
  • Agent - Autonomous units serving as knowledge sources in the blackboard architecture
  • Agent Registry - Registry of 552 agents interacting via the blackboard
  • Data Pipeline - Pipeline patterns feeding data into and consuming data from the blackboard
  • GARDEN - Source of the blackboard pattern with 22 repos of legacy knowledge
  • KuzuDB - Graph storage backing relationship-rich blackboard queries
  • ETS - In-memory storage for high-speed blackboard access
  • PubSub - Event distribution for blackboard change notifications
  • Color Teams - Security teams coordinating through blackboard signal flow
  • Intelligence Fusion - Multi-source correlation leveraging blackboard data
  • Mycelial Network - Knowledge propagation network complementing blackboard coordination

See Also


Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL