Definition
Cooperative systems are computational architectures in which multiple autonomous components -- whether processes, agents, services, or nodes -- work together through shared protocols, negotiated interactions, and coordinated behavior to achieve collective outcomes that exceed the capabilities of any individual component. Unlike centrally controlled systems where a single orchestrator dictates all behavior, cooperative systems distribute decision-making authority among participants who voluntarily adhere to shared rules and communication protocols.
In the Prismatic Platform, cooperative systems manifest at multiple architectural levels: at the BEAM VM level through cooperative process scheduling, at the OTP level through supervision trees and inter-process message passing, and at the AIAD level through multi-agent systems where 530+ agents coordinate their activities to perform complex intelligence, security, and quality operations. The platform's entire design philosophy rests on the principle that sophisticated behavior emerges from the cooperation of simple, well-defined components.
Theoretical Foundations
Cooperative vs. Competitive Systems
System design exists on a spectrum between pure cooperation and pure competition. In cooperative systems, components share information freely, align their objectives, and sacrifice local optimization for global benefit. In competitive systems, components pursue independent objectives and interact through market-like mechanisms. Most real-world systems blend both approaches.
| Characteristic | Cooperative | Competitive | Prismatic Approach |
|---|---|---|---|
| Information sharing | Open, broadcast | Strategic, selective | Open within teams, controlled between tiers |
| Objective alignment | Shared global goals | Independent local goals | Aligned through doctrine and policies |
| Resource allocation | Coordinated | Market-based | Supervised allocation with backpressure |
| Failure handling | Mutual aid (supervision) | Individual responsibility | Cooperative supervision trees |
| Decision making | Consensus or delegation | Autonomous | Hierarchical with cooperative input |
The Actor Model as Cooperative Foundation
The Actor Model, which underpins the BEAM virtual machine, provides the theoretical foundation for cooperative systems in the Prismatic Platform. Each actor (process) is an autonomous entity that communicates exclusively through asynchronous message passing. Cooperation emerges from the protocols that processes follow when exchanging messages, not from shared memory or global state.
This shared-nothing architecture is fundamentally cooperative: processes cannot forcefully access another process's state. All interaction requires the willing participation of both parties through message send and receive. This constraint ensures that cooperation is explicit and observable, eliminating entire categories of concurrency bugs that plague shared-memory systems.
BEAM VM Cooperative Scheduling
The BEAM virtual machine implements a cooperative-preemptive hybrid scheduling model that is central to understanding cooperative systems in Elixir/OTP. Each BEAM process receives a budget of approximately 4,000 reductions (roughly equivalent to function calls), after which it yields control to the scheduler. This is cooperative in spirit -- processes are expected to perform reasonable units of work -- but preemptive in mechanism, as the scheduler enforces the yield after the reduction budget is exhausted.
defmodule Prismatic.Scheduling.CooperativeDemo do
@moduledoc """
Demonstrates cooperative scheduling properties of the BEAM VM.
Processes yield after their reduction budget, ensuring fair
scheduling across all platform processes regardless of workload.
"""
@spec demonstrate_fair_scheduling(pos_integer()) :: [map()]
def demonstrate_fair_scheduling(process_count) do
parent = self()
processes =
for i <- 1..process_count do
spawn(fn ->
start = System.monotonic_time(:microsecond)
result = do_work(1_000_000)
elapsed = System.monotonic_time(:microsecond) - start
send(parent, {:done, i, elapsed, result})
end)
end
results =
for _ <- processes do
receive do
{:done, id, elapsed, _result} -> %{process: id, elapsed_us: elapsed}
end
end
# All processes complete in roughly similar time due to
# cooperative scheduling - no single process starves others
Enum.sort_by(results, & &1.elapsed_us)
end
@spec do_work(pos_integer()) :: non_neg_integer()
defp do_work(iterations) do
Enum.reduce(1..iterations, 0, fn i, acc -> acc + rem(i, 7) end)
end
endScheduler Fairness Guarantees
The BEAM's cooperative scheduling provides several guarantees critical for cooperative systems:
- No starvation: Every runnable process will eventually receive CPU time, regardless of how many processes are running.
- Bounded latency: The reduction limit ensures that no single process can monopolize a scheduler for more than a few milliseconds.
- IO fairness: Processes waiting on IO are handled through a separate mechanism that does not consume reduction budgets.
- Dirty schedulers: Long-running NIFs and BIFs execute on separate dirty schedulers to avoid disrupting cooperative scheduling of normal processes.
OTP Cooperation Patterns
OTP provides several battle-tested patterns for building cooperative systems that go beyond simple message passing.
Supervision as Cooperative Infrastructure
Supervision trees are the primary cooperative infrastructure in OTP. A supervisor cooperates with its children by monitoring their health, restarting them when they fail, and enforcing lifecycle contracts. Children cooperate by adhering to the child_spec contract, implementing proper init/1 and terminate/2 callbacks, and respecting shutdown signals.
defmodule Prismatic.Cooperation.TeamSupervisor do
@moduledoc """
Supervises a team of cooperative agent processes that work together
on a shared objective. Implements the :rest_for_one strategy to
ensure dependent agents restart together, maintaining team coherence.
"""
use Supervisor
@spec start_link(keyword()) :: Supervisor.on_start()
def start_link(opts) do
team_name = Keyword.fetch!(opts, :team_name)
Supervisor.start_link(__MODULE__, opts, name: via_tuple(team_name))
end
@impl Supervisor
@spec init(keyword()) :: {:ok, {Supervisor.sup_flags(), [Supervisor.child_spec()]}}
def init(opts) do
team_config = Keyword.fetch!(opts, :config)
children = [
{Prismatic.Cooperation.SharedState, team_config},
{Prismatic.Cooperation.Coordinator, team_config},
{Prismatic.Cooperation.WorkerPool, team_config}
]
# :rest_for_one ensures that if SharedState crashes,
# Coordinator and WorkerPool also restart with fresh state
Supervisor.init(children, strategy: :rest_for_one)
end
defp via_tuple(name), do: {:via, Registry, {Prismatic.TeamRegistry, name}}
endGenServer Cooperation Protocols
GenServer processes cooperate through well-defined message protocols. The call/cast/info trichotomy provides different cooperation semantics:
call: Synchronous cooperation where the caller waits for a response. Used when the calling process needs the result before continuing.cast: Asynchronous cooperation where the sender fires and forgets. Used for notifications, updates, and non-blocking commands.info: Asynchronous system messages for timers, monitors, and inter-process signals.
defmodule Prismatic.Cooperation.WorkDistributor do
@moduledoc """
Distributes work items across a pool of cooperative worker processes.
Workers register their availability, receive assignments, and report
completion back to the distributor, forming a cooperative work loop.
"""
use GenServer
@type state :: %{
pending: :queue.queue(work_item()),
active: %{pid() => work_item()},
available: [pid()],
completed: non_neg_integer()
}
@type work_item :: %{id: binary(), payload: term(), assigned_at: DateTime.t() | nil}
@spec submit_work(GenServer.server(), term()) :: :ok
def submit_work(server, payload) do
GenServer.cast(server, {:submit, payload})
end
@spec register_worker(GenServer.server()) :: :ok
def register_worker(server) do
GenServer.cast(server, {:register, self()})
end
@spec report_completion(GenServer.server(), binary(), term()) :: :ok
def report_completion(server, work_id, result) do
GenServer.cast(server, {:complete, work_id, self(), result})
end
@impl GenServer
def handle_cast({:submit, payload}, state) do
item = %{id: generate_id(), payload: payload, assigned_at: nil}
case state.available do
[worker | rest] ->
assign_work(worker, item)
{:noreply, %{state | active: Map.put(state.active, worker, item), available: rest}}
[] ->
{:noreply, %{state | pending: :queue.in(item, state.pending)}}
end
end
@impl GenServer
def handle_cast({:complete, _work_id, worker, _result}, state) do
state = %{state |
active: Map.delete(state.active, worker),
completed: state.completed + 1
}
case :queue.out(state.pending) do
{{:value, item}, remaining} ->
assign_work(worker, item)
{:noreply, %{state | active: Map.put(state.active, worker, item), pending: remaining}}
{:empty, _} ->
{:noreply, %{state | available: [worker | state.available]}}
end
end
defp assign_work(worker, item) do
send(worker, {:work_assignment, %{item | assigned_at: DateTime.utc_now()}})
end
defp generate_id, do: :crypto.strong_rand_bytes(16) |> Base.encode16(case: :lower)
endPubSub for Broadcast Cooperation
The PubSub pattern enables one-to-many cooperative communication where publishers share information with all interested subscribers without needing to know who they are.
defmodule Prismatic.Cooperation.EventBus do
@moduledoc """
Platform-wide event bus enabling cooperative communication between
loosely coupled system components. Events propagate through topic
subscriptions, allowing components to react to changes without
direct coupling.
"""
@spec broadcast(binary(), term()) :: :ok
def broadcast(topic, event) do
Phoenix.PubSub.broadcast(Prismatic.PubSub, topic, event)
end
@spec subscribe(binary()) :: :ok
def subscribe(topic) do
Phoenix.PubSub.subscribe(Prismatic.PubSub, topic)
end
@spec cooperative_event_topics() :: [binary()]
def cooperative_event_topics do
[
"agent:status_change",
"quality:gate_result",
"security:threat_detected",
"osint:entity_updated",
"boot:domain_ready"
]
end
endAIAD Multi-Agent Cooperation
The Prismatic Platform's AIAD (AI Agent Definition) framework implements sophisticated multi-agent cooperation across 530+ agents organized into hierarchical tiers with cooperative protocols.
Agent Hierarchy and Cooperation Levels
| Tier | Role | Cooperation Model |
|---|---|---|
| L1 Operational | Task execution | Cooperates with peers via message passing |
| L2 Tactical | Domain coordination | Cooperates with L1 agents through task delegation |
| L3 Strategic | Cross-domain orchestration | Cooperates with L2 agents through strategic directives |
| L4 Safety-Critical | Override authority | Cooperates through veto and escalation protocols |
| L5 Supreme | Platform-wide authority | Cooperates through doctrine and policy enforcement |
Cooperative Agent Patterns
defmodule Prismatic.Agents.CooperativeProtocol do
@moduledoc """
Defines the cooperative protocol that all AIAD agents must implement.
Ensures consistent cooperation semantics across the 530+ agent ecosystem.
"""
@callback accept_task(task :: map()) :: {:ok, task_id :: binary()} | {:reject, reason :: term()}
@callback report_progress(task_id :: binary(), progress :: float()) :: :ok
@callback request_assistance(task_id :: binary(), capability :: atom()) :: {:ok, helper_pid :: pid()} | {:error, :no_helper}
@callback share_finding(finding :: map()) :: :ok
@callback yield_to_authority(authority_level :: pos_integer()) :: :ok
@spec cooperation_contract() :: map()
def cooperation_contract do
%{
max_response_time_ms: 5_000,
must_acknowledge_tasks: true,
must_report_failures: true,
must_share_findings: true,
must_respect_authority: true,
must_yield_on_request: true
}
end
endColor-Team Cooperative Security
The Color Teams architecture represents a specialized cooperative system where teams with deliberately different objectives cooperate to improve platform security. The Red Team attacks while the Blue Team defends, but both cooperate within the larger goal of identifying and closing security gaps. The Purple Team serves as the cooperative synthesis layer, ensuring that adversarial findings translate into defensive improvements.
This adversarial cooperation pattern -- where agents cooperate at the meta-level while competing at the operational level -- is one of the most sophisticated cooperation models in the platform.
Backpressure as Cooperative Flow Control
Backpressure is a cooperative flow control mechanism where downstream consumers signal upstream producers to slow down when they cannot process messages fast enough. Without backpressure, cooperative systems degrade into overwhelmed systems where message queues grow unbounded.
defmodule Prismatic.Cooperation.BackpressureAware do
@moduledoc """
Implements cooperative backpressure between producer and consumer
processes. The consumer advertises its capacity, and the producer
respects the limit, creating a cooperative flow control loop.
"""
use GenServer
@type state :: %{
demand: non_neg_integer(),
buffer: :queue.queue(term()),
consumers: [pid()]
}
@spec request_demand(GenServer.server(), non_neg_integer()) :: :ok
def request_demand(server, amount) do
GenServer.cast(server, {:demand, self(), amount})
end
@impl GenServer
def handle_cast({:produce, item}, state) do
case state.demand do
0 ->
# No demand - buffer cooperatively instead of dropping
{:noreply, %{state | buffer: :queue.in(item, state.buffer)}}
demand when demand > 0 ->
dispatch_to_consumer(item, state.consumers)
{:noreply, %{state | demand: demand - 1}}
end
end
@impl GenServer
def handle_cast({:demand, consumer, amount}, state) do
state = add_consumer(state, consumer)
{items, remaining_buffer} = drain_buffer(state.buffer, amount)
for item <- items do
dispatch_to_consumer(item, [consumer])
end
remaining_demand = amount - length(items)
{:noreply, %{state | demand: state.demand + remaining_demand, buffer: remaining_buffer}}
end
endDistributed Cooperation Across Nodes
When the Prismatic Platform runs in a distributed configuration across multiple BEAM nodes, cooperation extends across network boundaries. Erlang's distribution protocol makes remote message passing syntactically identical to local message passing, enabling cooperative patterns to scale transparently across nodes.
Cluster Cooperation Patterns
- Leader election: Nodes cooperate to select a single leader for global coordination tasks, with automatic re-election on leader failure.
- Consistent hashing: Work is distributed across nodes using consistent hashing, with cooperative handoff when nodes join or leave.
- CRDT synchronization: Conflict-free replicated data types allow cooperative state sharing without coordination overhead.
- Gossip protocols: Nodes cooperatively disseminate cluster metadata through epidemic-style gossip.
Cooperation Failure Modes
Cooperative systems can fail in ways that non-cooperative systems cannot. Understanding these failure modes is essential for building resilient cooperative architectures.
Byzantine Failures
A cooperating component that begins sending incorrect or malicious messages can corrupt the entire cooperative system. OTP's supervision model addresses this through crash-and-restart semantics: a misbehaving process is terminated and replaced with a fresh instance.
Cooperation Deadlocks
Two processes waiting for each other's response create a cooperation deadlock. OTP mitigates this through timeout-based calls (GenServer.call/3 with timeout) and the general preference for asynchronous cast operations.
Cascading Cooperation Failures
When a critical cooperative component fails, all components depending on it may also fail. Supervision trees address this through restart strategies (:one_for_all, :rest_for_one) that ensure dependent components restart together in a consistent state.
Split-Brain in Distributed Cooperation
Network partitions can split a cooperative cluster into isolated groups, each believing the other has failed. The platform uses CRDT-based state and configurable partition handling strategies to maintain cooperative consistency during network splits.
Measuring Cooperation Effectiveness
| Metric | Description | Healthy Range |
|---|---|---|
| Message Throughput | Messages exchanged per second between cooperating components | 10K-100K/s |
| Cooperation Latency | Time from cooperative request to response | < 5 ms (local), < 50 ms (remote) |
| Utilization Balance | Standard deviation of work distribution across cooperating agents | < 15% |
| Failure Recovery Time | Time from cooperative failure to restored operation | < 500 ms |
| Backpressure Events | Frequency of backpressure activation | < 1% of total operations |
Comparison with Other Cooperation Models
| Model | Communication | State | Scheduling | Example |
|---|---|---|---|---|
| BEAM/OTP | Message passing | Shared-nothing | Cooperative-preemptive | Prismatic Platform |
| Akka (JVM) | Message passing | Shared-nothing | Thread-based | Reactive applications |
| Go (goroutines) | Channels | Shared memory possible | Cooperative-preemptive | Cloud infrastructure |
| Kubernetes | API calls | External state stores | Container scheduling | Microservices |
| MPI | Message passing | Distributed memory | Cooperative | Scientific computing |
The BEAM/OTP model provides the strongest guarantees for cooperative systems due to its process isolation, lightweight processes (supporting millions of concurrent actors), and built-in distribution protocol. These properties make it uniquely suited for the Prismatic Platform's cooperative agent architecture.
Best Practices
-
Define cooperation contracts explicitly: Every cooperative interaction should have a documented contract specifying message formats, timeouts, error handling, and backpressure semantics.
-
Prefer asynchronous cooperation: Use
castandinfomessages for cooperation that does not require an immediate response. This reduces coupling and eliminates deadlock risk. -
Implement backpressure at every boundary: Every producer-consumer relationship should include backpressure to prevent unbounded queue growth.
-
Monitor cooperation health: Track message latencies, queue depths, and cooperation failure rates to detect degradation before it becomes critical.
-
Design for partial cooperation failure: Assume that any cooperating component can fail at any time. Use supervision trees, circuit breakers, and graceful degradation to maintain system availability.
-
Test cooperation under adversarial conditions: Use the Red Team to inject cooperation failures, delays, and Byzantine behavior to validate system resilience.
-
Keep cooperation protocols simple: Complex multi-step cooperation protocols are fragile. Prefer simple request-response or event-driven patterns.
Related Concepts
- Multi-Agent System -- The broader framework for autonomous agent cooperation
- Actor Model -- The theoretical foundation for BEAM cooperative processes
- Message Passing -- The communication mechanism enabling cooperation
- Collaborative Intelligence -- Intelligence emerging from agent cooperation
- Supervision Tree -- The cooperative failure handling infrastructure
- Backpressure -- Cooperative flow control between components
- Fault Tolerance -- System resilience enabled by cooperative recovery
- PubSub -- Broadcast cooperation pattern
- Concurrency -- The execution model for cooperative processes
- Agent Orchestration -- Higher-level coordination of cooperative agents
Further Reading
- Armstrong, Joe. "A History of Erlang." Proceedings of the Third ACM SIGPLAN Conference on History of Programming Languages, 2007.
- Hewitt, Carl. "A Universal Modular ACTOR Formalism for Artificial Intelligence." IJCAI, 1973.
- Wooldridge, Michael. "An Introduction to MultiAgent Systems." John Wiley & Sons, 2009.
- Prismatic Platform AIAD Standard:
.aiad/README.md
Connect & Contribute: Created by Tomas Korcak (korczis) | GitHub | GitLab | LinkedIn | Open Source under GHL