Definition

Scalability is the property of a system that describes its ability to handle a growing amount of work by adding resources to the system without requiring fundamental changes to its architecture, algorithms, or application code. A scalable system maintains acceptable performance characteristics (latency, throughput, resource utilization) as workload increases, whether that increase comes from more users, more data, more complex computations, or higher request rates.

Scalability is not binary -- it is a spectrum defined by the relationship between resources added and capacity gained. A perfectly scalable system exhibits linear scaling: doubling resources doubles capacity. In practice, coordination overhead, shared state, and Amdahl's Law introduce sublinear scaling factors that must be understood and minimized through architectural choices.

In the Prismatic Platform, scalability is architected from the ground up through the BEAM virtual machine's lightweight process model, OTP supervision trees for fault-tolerant process management, Horde for distributed process coordination across cluster nodes, and Fly.io for elastic infrastructure scaling. The platform's 115-app umbrella architecture is designed for independent scaling of individual subsystems based on their specific load profiles.

Overview

Scalability manifests in three primary dimensions, each addressing different growth vectors:

Vertical Scaling (Scale Up)

Vertical scaling increases capacity by adding more resources (CPU, memory, I/O bandwidth) to existing nodes. This is the simplest scaling approach: a larger server runs the same code faster or handles more concurrent work. Vertical scaling is limited by hardware ceilings and provides diminishing returns due to increased coordination costs within a single machine.

The BEAM excels at vertical scaling because its scheduler automatically distributes work across all available CPU cores. Adding cores to a BEAM node immediately increases concurrent processing capacity without code changes. BEAM's per-process garbage collection means memory scaling is also efficient: doubling available memory allows proportionally more concurrent processes.

Horizontal Scaling (Scale Out)

Horizontal scaling increases capacity by adding more nodes to the system. This approach has no theoretical ceiling and is the foundation for cloud-native architectures. However, horizontal scaling introduces complexity in data consistency, state management, service discovery, and network partitioning.

The BEAM's distributed computing model, built into the runtime since its inception for telephone switching, provides native support for horizontal scaling. BEAM nodes can form clusters with transparent message passing between processes on different nodes, location-transparent process registration, and distributed process monitoring. This is not an afterthought library but a core runtime capability.

Diagonal Scaling

In practice, scalable systems combine vertical and horizontal scaling. The Prismatic Platform uses vertical scaling within each Fly.io machine (multiple CPU cores, adequate memory) and horizontal scaling across machines (multi-region deployment, node clustering).

Scaling Dimensions Beyond Compute

Scalability extends beyond compute resources:

Dimension	Description	Prismatic Approach
Data Volume	Growing dataset size	PostgreSQL partitioning, ETS sharding
Request Rate	Increasing requests/second	BEAM process pools, backpressure
Team Size	More developers working concurrently	Umbrella app isolation, domain boundaries
Feature Complexity	Growing codebase and feature set	Modular architecture, composable apps
Geographic Distribution	Users across regions	Fly.io multi-region deployment

Technical Details

Amdahl's Law and Scalability Limits

Amdahl's Law defines the theoretical maximum speedup achievable through parallelization:

Speedup(N) = 1 / (S + (1-S)/N)

Where:
  N = number of parallel resources (cores, nodes)
  S = serial fraction (portion that cannot be parallelized)
  1-S = parallel fraction

For a system with 5% serial fraction:

2 nodes: 1.9x speedup (95% of linear)
10 nodes: 6.9x speedup (69% of linear)
100 nodes: 16.8x speedup (17% of linear)
1000 nodes: 19.6x speedup (2% of linear)

This demonstrates why minimizing serial fraction is critical for horizontal scaling. In BEAM systems, serial fractions arise from:

Single-process bottlenecks: A GenServer that all requests must pass through becomes a serial bottleneck. Solution: partition state across multiple processes (sharding).
Global locks: Global registration or global locks serialize distributed operations. Solution: local-first design with eventual consistency.
Ordered operations: Operations that require strict global ordering cannot be parallelized. Solution: use causal ordering or vector clocks instead of total ordering where possible.

Universal Scalability Law (USL)

Neil Gunther's Universal Scalability Law extends Amdahl's Law by adding a coherency penalty term that accounts for cross-node coordination overhead:

Capacity(N) = N / (1 + alpha*(N-1) + beta*N*(N-1))

Where:
  N = number of nodes
  alpha = contention parameter (serialization)
  beta = coherency parameter (cross-node coordination)

When beta > 0, the system exhibits retrograde behavior: adding more nodes actually decreases capacity beyond a certain point. This is the mathematical explanation for why poorly designed distributed systems get slower as they scale. The BEAM's share-nothing process model minimizes beta by avoiding shared state that requires cross-node coordination.

CAP Theorem Implications

The CAP Theorem states that a distributed system can provide at most two of three guarantees: Consistency, Availability, and Partition Tolerance. Since network partitions are inevitable in distributed systems, the practical choice is between CP (consistent but potentially unavailable during partitions) and AP (available but potentially inconsistent during partitions).

The Prismatic Platform makes context-dependent CAP tradeoffs:

Subsystem	CAP Choice	Rationale
Configuration	CP	Correctness of configuration overrides availability
Quality State	CP	Quality metrics must be accurate
Agent Registry	AP	Agent availability preferred; reconcile post-partition
Session State	AP	User session continuity preferred
OSINT Cache	AP	Stale data acceptable; freshness is best-effort

BEAM Distribution Model

The BEAM provides a built-in distribution layer with the following properties:

Node Discovery: Nodes connect via Node.connect/1 using a shared cookie for authentication. In clustered deployments, libcluster automates node discovery through DNS, Kubernetes API, or gossip protocols.
Transparent Message Passing: GenServer.call({:name, :node@host}, message) sends messages across nodes with the same semantics as local calls. The developer writes the same code for local and distributed operation.
Global Process Registration: :global and Horde provide cluster-wide process registration, ensuring exactly one instance of a singleton process across all nodes.
Distributed Monitors and Links: Process monitors and links work across nodes, enabling supervision trees to span cluster boundaries.

Implementation in Prismatic Platform

Horde-Based Distributed Coordination

The platform uses Horde for distributed process registration and supervision, with a behaviour-based backend that supports both local (ETS) and distributed (Horde) operation:

defmodule PrismaticSupervisor.Registry.Behaviour do
  @moduledoc """
  Behaviour defining the interface for process registry backends.
  Supports pluggable ETS (development) and Horde (production) implementations.
  """

  @type registry_name :: atom()
  @type process_key :: {atom(), term()}
  @type process_value :: pid()

  @callback register(registry_name(), process_key(), process_value()) ::
              {:ok, pid()} | {:error, {:already_registered, pid()}}

  @callback lookup(registry_name(), process_key()) ::
              [{pid(), term()}]

  @callback unregister(registry_name(), process_key()) :: :ok

  @callback count(registry_name()) :: non_neg_integer()
end

defmodule PrismaticSupervisor.Registry.ETS do
  @moduledoc """
  ETS-backed local registry for development and single-node deployment.
  """

  @behaviour PrismaticSupervisor.Registry.Behaviour

  @impl true
  @spec register(atom(), {atom(), term()}, pid()) ::
          {:ok, pid()} | {:error, {:already_registered, pid()}}
  def register(registry, key, pid) do
    case Registry.register(registry, key, pid) do
      {:ok, _} -> {:ok, pid}
      {:error, {:already_registered, existing}} -> {:error, {:already_registered, existing}}
    end
  end

  @impl true
  @spec lookup(atom(), {atom(), term()}) :: [{pid(), term()}]
  def lookup(registry, key) do
    Registry.lookup(registry, key)
  end

  @impl true
  @spec unregister(atom(), {atom(), term()}) :: :ok
  def unregister(registry, key) do
    Registry.unregister(registry, key)
  end

  @impl true
  @spec count(atom()) :: non_neg_integer()
  def count(registry) do
    Registry.count(registry)
  end
end

defmodule PrismaticSupervisor.Registry.Horde do
  @moduledoc """
  Horde-backed distributed registry for multi-node production deployment.
  Provides cluster-wide process registration with CRDT-based consistency.
  """

  @behaviour PrismaticSupervisor.Registry.Behaviour

  @impl true
  @spec register(atom(), {atom(), term()}, pid()) ::
          {:ok, pid()} | {:error, {:already_registered, pid()}}
  def register(registry, key, pid) do
    case Horde.Registry.register(registry, key, pid) do
      {:ok, _} -> {:ok, pid}
      {:error, {:already_registered, existing}} -> {:error, {:already_registered, existing}}
    end
  end

  @impl true
  @spec lookup(atom(), {atom(), term()}) :: [{pid(), term()}]
  def lookup(registry, key) do
    Horde.Registry.lookup(registry, key)
  end

  @impl true
  @spec unregister(atom(), {atom(), term()}) :: :ok
  def unregister(registry, key) do
    Horde.Registry.unregister(registry, key)
  end

  @impl true
  @spec count(atom()) :: non_neg_integer()
  def count(registry) do
    Horde.Registry.count(registry)
  end
end

Dynamic Process Pool Scaling

The platform uses DynamicSupervisor for elastic process pool management that scales with demand:

defmodule PrismaticScaling.ProcessPool do
  @moduledoc """
  Elastic process pool that scales worker count based on demand.
  Uses DynamicSupervisor for on-demand process creation and
  backpressure for demand management.
  """

  use DynamicSupervisor

  @type pool_config :: %{
    min_workers: pos_integer(),
    max_workers: pos_integer(),
    scale_threshold: float(),
    cooldown_ms: pos_integer()
  }

  @default_config %{
    min_workers: 4,
    max_workers: 100,
    scale_threshold: 0.8,
    cooldown_ms: 5_000
  }

  @spec start_link(keyword()) :: Supervisor.on_start()
  def start_link(opts) do
    name = Keyword.get(opts, :name, __MODULE__)
    DynamicSupervisor.start_link(__MODULE__, opts, name: name)
  end

  @impl true
  @spec init(keyword()) :: {:ok, DynamicSupervisor.sup_flags()}
  def init(_opts) do
    DynamicSupervisor.init(strategy: :one_for_one, max_restarts: 10, max_seconds: 60)
  end

  @spec add_worker(atom(), module(), keyword()) :: {:ok, pid()} | {:error, term()}
  def add_worker(pool, worker_module, args) do
    config = get_config(pool)
    current_count = DynamicSupervisor.count_children(pool).active

    case current_count < config.max_workers do
      true ->
        spec = {worker_module, args}
        DynamicSupervisor.start_child(pool, spec)

      false ->
        {:error, :pool_exhausted}
    end
  end

  @spec remove_worker(atom(), pid()) :: :ok | {:error, :not_found}
  def remove_worker(pool, pid) do
    DynamicSupervisor.terminate_child(pool, pid)
  end

  @spec pool_stats(atom()) :: map()
  def pool_stats(pool) do
    counts = DynamicSupervisor.count_children(pool)
    config = get_config(pool)

    %{
      active: counts.active,
      min: config.min_workers,
      max: config.max_workers,
      utilization: counts.active / config.max_workers,
      can_scale: counts.active < config.max_workers
    }
  end

  @spec should_scale_up?(atom()) :: boolean()
  def should_scale_up?(pool) do
    stats = pool_stats(pool)
    config = get_config(pool)
    stats.utilization >= config.scale_threshold && stats.can_scale
  end

  @spec get_config(atom()) :: pool_config()
  defp get_config(_pool), do: @default_config
end

Cluster Formation and Node Management

The platform uses libcluster for automatic node discovery in Fly.io deployments:

defmodule PrismaticCluster.Formation do
  @moduledoc """
  Cluster formation and node management for distributed deployment.
  Supports DNS-based discovery for Fly.io and gossip for local clusters.
  """

  @type node_info :: %{
    name: atom(),
    host: String.t(),
    region: String.t(),
    connected_at: DateTime.t()
  }

  @spec topology_config(atom()) :: keyword()
  def topology_config(:fly_io) do
    [
      fly6pn: [
        strategy: Cluster.Strategy.DNSPoll,
        config: [
          polling_interval: 5_000,
          query: fly_dns_query(),
          node_basename: "prismatic"
        ]
      ]
    ]
  end

  def topology_config(:local) do
    [
      gossip: [
        strategy: Cluster.Strategy.Gossip,
        config: [
          port: 45892,
          if_addr: "0.0.0.0",
          multicast_if: "0.0.0.0",
          multicast_addr: "230.1.1.251",
          multicast_ttl: 1
        ]
      ]
    ]
  end

  @spec connected_nodes() :: [node_info()]
  def connected_nodes do
    Node.list()
    |> Enum.map(fn node ->
      %{
        name: node,
        host: node |> Atom.to_string() |> String.split("@") |> List.last(),
        region: get_node_region(node),
        connected_at: DateTime.utc_now()
      }
    end)
  end

  @spec cluster_size() :: pos_integer()
  def cluster_size do
    length(Node.list()) + 1
  end

  @spec healthy?() :: boolean()
  def healthy? do
    cluster_size() >= min_cluster_size()
  end

  @spec fly_dns_query() :: String.t()
  defp fly_dns_query do
    app_name = System.get_env("FLY_APP_NAME", "prismatic")
    "#{app_name}.internal"
  end

  @spec get_node_region(atom()) :: String.t()
  defp get_node_region(_node) do
    System.get_env("FLY_REGION", "local")
  end

  @spec min_cluster_size() :: pos_integer()
  defp min_cluster_size do
    String.to_integer(System.get_env("MIN_CLUSTER_SIZE", "1"))
  end
end

Backpressure for Load Management

Scalable systems must handle overload gracefully. The platform implements backpressure through GenStage and Broadway to prevent resource exhaustion under spike loads:

defmodule PrismaticScaling.BackpressureGate do
  @moduledoc """
  Backpressure mechanism that regulates request admission
  based on current system load. Prevents overload-induced
  performance degradation.
  """

  use GenServer

  @type load_state :: :normal | :elevated | :critical | :overloaded
  @type admission_result :: :admitted | {:rejected, String.t()}

  @thresholds %{
    normal: 0.0..0.7,
    elevated: 0.7..0.85,
    critical: 0.85..0.95,
    overloaded: 0.95..1.0
  }

  @spec start_link(keyword()) :: GenServer.on_start()
  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  @spec request_admission(map()) :: admission_result()
  def request_admission(request_metadata) do
    GenServer.call(__MODULE__, {:admit, request_metadata})
  end

  @spec current_load() :: {load_state(), float()}
  def current_load do
    GenServer.call(__MODULE__, :load)
  end

  @impl true
  @spec init(keyword()) :: {:ok, map()}
  def init(_opts) do
    state = %{
      active_requests: 0,
      max_capacity: System.schedulers_online() * 100,
      load_history: []
    }

    schedule_load_sample()
    {:ok, state}
  end

  @impl true
  def handle_call({:admit, _metadata}, _from, state) do
    utilization = state.active_requests / state.max_capacity
    load_state = classify_load(utilization)

    case load_state do
      :overloaded ->
        {:reply, {:rejected, "System overloaded (#{Float.round(utilization * 100, 1)}% utilization)"}, state}

      :critical ->
        # Probabilistic rejection: reject with probability proportional to overload
        rejection_probability = (utilization - 0.85) / 0.10

        case :rand.uniform() > rejection_probability do
          true ->
            new_state = %{state | active_requests: state.active_requests + 1}
            {:reply, :admitted, new_state}

          false ->
            {:reply, {:rejected, "Load shedding active"}, state}
        end

      _ ->
        new_state = %{state | active_requests: state.active_requests + 1}
        {:reply, :admitted, new_state}
    end
  end

  @impl true
  def handle_call(:load, _from, state) do
    utilization = state.active_requests / state.max_capacity
    {:reply, {classify_load(utilization), utilization}, state}
  end

  @impl true
  def handle_info(:sample_load, state) do
    utilization = state.active_requests / state.max_capacity

    :telemetry.execute(
      [:prismatic, :scaling, :load],
      %{utilization: utilization, active: state.active_requests, capacity: state.max_capacity},
      %{state: classify_load(utilization)}
    )

    schedule_load_sample()
    {:noreply, state}
  end

  @spec classify_load(float()) :: load_state()
  defp classify_load(utilization) when utilization < 0.7, do: :normal
  defp classify_load(utilization) when utilization < 0.85, do: :elevated
  defp classify_load(utilization) when utilization < 0.95, do: :critical
  defp classify_load(_utilization), do: :overloaded

  @spec schedule_load_sample() :: reference()
  defp schedule_load_sample do
    Process.send_after(self(), :sample_load, 1_000)
  end
end

Comparison with Alternatives

BEAM vs Kubernetes-Based Scaling

Aspect	BEAM Native Scaling	Kubernetes Scaling
Process creation	~1-10 microseconds	~1-30 seconds (container startup)
Granularity	Per-request process isolation	Per-pod scaling
State migration	Built-in process handoff (Horde)	External state store required
Service discovery	Native node discovery, :global registry	DNS, service mesh (Istio)
Failover	Millisecond supervisor restart	Seconds for pod rescheduling
Code deployment	Hot code reload, zero downtime	Rolling update, brief unavailability
Complexity	Single runtime, built-in clustering	Multi-layer infrastructure stack

The BEAM provides finer-grained scaling within each node (per-process) while Kubernetes provides infrastructure-level scaling (per-pod). The Prismatic Platform uses both: BEAM for application-level scaling and Fly.io (which uses Firecracker microVMs) for infrastructure-level scaling.

Horizontal vs Vertical: When Each Applies

Scenario	Recommended Approach	Rationale
CPU-bound computation	Vertical first (more cores)	BEAM auto-scales across cores
Memory-intensive workloads	Vertical first (more RAM)	Avoids distributed memory complexity
High request rates	Horizontal (more nodes)	Distributes connection handling
Data-heavy workloads	Horizontal with sharding	Distributes storage and I/O
Geographic distribution	Horizontal (multi-region)	Reduces latency for distributed users
Development team growth	Domain-based horizontal split	Umbrella app isolation

Elixir/BEAM vs Go for Scalable Services

Both languages produce highly scalable systems, but with different characteristics:

Concurrency model: BEAM processes (isolated, message-passing) vs Go goroutines (shared memory, channels). BEAM provides stronger isolation guarantees; Go provides lower per-goroutine overhead.
Distribution: BEAM has built-in distribution primitives. Go requires external libraries (gRPC, NATS, etcd) for distributed coordination.
Fault tolerance: BEAM's supervision trees provide automatic recovery. Go relies on explicit error handling and external orchestration.
Hot code reload: BEAM supports zero-downtime code updates. Go requires process restart for code changes.

Best Practices

1. Design for Horizontal Scaling from the Start

Retrofitting horizontal scaling onto a system designed for single-node operation is difficult and error-prone. Design for distribution from the beginning, even if initial deployment is single-node. Use location-transparent registries, avoid global singletons, and minimize shared state.

2. Partition State for Parallelism

A single GenServer handling all requests becomes a serial bottleneck. Partition state across multiple processes using consistent hashing, sharding by entity ID, or domain-based partitioning. The platform's umbrella architecture provides natural partitioning boundaries.

3. Implement Backpressure Before Scaling

Before adding more capacity, ensure the system can handle overload gracefully. Backpressure mechanisms prevent cascading failures that no amount of horizontal scaling can solve. A system that crashes under load will crash regardless of how many nodes it runs on.

4. Measure Scaling Efficiency

Track the relationship between resources added and capacity gained. If doubling nodes provides less than 1.5x capacity, investigate serialization bottlenecks and coherency overhead before adding more nodes.

5. Scale Database Access Independently

Database connections do not scale linearly with application nodes. Use connection pooling, read replicas, and caching layers to prevent the database from becoming the scaling bottleneck.

6. Test at Scale

Performance testing at production scale is essential. Behavior at 10 concurrent users does not predict behavior at 10,000. Use load testing tools to validate scaling characteristics before production deployment.

Common Pitfalls

Premature Distributed Systems

Adding distributed system complexity (clustering, consensus, distributed state) before it is needed introduces failure modes and operational overhead without benefits. Start with a well-designed single-node system and scale horizontally when measurement demonstrates the need.

Ignoring Network Partitions

Distributed systems will experience network partitions. Code that assumes reliable, low-latency communication between nodes will fail in production. Design for partition tolerance using the CAP Theorem as a guide.

Shared Database as Coordination Point

Using a shared database for inter-service coordination creates a scaling bottleneck and single point of failure. Use message passing, event sourcing, or CRDT-based coordination instead.

Scaling Without Observability

Scaling a system you cannot observe is dangerous. Without telemetry, metrics, and distributed tracing, you cannot determine whether scaling is effective or identify bottlenecks. Implement observability before scaling.

Connection Explosion

Each node in an N-node cluster maintains connections to every other node, creating N*(N-1)/2 connections. At 100 nodes, that is 4,950 connections. Use connection pooling and hierarchical communication patterns to prevent connection explosion.

Stateful Session Affinity

Routing requests to specific nodes based on session state (sticky sessions) prevents horizontal scaling and complicates failover. Store session state externally (Redis, ETS with Horde) so any node can handle any request.

Use Cases

Multi-Region Deployment on Fly.io

The Prismatic Platform deploys to Fly.io with multi-region support. Fly.io's Firecracker microVMs provide fast startup times, and the BEAM's built-in clustering enables transparent process communication across regions. Requests are routed to the nearest region, with cross-region communication for data that requires global consistency.

OSINT Data Processing Pipeline

The OSINT toolbox processes data from 120+ intelligence sources concurrently. Broadway provides backpressure-aware processing with configurable concurrency limits per source, automatically scaling processing rate based on source availability and downstream capacity.

Agent Pool Management

The platform's 530+ AIAD agents are managed through DynamicSupervisor pools that scale based on demand. During high-activity periods, additional agent instances are spawned. During quiet periods, idle agents are terminated to free resources.

Quality Gate Parallel Execution

The quality gate pipeline executes independent checks in parallel using Task.async_stream/3, scaling check execution across available CPU cores. This provides linear speedup for the 11-phase pre-commit pipeline on multi-core machines.

LiveView Connection Scaling

Each LiveView connection is a BEAM process. The BEAM can handle millions of concurrent processes on a single node, enabling hundreds of thousands of concurrent LiveView connections without architectural changes. When single-node capacity is exceeded, horizontal scaling through Fly.io machine scaling provides additional capacity.

Performance: The measurable characteristics that scalability must maintain as load grows
Distributed System: Systems spanning multiple nodes that enable horizontal scaling
Load Balancing: Distribution of work across resources for optimal utilization
BEAM: Virtual machine providing lightweight processes and native distribution
Cluster: Group of connected BEAM nodes forming a distributed system
Fault Tolerance: System resilience that must be maintained during scaling
Backpressure: Flow control preventing overload during scaling transitions
Supervision Tree: OTP process hierarchy enabling scalable fault recovery
CAP Theorem: Fundamental constraint on distributed system design
Dynamic Supervisor: OTP component for elastic process pool management
Broadway: Concurrent data processing with built-in backpressure
Fly.io: Infrastructure platform for elastic deployment scaling

Connect & Contribute

Created by Tomas Korcak (korczis) | Open Source under GHL

GitHub | GitLab | LinkedIn | Contact
Developer Portal | Architecture | Meet the Creator

Scalability