Definition

Monte Carlo Verification is the probabilistic robustness testing methodology employed in the final stage of the QEVE pipeline. It systematically subjects verified conclusions to thousands of randomized perturbation scenarios -- varying evidence weights, removing signals, injecting failures, and simulating source degradation -- to quantify how stable a conclusion remains under stress. The output is not a binary pass/fail but a continuous robustness distribution that reveals which evidence is load-bearing, which assumptions are fragile, and what percentage of plausible alternative scenarios preserve the original conclusion.

The method takes its name from the Monte Carlo casino in Monaco, following the convention established by Stanislaw Ulam and John von Neumann during the Manhattan Project in the 1940s. They recognized that deterministic computation of neutron diffusion was intractable but that repeated random sampling could converge on accurate estimates. The same principle applies to epistemic verification: deterministically enumerating all possible perturbations to a belief graph is combinatorially explosive, but random sampling from the perturbation space converges on reliable robustness estimates with quantifiable error bounds.

Within the Prismatic Platform, Monte Carlo Verification occupies Stage 5 of the QEVE pipeline, executing after structural validation (Stage 2), logical consistency checking (Stage 3), and Lean4 formal proof (Stage 4). This ordering is deliberate: Monte Carlo simulation is computationally expensive, and running it against structurally broken or logically inconsistent belief graphs would produce meaningless results. The prior stages act as filters, ensuring that only well-formed, logically sound, formally verified conclusions undergo robustness testing.

Overview

Monte Carlo methods are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results. In the verification domain, the technique transforms the question "is this conclusion robust?" into a statistical estimation problem: "what fraction of plausible perturbation scenarios preserve the conclusion?" By running thousands of independent trials, each applying a randomized combination of evidence modifications, the platform builds a statistical profile of conclusion stability that is far more informative than any single deterministic test.

The approach is particularly well-suited to the NABLA Infinity epistemic framework because it naturally handles the uncertainty and plurality that NABLA axioms demand. Rather than claiming absolute certainty about a conclusion's robustness, Monte Carlo Verification produces a probability distribution with quantified error bounds -- an honest representation of what the evidence supports.

Mathematical Foundations

The Law of Large Numbers

Monte Carlo methods derive their validity from the Strong Law of Large Numbers (SLLN). For a sequence of independent, identically distributed random variables X_1, X_2, ..., X_n with expected value E[X], the sample mean converges almost surely to the true expected value as n approaches infinity:

P(lim_{n->inf} (1/n) * sum(X_i) = E[X]) = 1

In the verification context, each X_i represents the outcome of a single perturbation scenario (1 if the conclusion survives, 0 if it breaks). The sample mean converges to the true robustness probability. With 10,000 simulation runs, the standard error of the robustness estimate is approximately 1/sqrt(10000) = 0.01, meaning the estimate is accurate to within approximately 1 percentage point with high probability.

Convergence Rate and Error Bounds

The Central Limit Theorem provides confidence intervals for the robustness estimate. For n simulation runs producing a robustness estimate p-hat, the 95% confidence interval is:

p_hat +/- 1.96 * sqrt(p_hat * (1 - p_hat) / n)

For a typical robustness score of 0.72 with n = 10,000 runs:

0.72 +/- 1.96 * sqrt(0.72 * 0.28 / 10000)
= 0.72 +/- 0.0088
= [0.7112, 0.7288]

This means the platform can report robustness scores with approximately +/- 1% precision at the 95% confidence level. Increasing to 100,000 runs would narrow this to +/- 0.3%, at the cost of 10x computational expense. The default of 10,000 runs represents a pragmatic balance between precision and throughput.

Stratified Sampling

Rather than purely random perturbation, the platform employs stratified Monte Carlo sampling to ensure coverage across the perturbation space. The perturbation dimensions are partitioned into strata:

Stratum	Perturbation Type	Allocation
Weight variation	+/-5%, +/-10%, +/-20% evidence weight changes	30% of runs
Signal removal	Single signal, pairwise, triple removal	25% of runs
Source failure	Complete source group failure simulation	20% of runs
Temporal shift	Time decay acceleration and deceleration	15% of runs
Combined	Multi-dimensional simultaneous perturbation	10% of runs

Stratified sampling reduces variance compared to pure random sampling, ensuring that rare but critical perturbation types (such as simultaneous multi-signal removal) are adequately represented even in a finite sample.

Perturbation Taxonomy

The Monte Carlo engine applies five categories of perturbation, each designed to stress-test a different dimension of conclusion stability.

Weight Perturbation

Evidence weights are randomly adjusted within configurable bounds. The default perturbation range is +/-20%, meaning a signal with weight 0.80 could be perturbed to anywhere in [0.64, 0.96]. Weight perturbation tests sensitivity to evidence strength assumptions -- if a conclusion survives only when a particular signal has weight above 0.75, the conclusion is sensitive to that signal's precise strength estimate.

The perturbation follows a truncated normal distribution centered on the original weight, ensuring that small perturbations are more likely than extreme ones. This reflects the real-world assumption that evidence weights are approximately correct but carry estimation uncertainty.

Signal Removal

Individual signals and signal combinations are systematically removed from the belief graph to identify load-bearing evidence. A signal is load-bearing if its removal causes the conclusion's robustness to drop below the decision threshold. Signal removal analysis directly supports the Signal Plurality axiom by identifying conclusions that effectively depend on a single critical signal despite nominally satisfying the two-signal minimum.

The engine tests:

Single removal: Each signal removed independently (n scenarios for n signals)
Pairwise removal: Each pair of signals removed simultaneously (n-choose-2 scenarios)
Critical path removal: Signals along the shortest evidence path to the conclusion removed together

Source Group Failure

Entire source groups (as defined by the independence_group field in evidence data) are removed to simulate source compromise or unavailability. This tests whether the conclusion survives if an entire intelligence provider, database, or scanning infrastructure becomes unavailable.

Source group failure is particularly relevant for EASM assessments where multiple signals may originate from the same scanning infrastructure. If a security rating depends heavily on Censys data, source group failure analysis reveals this dependency explicitly.

Temporal Shift

Time decay parameters are perturbed to test sensitivity to evidence freshness assumptions. The engine accelerates and decelerates decay functions, simulating scenarios where evidence ages faster or slower than expected. This tests whether conclusions are robust to uncertainty in the decay model itself, not just uncertainty in the evidence.

Temporal shift perturbation interacts directly with the NABLA Infinity Time Decay axiom, verifying that the axiom's enforcement produces stable results across a range of plausible decay parameterizations.

Combined Perturbation

The most aggressive stratum applies multiple perturbation types simultaneously: weight changes combined with signal removal, temporal shifts combined with source failures. Combined perturbation reveals interaction effects that single-dimension perturbation misses. A conclusion might survive any single perturbation type but fail when two types coincide.

Robustness Scoring

The robustness score is the primary output of Monte Carlo Verification and feeds directly into the confidence scoring formula used throughout the platform.

Score Computation

robustness_score = (scenarios_preserving_conclusion / total_scenarios)

For 10,000 simulation runs where 7,200 preserve the original conclusion:

robustness_score = 7200 / 10000 = 0.72

Score Interpretation

Score Range	Interpretation	Recommendation
0.90 - 1.00	Highly robust	Conclusion is stable under extreme perturbation
0.75 - 0.89	Robust	Conclusion is reliable for standard operations
0.50 - 0.74	Moderately robust	Conclusion should be flagged for additional review
0.25 - 0.49	Fragile	Conclusion should not be acted upon without additional evidence
0.00 - 0.24	Extremely fragile	Conclusion is likely an artifact of specific evidence configuration

The score feeds into the multiplicative confidence scoring formula:

final_confidence = belief_strength * robustness_score * (1 - contradiction_index)

A robustness score of 0.40 will drag final confidence below actionable thresholds regardless of how strong the underlying evidence appears, ensuring that fragile conclusions are never mistaken for reliable ones.

Sensitivity Report

Beyond the aggregate score, the Monte Carlo engine produces a sensitivity report identifying:

Load-bearing signals: Signals whose removal drops robustness by more than 20 percentage points
Redundant signals: Signals whose removal has less than 2 percentage points impact
Critical thresholds: Weight values at which the conclusion transitions from surviving to failing
Interaction effects: Signal pairs or groups whose combined removal has disproportionate impact

This report enables analysts to understand not just whether a conclusion is robust but why it is or is not, and what additional evidence would most effectively improve robustness.

Integration with Formal Verification

Monte Carlo Verification and Lean4 formal verification serve complementary roles in the QEVE pipeline. Formal verification answers: "Is this conclusion logically necessary?" Monte Carlo verification answers: "Is this conclusion practically stable?"

A conclusion can be formally necessary yet practically fragile. Consider a theorem that is true given its axioms, where one axiom depends on a signal with weight 0.51. The formal proof is valid, but the Monte Carlo analysis reveals that a 2% weight perturbation would invalidate the axiom and collapse the proof. This interaction between formal and probabilistic verification is why both stages are required.

The Trinity Gate evaluates formal proofs from Stage 4 and robustness scores from Stage 5 independently. A conclusion must pass both: formally necessary AND practically robust. Neither alone is sufficient.

Implementation Architecture

The Prismatic Platform implements Monte Carlo Verification using Elixir's concurrency primitives for parallel simulation execution.

Parallel Execution Model

Each simulation run is independent, making Monte Carlo Verification embarrassingly parallel. The engine uses Task.async_stream/3 to distribute runs across available BEAM schedulers:

defmodule Prismatic.MonteCarloVerification.Engine do
  @moduledoc """
  Monte Carlo Verification engine using parallel simulation execution.
  Distributes perturbation scenarios across BEAM schedulers for
  maximum throughput on multi-core systems.
  """

  @default_runs 10_000
  @default_concurrency System.schedulers_online()

  @type perturbation :: %{
    type: :weight | :signal_removal | :source_failure | :temporal | :combined,
    parameters: map(),
    seed: non_neg_integer()
  }

  @type run_result :: %{
    perturbation: perturbation(),
    conclusion_preserved: boolean(),
    confidence_after: float(),
    elapsed_us: non_neg_integer()
  }

  @spec verify(map(), keyword()) :: {:ok, map()} | {:error, term()}
  def verify(belief_graph, opts \\ []) do
    runs = Keyword.get(opts, :runs, @default_runs)
    concurrency = Keyword.get(opts, :concurrency, @default_concurrency)

    perturbations = generate_stratified_perturbations(belief_graph, runs)

    results =
      perturbations
      |> Task.async_stream(
        &execute_single_run(belief_graph, &1),
        max_concurrency: concurrency,
        timeout: 30_000
      )
      |> Enum.map(fn {:ok, result} -> result end)

    robustness_score = compute_robustness(results)
    sensitivity = compute_sensitivity(results, belief_graph)

    {:ok, %{
      robustness_score: robustness_score,
      total_runs: runs,
      preserved_count: Enum.count(results, & &1.conclusion_preserved),
      sensitivity_report: sensitivity,
      confidence_interval: compute_ci(robustness_score, runs)
    }}
  end

  defp generate_stratified_perturbations(belief_graph, total_runs) do
    weight_runs = round(total_runs * 0.30)
    signal_runs = round(total_runs * 0.25)
    source_runs = round(total_runs * 0.20)
    temporal_runs = round(total_runs * 0.15)
    combined_runs = total_runs - weight_runs - signal_runs - source_runs - temporal_runs

    Enum.concat([
      generate_weight_perturbations(belief_graph, weight_runs),
      generate_signal_perturbations(belief_graph, signal_runs),
      generate_source_perturbations(belief_graph, source_runs),
      generate_temporal_perturbations(belief_graph, temporal_runs),
      generate_combined_perturbations(belief_graph, combined_runs)
    ])
  end

  defp execute_single_run(belief_graph, perturbation) do
    start = System.monotonic_time(:microsecond)
    perturbed = apply_perturbation(belief_graph, perturbation)
    confidence = evaluate_conclusion(perturbed)
    elapsed = System.monotonic_time(:microsecond) - start

    %{
      perturbation: perturbation,
      conclusion_preserved: confidence >= perturbed.decision_threshold,
      confidence_after: confidence,
      elapsed_us: elapsed
    }
  end

  defp compute_robustness(results) do
    preserved = Enum.count(results, & &1.conclusion_preserved)
    Float.round(preserved / length(results), 4)
  end

  defp compute_ci(p_hat, n) do
    margin = 1.96 * :math.sqrt(p_hat * (1 - p_hat) / n)
    %{lower: Float.round(p_hat - margin, 4), upper: Float.round(p_hat + margin, 4)}
  end
end

Deterministic Reproducibility

Despite using random perturbations, every Monte Carlo run is reproducible. The engine seeds its random number generator with a deterministic seed derived from the belief graph hash and a run identifier. This means that given the same belief graph and the same run configuration, the engine produces identical results. Reproducibility is essential for audit trail compliance and debugging.

ETS-Backed Result Caching

Intermediate results are stored in ETS (Erlang Term Storage) tables for rapid aggregation. The sensitivity analysis requires cross-referencing individual run outcomes with their perturbation parameters, which ETS enables without serialization overhead. On a system with 8 CPU cores, the 10,000 runs execute in approximately 8x less wall-clock time than sequential execution. The BEAM's lightweight process model means each simulation run carries minimal overhead -- approximately 2KB per process versus megabytes per thread in JVM or OS thread-based systems.

Comparison with Traditional Testing

Dimension	Traditional Testing	Property-Based Testing	Monte Carlo Verification
Coverage	Specific test cases	Random inputs within constraints	Random perturbations of evidence
Scope	Code correctness	Property universality	Conclusion robustness
Output	Pass/fail per case	Counterexample or pass	Robustness distribution
Failure mode	Missed edge case	Shrunk counterexample	Sensitivity report
Guarantees	None beyond tested cases	Statistical for tested property	Probabilistic for robustness

Property-based testing tests whether code properties hold across random inputs. Monte Carlo Verification tests whether epistemic conclusions hold across random evidence perturbations. The domains are orthogonal: one verifies software correctness, the other verifies reasoning stability.

Application in Due Diligence

Monte Carlo Verification is particularly valuable in due diligence contexts where stakeholders need to understand not just what the risk assessment says but how sensitive it is to assumptions.

Consider an acquisition due diligence assessment where QEVE produces a risk rating of "elevated" with confidence 0.87. The Monte Carlo sensitivity report reveals:

Removing the sanctions database signal drops robustness from 72% to 41%
The ownership structure signal has a critical threshold at weight 0.60 (current weight: 0.65)
Media reputation signals are redundant (removal impact: 1.3 percentage points)

This tells the acquisition team precisely where to focus additional investigation: the sanctions link is the load-bearing evidence, the ownership analysis needs independent corroboration, and the media signals can be deprioritized.

Best Practices

Start with stratified sampling over pure random. Stratified allocation ensures adequate coverage of rare but critical perturbation types, improving the quality of sensitivity analysis without increasing total run count.
Use adaptive run counts for efficiency. Begin with 1,000 runs to get a rough estimate. If the confidence interval is wide (greater than +/- 5%), increase to 10,000 or 100,000 runs. This avoids spending computational resources on conclusions that are clearly robust or clearly fragile.
Record full perturbation parameters for each run. The sensitivity report is only as good as the traceability of individual run results to their perturbation inputs. Store perturbation seeds and parameters alongside outcomes.
Validate the perturbation model against adversarial review. The robustness score is only meaningful if the perturbation model reflects realistic threats. Periodically review perturbation distributions with the Red Team to ensure they match adversarial reality.
Combine with formal verification, never replace it. Monte Carlo Verification and Lean4 formal proofs answer different questions. A conclusion that passes both has much stronger epistemic standing than one that passes either alone.

Limitations and Edge Cases

Monte Carlo Verification has known limitations that the platform addresses through complementary methods:

Rare events: Perturbation scenarios with very low probability may not appear in 10,000 runs. The platform mitigates this through stratified sampling and importance sampling for tail risk analysis.

Correlation blindness: Standard Monte Carlo treats perturbations as independent, but real-world evidence degradation often occurs in correlated patterns (e.g., a regulatory change affects multiple compliance signals simultaneously). The combined perturbation stratum partially addresses this, but correlation modeling remains an active area of platform development.

Computational cost: 10,000 runs per conclusion is expensive for real-time applications. The platform uses adaptive sampling -- starting with 1,000 runs and increasing only if the initial estimate has high variance -- to balance speed with precision.

Model dependency: The robustness score depends on the perturbation model (which types of perturbation, what distributions, what bounds). If the perturbation model does not reflect realistic threats to the evidence, the robustness score may be misleadingly high. The Red Team periodically reviews and challenges the perturbation model to ensure it reflects adversarial reality.

Stationarity assumption: The engine assumes that the underlying evidence distribution is stationary during the simulation run. If evidence is actively changing (e.g., during a live incident), the robustness score may not reflect the current evidence state. The platform addresses this by timestamping snapshots and re-running verification when evidence changes significantly.

QEVE -- Verification engine where Monte Carlo Verification operates as Stage 5
Lean4 -- Formal proof system complementing probabilistic robustness testing
Trinity Gate -- Verification gate consuming both formal proofs and robustness scores
NABLA Infinity -- Epistemic framework whose axioms are stress-tested by perturbation
Confidence Threshold -- Decision thresholds applied to Monte Carlo robustness-adjusted scores
Epistemic Robustness -- The quality that Monte Carlo Verification quantifies
Belief Graph -- The data structure subjected to perturbation analysis
Confidence Scoring -- The formula incorporating robustness scores
Property-Based Testing -- Complementary randomized testing for software correctness
Signal Plurality -- Axiom whose effective enforcement Monte Carlo analysis validates
White Team -- Verification team that interprets and acts on robustness results
Red Team -- Adversarial team that challenges the perturbation model
Audit Trail -- Immutable logging of all Monte Carlo run parameters and results

Connect & Contribute

Created by Tomas Korcak (korczis) | Open Source under GHL

GitHub | GitLab | LinkedIn | Contact
Developer Portal | Architecture | Meet the Creator

Monte Carlo Verification