Bias Detection

quality

Definition

Bias Detection is the systematic process of identifying, measuring, and mitigating cognitive, algorithmic, and epistemic biases that can distort decision-making, data analysis, and intelligence outputs. In software systems, bias manifests in multiple forms: training data that underrepresents certain populations, analysis pipelines that amplify pre-existing assumptions, confidence scoring that overweights certain sources, and human analysts who unconsciously seek confirming evidence while ignoring contradictions.

Within the Prismatic Platform, bias detection is deeply integrated into the NABLA Infinity epistemic framework, which treats bias as an existential threat to system integrity. The seven non-negotiable axioms -- Signal Plurality, Contradiction Preservation, Absence Informative, Time Decay, Unknown Valid, Source Independence, and Provenance Mandatory -- collectively form an anti-bias architecture that makes it structurally difficult for biased conclusions to survive the analysis pipeline.

Overview

Bias in information systems is insidious precisely because it is invisible to those affected by it. A confirmation-biased analysis feels more confident, not less. A selection-biased dataset appears comprehensive to those who do not know what is missing. An algorithmically biased model produces outputs that look correct because the evaluation metrics themselves share the same bias.

The Prismatic Platform confronts bias across three distinct domains:

Cognitive Bias in Human Analysis

Human analysts and developers bring cognitive biases to every decision. Confirmation bias leads to seeking evidence that supports existing hypotheses. Anchoring bias causes over-reliance on the first piece of information encountered. Availability bias weights recent or dramatic events disproportionately. The platform's structured decision-making processes and epistemic frameworks are designed to surface and counteract these natural human tendencies.

Algorithmic Bias in Automated Systems

Machine learning models, scoring algorithms, and classification systems can embed and amplify biases present in their training data or design assumptions. The platform's security rating system, OSINT intelligence fusion, and entity resolution algorithms all require systematic bias testing to ensure their outputs do not systematically disadvantage particular categories of entities.

Epistemic Bias in Knowledge Systems

Knowledge representation systems can encode structural biases through what they choose to model, how they weight evidence, and which sources they consider authoritative. The NABLA Infinity framework directly addresses epistemic bias through mandatory axioms that enforce evidence plurality and contradiction preservation.

Technical Details

Bias Detection Engine

The Prismatic Platform implements a multi-dimensional bias detection engine:

defmodule PrismaticBias.DetectionEngine do
  @moduledoc """
  Core bias detection engine that analyzes data pipelines,
  decision processes, and intelligence outputs for cognitive,
  algorithmic, and epistemic biases. Integrates with the
  NABLA Infinity axiom enforcement system.
  """

  @type bias_finding :: %{
    type: bias_type(),
    severity: :low | :medium | :high | :critical,
    source: String.t(),
    description: String.t(),
    evidence: [term()],
    remediation: String.t(),
    confidence: float()
  }

  @type bias_type ::
    :confirmation | :selection | :survivorship | :anchoring |
    :availability | :source_dependency | :temporal | :sampling |
    :algorithmic | :representation | :measurement | :aggregation

  @doc """
  Performs a comprehensive bias scan across the specified
  analysis pipeline. Returns a list of bias findings with
  severity, evidence, and remediation recommendations.
  """
  @spec scan(atom(), keyword()) :: {:ok, [bias_finding()]} | {:error, term()}
  def scan(pipeline, opts \\ []) do
    scanners = [
      &detect_confirmation_bias/2,
      &detect_selection_bias/2,
      &detect_source_dependency/2,
      &detect_temporal_bias/2,
      &detect_representation_bias/2,
      &detect_aggregation_bias/2
    ]

    findings =
      scanners
      |> Task.async_stream(fn scanner -> scanner.(pipeline, opts) end,
        max_concurrency: System.schedulers_online(),
        timeout: 30_000
      )
      |> Enum.flat_map(fn
        {:ok, {:ok, results}} -> results
        {:ok, {:error, _reason}} -> []
        {:exit, _reason} -> []
      end)
      |> Enum.sort_by(& &1.severity, &severity_order/1)

    {:ok, findings}
  end

  defp severity_order(:critical), do: 0
  defp severity_order(:high), do: 1
  defp severity_order(:medium), do: 2
  defp severity_order(:low), do: 3

  @doc """
  Detects confirmation bias by checking if the analysis
  pipeline systematically filters out contradicting evidence
  or overweights confirming evidence.
  """
  @spec detect_confirmation_bias(atom(), keyword()) :: {:ok, [bias_finding()]}
  def detect_confirmation_bias(pipeline, _opts) do
    with {:ok, inputs} <- get_pipeline_inputs(pipeline),
         {:ok, outputs} <- get_pipeline_outputs(pipeline) do
      dropped_contradictions = find_dropped_contradictions(inputs, outputs)

      findings =
        if length(dropped_contradictions) > 0 do
          [%{
            type: :confirmation,
            severity: severity_for_count(length(dropped_contradictions)),
            source: "#{pipeline}",
            description: "Pipeline drops #{length(dropped_contradictions)} contradicting signals",
            evidence: Enum.take(dropped_contradictions, 5),
            remediation: "Enable contradiction preservation per NABLA axiom",
            confidence: 0.85
          }]
        else
          []
        end

      {:ok, findings}
    end
  end

  defp find_dropped_contradictions(inputs, outputs) do
    input_signals = extract_signal_directions(inputs)
    output_signals = extract_signal_directions(outputs)

    contradicting_inputs =
      Enum.filter(input_signals, fn signal ->
        Enum.any?(input_signals, fn other ->
          signal.direction != other.direction and signal.topic == other.topic
        end)
      end)

    Enum.filter(contradicting_inputs, fn signal ->
      not Enum.any?(output_signals, fn out ->
        out.id == signal.id
      end)
    end)
  end

  defp extract_signal_directions(data) do
    Enum.map(data, fn item ->
      %{
        id: Map.get(item, :id, Ecto.UUID.generate()),
        topic: Map.get(item, :topic, :unknown),
        direction: Map.get(item, :direction, :neutral)
      }
    end)
  end

  defp severity_for_count(count) when count > 10, do: :critical
  defp severity_for_count(count) when count > 5, do: :high
  defp severity_for_count(count) when count > 2, do: :medium
  defp severity_for_count(_count), do: :low

  defp get_pipeline_inputs(_pipeline), do: {:ok, []}
  defp get_pipeline_outputs(_pipeline), do: {:ok, []}

  defp detect_selection_bias(_pipeline, _opts), do: {:ok, []}
  defp detect_source_dependency(_pipeline, _opts), do: {:ok, []}
  defp detect_temporal_bias(_pipeline, _opts), do: {:ok, []}
  defp detect_representation_bias(_pipeline, _opts), do: {:ok, []}
  defp detect_aggregation_bias(_pipeline, _opts), do: {:ok, []}
end

NABLA Axiom Enforcement for Bias Prevention

The NABLA Infinity framework's seven axioms directly prevent specific bias categories:

defmodule PrismaticBias.NablaEnforcement do
  @moduledoc """
  Enforces NABLA Infinity axioms as structural bias prevention
  mechanisms. Each axiom corresponds to specific bias categories
  that it prevents when properly enforced.
  """

  @axiom_bias_mapping %{
    signal_plurality: [:confirmation, :anchoring, :availability],
    contradiction_preservation: [:confirmation, :cherry_picking],
    absence_informative: [:survivorship, :selection],
    time_decay: [:temporal, :recency, :anchoring],
    unknown_valid: [:overconfidence, :premature_closure],
    source_independence: [:source_dependency, :echo_chamber],
    provenance_mandatory: [:attribution, :authority_bias]
  }

  @type axiom :: :signal_plurality | :contradiction_preservation |
    :absence_informative | :time_decay | :unknown_valid |
    :source_independence | :provenance_mandatory

  @type enforcement_result :: %{
    axiom: axiom(),
    status: :compliant | :violated | :warning,
    biases_prevented: [atom()],
    biases_detected: [atom()],
    evidence: [term()]
  }

  @doc """
  Checks a belief or claim against all seven NABLA axioms,
  reporting which biases are prevented and which are detected.
  """
  @spec enforce_all(map()) :: {:ok, [enforcement_result()]} | {:error, term()}
  def enforce_all(claim) do
    results =
      @axiom_bias_mapping
      |> Enum.map(fn {axiom, prevented_biases} ->
        {status, detected} = check_axiom(axiom, claim)

        %{
          axiom: axiom,
          status: status,
          biases_prevented: if(status == :compliant, do: prevented_biases, else: []),
          biases_detected: detected,
          evidence: extract_evidence(axiom, claim)
        }
      end)

    {:ok, results}
  end

  @doc """
  Enforces Signal Plurality: requires minimum 2 independent
  signals before establishing any belief. Prevents confirmation
  bias, anchoring bias, and availability bias.
  """
  @spec check_signal_plurality(map()) :: :compliant | {:violated, [atom()]}
  def check_signal_plurality(claim) do
    signals = Map.get(claim, :signals, [])
    independent_sources = signals |> Enum.map(& &1.source) |> Enum.uniq()

    if length(independent_sources) >= 2 do
      :compliant
    else
      {:violated, [:confirmation, :anchoring, :availability]}
    end
  end

  @doc """
  Enforces Contradiction Preservation: requires that contradicting
  signals are preserved alongside supporting ones. Prevents
  confirmation bias and cherry-picking.
  """
  @spec check_contradiction_preservation(map()) :: :compliant | {:violated, [atom()]}
  def check_contradiction_preservation(claim) do
    signals = Map.get(claim, :signals, [])
    directions = Enum.map(signals, & &1.direction) |> Enum.uniq()

    cond do
      length(signals) < 2 ->
        :compliant

      length(directions) == 1 and length(signals) > 3 ->
        {:violated, [:confirmation, :cherry_picking]}

      true ->
        :compliant
    end
  end

  @doc """
  Enforces Source Independence: checks that supporting signals
  come from genuinely independent sources, not derived or
  correlated sources. Prevents echo chamber effects.
  """
  @spec check_source_independence(map()) :: :compliant | {:violated, [atom()]}
  def check_source_independence(claim) do
    signals = Map.get(claim, :signals, [])
    sources = Enum.map(signals, & &1.source)
    source_families = Enum.map(sources, &get_source_family/1) |> Enum.uniq()

    if length(source_families) >= 2 do
      :compliant
    else
      {:violated, [:source_dependency, :echo_chamber]}
    end
  end

  defp check_axiom(:signal_plurality, claim) do
    case check_signal_plurality(claim) do
      :compliant -> {:compliant, []}
      {:violated, biases} -> {:violated, biases}
    end
  end

  defp check_axiom(:contradiction_preservation, claim) do
    case check_contradiction_preservation(claim) do
      :compliant -> {:compliant, []}
      {:violated, biases} -> {:violated, biases}
    end
  end

  defp check_axiom(:source_independence, claim) do
    case check_source_independence(claim) do
      :compliant -> {:compliant, []}
      {:violated, biases} -> {:violated, biases}
    end
  end

  defp check_axiom(_axiom, _claim), do: {:compliant, []}

  defp extract_evidence(_axiom, claim) do
    Map.get(claim, :signals, []) |> Enum.take(3)
  end

  defp get_source_family(source) when is_atom(source), do: source
  defp get_source_family(source) when is_binary(source), do: String.split(source, ".") |> List.first()
end

Statistical Bias Testing

For algorithmic outputs, the platform employs statistical tests:

defmodule PrismaticBias.StatisticalTests do
  @moduledoc """
  Statistical bias testing for algorithmic outputs. Applies
  fairness metrics and distribution tests to detect systematic
  biases in scoring, classification, and ranking algorithms.
  """

  @doc """
  Computes the disparate impact ratio between two groups.
  A ratio below 0.8 or above 1.25 indicates potential
  algorithmic bias per the four-fifths rule.
  """
  @spec disparate_impact(list(), list(), (term() -> boolean())) :: float()
  def disparate_impact(group_a, group_b, positive_outcome_fn) do
    rate_a = positive_rate(group_a, positive_outcome_fn)
    rate_b = positive_rate(group_b, positive_outcome_fn)

    if rate_b == 0.0, do: :infinity, else: rate_a / rate_b
  end

  @doc """
  Performs a chi-squared test for independence between
  group membership and outcome. Returns the test statistic
  and p-value. A low p-value suggests the outcome depends
  on group membership, indicating potential bias.
  """
  @spec chi_squared_test(list(), list(), (term() -> boolean())) ::
    {:ok, %{statistic: float(), p_value: float(), biased: boolean()}}
  def chi_squared_test(group_a, group_b, outcome_fn) do
    a_pos = Enum.count(group_a, outcome_fn)
    a_neg = length(group_a) - a_pos
    b_pos = Enum.count(group_b, outcome_fn)
    b_neg = length(group_b) - b_pos

    total = a_pos + a_neg + b_pos + b_neg
    expected_a_pos = (a_pos + a_neg) * (a_pos + b_pos) / total
    expected_a_neg = (a_pos + a_neg) * (a_neg + b_neg) / total
    expected_b_pos = (b_pos + b_neg) * (a_pos + b_pos) / total
    expected_b_neg = (b_pos + b_neg) * (a_neg + b_neg) / total

    chi2 =
      chi_term(a_pos, expected_a_pos) +
      chi_term(a_neg, expected_a_neg) +
      chi_term(b_pos, expected_b_pos) +
      chi_term(b_neg, expected_b_neg)

    p_value = 1.0 - chi_squared_cdf(chi2, 1)

    {:ok, %{
      statistic: chi2,
      p_value: p_value,
      biased: p_value < 0.05
    }}
  end

  @doc """
  Computes equalized odds difference: the maximum difference
  in true positive rates and false positive rates between
  groups. A value close to 0 indicates fairness.
  """
  @spec equalized_odds_difference(list(), list(), (term() -> boolean()), (term() -> boolean())) ::
    float()
  def equalized_odds_difference(group_a, group_b, prediction_fn, actual_fn) do
    tpr_a = true_positive_rate(group_a, prediction_fn, actual_fn)
    tpr_b = true_positive_rate(group_b, prediction_fn, actual_fn)
    fpr_a = false_positive_rate(group_a, prediction_fn, actual_fn)
    fpr_b = false_positive_rate(group_b, prediction_fn, actual_fn)

    max(abs(tpr_a - tpr_b), abs(fpr_a - fpr_b))
  end

  defp positive_rate(group, outcome_fn) do
    if length(group) == 0 do
      0.0
    else
      Enum.count(group, outcome_fn) / length(group)
    end
  end

  defp true_positive_rate(group, prediction_fn, actual_fn) do
    actual_positives = Enum.filter(group, actual_fn)

    if length(actual_positives) == 0 do
      0.0
    else
      Enum.count(actual_positives, prediction_fn) / length(actual_positives)
    end
  end

  defp false_positive_rate(group, prediction_fn, actual_fn) do
    actual_negatives = Enum.reject(group, actual_fn)

    if length(actual_negatives) == 0 do
      0.0
    else
      Enum.count(actual_negatives, prediction_fn) / length(actual_negatives)
    end
  end

  defp chi_term(observed, expected) when expected > 0 do
    :math.pow(observed - expected, 2) / expected
  end

  defp chi_term(_observed, _expected), do: 0.0

  defp chi_squared_cdf(x, _df) do
    :math.erf(:math.sqrt(x / 2))
  end
end

Red Team Bias Adversarial Testing

The Red team conducts adversarial bias testing to probe for hidden biases:

defmodule PrismaticBias.AdversarialTesting do
  @moduledoc """
  Red team adversarial bias testing that probes platform
  systems for hidden biases through targeted perturbation,
  counterfactual analysis, and stress testing of fairness
  assumptions.
  """

  @doc """
  Performs counterfactual bias testing by changing protected
  attributes while keeping all other inputs constant. If
  the output changes, the system may be biased with respect
  to that attribute.
  """
  @spec counterfactual_test(
    (map() -> term()),
    map(),
    atom(),
    [term()]
  ) :: {:ok, %{biased: boolean(), variations: map()}}
  def counterfactual_test(system_fn, base_input, attribute, values) do
    base_output = system_fn.(base_input)

    variations =
      Enum.map(values, fn value ->
        modified_input = Map.put(base_input, attribute, value)
        modified_output = system_fn.(modified_input)
        {value, modified_output}
      end)
      |> Map.new()

    unique_outputs = [base_output | Map.values(variations)] |> Enum.uniq()
    biased = length(unique_outputs) > 1

    {:ok, %{biased: biased, variations: variations}}
  end

  @doc """
  Performs perturbation testing by adding random noise to
  non-protected attributes. Systems that are overly sensitive
  to small perturbations may be fragile to bias-inducing
  data distribution shifts.
  """
  @spec perturbation_test(
    (map() -> term()),
    map(),
    [atom()],
    float()
  ) :: {:ok, %{stable: boolean(), sensitivity: float()}}
  def perturbation_test(system_fn, base_input, numeric_fields, noise_level) do
    base_output = system_fn.(base_input)
    num_trials = 100

    different_outputs =
      Enum.count(1..num_trials, fn _i ->
        perturbed = add_noise(base_input, numeric_fields, noise_level)
        system_fn.(perturbed) != base_output
      end)

    sensitivity = different_outputs / num_trials
    stable = sensitivity < 0.1

    {:ok, %{stable: stable, sensitivity: sensitivity}}
  end

  defp add_noise(input, fields, noise_level) do
    Enum.reduce(fields, input, fn field, acc ->
      case Map.get(acc, field) do
        value when is_number(value) ->
          noise = (:rand.uniform() - 0.5) * 2 * noise_level * abs(value)
          Map.put(acc, field, value + noise)

        _other ->
          acc
      end
    end)
  end
end

Implementation in the Prismatic Platform

NABLA Infinity Integration

The NABLA Infinity epistemic framework serves as the platform's primary structural bias prevention mechanism. Every claim, belief, and decision that passes through the Trinity Gate must satisfy all seven axioms, each of which targets specific bias categories. This makes bias detection not an afterthought but a fundamental architectural constraint.

Color-Team Adversarial Bias Testing

The Red team specifically includes bias exploitation in its adversarial scenarios. Red team agents probe platform systems for confirmation bias, selection bias, and source dependency vulnerabilities. Blue team agents maintain defensive postures against bias-inducing attacks. Purple team synthesis ensures that detected biases are actually resolved, not just acknowledged.

OSINT Intelligence Fusion

With 120 OSINT tools drawing from diverse sources (Czech registries, global intelligence providers, sanctions databases), the platform must detect and mitigate biases introduced by source selection, geographic coverage gaps, and data freshness differences. The Source Independence axiom directly addresses this.

Security Rating Fairness

The Prismatic Perimeter's security rating system (A-F grades, 300-900 scores) requires rigorous bias testing to ensure that ratings do not systematically disadvantage organizations based on size, geography, industry, or technology stack. Statistical fairness metrics are applied to rating distributions.

AI Drift Detection

The AI Drift MVP milestone specifically targets bias drift in AI-powered decision systems. Over time, models can drift toward biased behavior as the underlying data distribution changes. The platform monitors for this drift and triggers alerts when fairness metrics deteriorate.

Comparison with Alternatives

ApproachStrengthsWeaknessesBest For
Structural Axiom EnforcementPrevents bias architecturally, hard to circumventRequires framework integration, complex designPlatforms with epistemic requirements
Statistical Fairness TestingQuantitative, well-understood metrics, actionableRequires protected group definitions, post-hoc onlyScoring and classification systems
Adversarial Red Team TestingDiscovers hidden biases, creative attack surfacesLabor-intensive, not exhaustiveSecurity-critical systems
Manual Bias AuditsDeep contextual understanding, holistic viewExpensive, subjective, infrequentRegulatory compliance
Automated Bias ScanningScalable, consistent, continuousLimited to known bias patterns, may miss novel biasesHigh-volume data pipelines
Counterfactual AnalysisDirectly tests causation, easy to interpretRequires system accessibility, combinatorial explosionIndividual decision verification

Best Practices

  1. Build bias detection into the architecture, not as an afterthought. The NABLA axioms demonstrate that structural bias prevention is more effective than post-hoc detection. Design systems that make biased outputs structurally difficult.

  2. Require multiple independent signals before establishing beliefs. The Signal Plurality axiom prevents the most common bias pattern: drawing conclusions from a single source or data point.

  3. Preserve contradictions explicitly. When evidence points in conflicting directions, the natural human tendency is to resolve the contradiction by discarding one side. Resist this. Contradictions are data.

  4. Test for bias with adversarial methods. Do not rely solely on statistical metrics. Use Red team techniques to probe for biases that standard tests miss.

  5. Monitor for bias drift over time. A system that is unbiased at launch can become biased as data distributions shift. Implement continuous monitoring of fairness metrics.

  6. Make bias findings actionable. Each detected bias should come with a remediation recommendation, severity assessment, and clear ownership for resolution.

  7. Document bias risk assessments. For every algorithm, scoring system, or classification pipeline, document the bias risks, mitigation strategies, and testing results.

  8. Include diverse perspectives in bias review. Automated tools catch known bias patterns. Human reviewers with diverse backgrounds catch biases that the tools were not designed to detect.

Common Pitfalls

  1. Treating bias as a binary property. Bias exists on a spectrum. A system is not simply "biased" or "unbiased" -- it has measurable fairness properties that can be improved.

  2. Optimizing for one fairness metric while ignoring others. Statistical fairness criteria can be mutually exclusive. Achieving demographic parity may violate equalized odds. Understand the trade-offs.

  3. Ignoring the bias in your bias detection. Bias detection tools themselves can be biased. If the protected groups are incorrectly defined, or the fairness metric is inappropriate for the context, the tool will report false confidence.

  4. Confusing correlation with bias. Not all group differences in outcomes indicate bias. Some differences reflect genuine underlying variation. The key question is whether the difference is caused by the protected attribute or by legitimate factors.

  5. Cherry-picking favorable bias test results. Running many statistical tests and reporting only the ones that show no bias is itself a form of confirmation bias. Report all results, including unfavorable ones.

  6. Neglecting temporal bias. Data from different time periods may encode different biases. Historical data often reflects historical discrimination. Fresh data may introduce recency bias.

  7. Assuming automated detection is sufficient. Automated tools catch quantifiable biases in structured data. They miss qualitative biases in framing, context selection, and narrative construction.

  8. Over-correcting for detected bias. Aggressive bias correction can introduce new distortions. Balance fairness improvements against accuracy and utility.

Use Cases

Intelligence Analysis Quality Assurance

OSINT intelligence analysts must guard against confirmation bias when investigating entities. The platform's NABLA-enforced analysis pipeline requires that conflicting intelligence signals are preserved and presented alongside supporting evidence.

Security Rating Fairness Verification

Before publishing security ratings for organizations, the platform runs statistical fairness tests across industry, size, and geographic dimensions to ensure that the rating methodology does not systematically disadvantage any category.

AI Model Monitoring

Machine learning models deployed in the platform undergo continuous bias monitoring. When fairness metrics drift beyond acceptable thresholds, alerts are generated and the model is flagged for retraining or decommissioning.

Due Diligence Investigations

Financial due diligence and KYC investigations must be free from bias related to nationality, ethnicity, or political affiliation. The platform's bias detection ensures that screening algorithms apply consistent standards across all subjects.

Adversarial Red Team Exercises

The Red team conducts specific bias exploitation scenarios where adversaries attempt to induce biased behavior in platform systems through carefully crafted inputs. These exercises reveal bias vulnerabilities that static analysis misses.

Compliance with AI Regulations

Emerging AI regulations (EU AI Act, algorithmic accountability requirements) mandate bias testing for high-risk AI systems. The platform's bias detection infrastructure provides the evidence base for regulatory compliance.

  • NABLA Infinity -- the epistemic framework whose seven axioms provide structural bias prevention
  • Signal Plurality -- the axiom requiring multiple independent signals, directly preventing confirmation and anchoring biases
  • Contradiction Preservation -- the axiom mandating that conflicting evidence be preserved, preventing cherry-picking
  • Epistemic Reasoning -- the broader discipline of reasoning under uncertainty that bias detection supports
  • Adversarial Testing -- the practice of probing systems for weaknesses, including hidden biases
  • Red Team -- the adversarial simulation team that conducts bias exploitation testing
  • Cherry Picking -- a specific bias pattern where only favorable evidence is selected
  • Confidence Scoring -- the mechanism that quantifies belief strength, which must itself be unbiased
  • Evidence -- the foundational data from which bias-free conclusions must be drawn
  • Quality Assurance -- the broader quality discipline that includes bias detection as a critical component

See Also


Connect & Contribute: Created by Tomas Korcak (korczis) | GitHub | GitLab | LinkedIn | Contact | Licensed under GHL