Prismatic HAWKEYE

Prismatic HAWKEYE

423 files Intelligence

Abstract

HAWKEYE is the Prismatic Platform's Visitor Intelligence system, providing real-time behavioral analysis, device fingerprinting, geolocation intelligence, and dynamic risk scoring for every visitor interacting with monitored web properties. The system correlates browser-side behavioral signals (mouse movement, scroll patterns, click cadence, navigation flow) with server-side intelligence from external threat feeds -- AbuseIPDB, GreyNoise, and Shodan -- to classify visitors on a 0-100 risk scale across five threat levels. HAWKEYE's architecture employs a GenStage-based processing pipeline that handles visitor profiling in under 100 milliseconds per request, with bot detection accuracy exceeding 98%. The system delivers its intelligence through a real-time Phoenix LiveView dashboard and programmatic APIs, enabling both human analysts and automated systems to act on visitor threat assessments.

1. Introduction

1.1 Problem Statement

Web-facing applications receive traffic from a spectrum of actors ranging from legitimate users to automated scanners, credential stuffers, web scrapers, and targeted attack tools. Traditional web analytics tools measure page views and session durations but lack the security intelligence needed to distinguish benign visitors from threats. Conversely, Web Application Firewalls (WAFs) operate on static rule sets that miss behavioral anomalies and context-dependent threats. The gap between analytics and security creates blind spots where sophisticated automated attacks succeed precisely because they mimic human behavior.

HAWKEYE bridges this gap by combining behavioral analysis with external threat intelligence, producing a contextual risk assessment that adapts in real time as visitor behavior evolves throughout a session.

1.2 Design Goals

  1. Sub-100ms profiling latency -- visitor risk assessment must complete within the HTTP request lifecycle to enable inline blocking decisions.
  2. Multi-signal correlation -- combine browser fingerprints, geolocation, behavioral patterns, and external threat feeds into a unified risk score.
  3. Adaptive bot detection -- detect automated behavior through behavioral analysis rather than static signatures, achieving over 98% accuracy.
  4. Real-time dashboard -- provide a LiveView interface showing live visitor feeds, geographic distribution, and threat indicators.
  5. Privacy compliance -- minimize data collection to what is necessary for security assessment, with configurable retention policies.
  6. Integration with platform intelligence -- share threat assessments with Prismatic Perimeter and the broader OSINT pipeline.

1.3 Scope

HAWKEYE monitors web traffic to Prismatic-managed properties. It does not function as a general-purpose web analytics platform, an intrusion detection system for network-level attacks, or a DDoS mitigation service. Its scope is visitor-level intelligence and behavioral risk assessment.

2. Architecture

2.1 System Design

Browser                                           Server
+------------------+                    +---------------------------+
| JS Collector     |  WebSocket/HTTP    | Collector GenServer       |
| - Mouse events   | ----------------> | - Event ingestion         |
| - Scroll events  |                    | - Session tracking        |
| - Click events   |                    +---------------------------+
| - Navigation     |                              |
| - Fingerprints   |                    +---------+---------+
+------------------+                    |                   |
                                  Fingerprinter       Geolocator
                                        |                   |
                                  +-----+-----+    +-------+-------+
                                  | Device ID  |    | IP Geoloc     |
                                  | Canvas     |    | ASN / ISP     |
                                  | WebGL      |    | VPN / Tor     |
                                  | Fonts      |    | Proxy detect  |
                                  +-----+------+    +-------+-------+
                                        |                   |
                                  +-----+-------------------+-----+
                                  |    Enrichment Pipeline         |
                                  | (Parallel external queries)    |
                                  | AbuseIPDB | GreyNoise | Shodan |
                                  +-----+-------------------------+
                                        |
                                  +-----+------+
                                  | Behavioral  |
                                  | Engine      |
                                  | - Mouse     |
                                  | - Scroll    |
                                  | - Click     |
                                  | - Bot Det.  |
                                  +-----+------+
                                        |
                                  +-----+------+
                                  | Risk Engine |
                                  | Score 0-100 |
                                  | Classify    |
                                  +-----+------+
                                        |
                                  +-----+------+
                                  | Dashboard   |
                                  | PubSub push |
                                  +------------+

2.2 Core Components

ModuleResponsibility
PrismaticVisitorIntelligence.CollectorWebSocket/HTTP event ingestion from browser-side JavaScript collector
PrismaticVisitorIntelligence.FingerprinterDevice fingerprinting from canvas, WebGL, font, and browser attributes
PrismaticVisitorIntelligence.GeolocatorIP geolocation, ASN resolution, VPN/Tor/proxy detection
PrismaticVisitorIntelligence.EnrichmentParallel external intelligence queries (AbuseIPDB, GreyNoise, Shodan)
PrismaticVisitorIntelligence.BehavioralEngineBehavioral pattern analysis: mouse, scroll, click, navigation
PrismaticVisitorIntelligence.BotDetectorAutomated behavior detection with ML-assisted classification
PrismaticVisitorIntelligence.RiskEngineDynamic 0-100 risk scoring and five-level classification
PrismaticVisitorIntelligence.SessionTrackerCross-request session management with visitor identity correlation
PrismaticVisitorIntelligence.DashboardLiveView components for real-time monitoring

2.3 Process Topology

PrismaticVisitorIntelligence.Application (Supervisor, :one_for_one)
+-- PrismaticVisitorIntelligence.Collector (GenServer)
|     Ingests browser events, manages session state in ETS
+-- PrismaticVisitorIntelligence.SessionTracker (GenServer)
|     Cross-session visitor identification via fingerprint matching
+-- PrismaticVisitorIntelligence.RiskEngine (GenServer)
|     Maintains risk computation state and classification thresholds
+-- PrismaticVisitorIntelligence.Enrichment.Supervisor (Task.Supervisor)
|     Supervises parallel external intelligence queries
+-- PrismaticVisitorIntelligence.AlertManager (GenServer)
      Manages alert generation and routing for high-risk visitors

2.4 Data Flow

Visitor data enters through the browser-side JavaScript collector, which transmits behavioral events over a WebSocket connection. The Collector GenServer ingests these events and routes them through a sequential pipeline: fingerprinting, geolocation, external enrichment (parallel), behavioral analysis, and risk scoring. The final risk assessment is stored in ETS and broadcast via PubSub to the LiveView dashboard.

3. Implementation

3.1 Key Algorithms

Behavioral Bot Detection. The bot detector analyzes four behavioral dimensions: mouse movement (acceleration, curvature, jitter), scroll behavior (speed variability, pause distribution), click patterns (inter-click intervals, target accuracy), and navigation flow (page visit order, dwell time distribution). Each dimension produces a 0-1 human-likeness score. Scores below 0.3 on any dimension trigger automated classification. Combined scores below 0.5 across all dimensions yield high-confidence bot detection.

Risk Score Computation. The risk engine aggregates signals from fingerprinting (known bad fingerprints), geolocation (high-risk countries, VPN/Tor usage), external intelligence (AbuseIPDB confidence score, GreyNoise classification), and behavioral analysis (bot likelihood). Each signal contributes a weighted sub-score; the final score is clamped to 0-100 and mapped to classification levels: Low (0-25), Medium (26-50), High (51-75), Critical (76-100).

3.2 Data Structures

defmodule PrismaticVisitorIntelligence.VisitorProfile do
  @type t :: %__MODULE__{
    session_id: String.t(),
    visitor_id: String.t(),
    fingerprint: Fingerprint.t(),
    geolocation: Geolocation.t(),
    behavior: BehaviorAnalysis.t(),
    risk_score: 0..100,
    risk_level: :low | :medium | :high | :critical,
    classification: :legitimate_user | :suspicious | :automated_scanner | :known_threat,
    threat_indicators: [ThreatIndicator.t()],
    first_seen: DateTime.t(),
    last_seen: DateTime.t()
  }
end

3.3 API Surface

# Profile a visitor session
@spec profile(String.t()) :: {:ok, VisitorProfile.t()} | {:error, term()}
PrismaticVisitorIntelligence.profile(session_id)

# Assess threat level for a request
@spec assess_threat(Plug.Conn.t()) :: {:ok, ThreatAssessment.t()} | {:error, term()}
PrismaticVisitorIntelligence.assess_threat(request)

# Analyze behavioral patterns
@spec analyze_behavior(String.t()) :: {:ok, BehaviorAnalysis.t()} | {:error, term()}
PrismaticVisitorIntelligence.analyze_behavior(session_id)
# => {:ok, %BehaviorAnalysis{is_bot: false, bot_confidence: 0.12, behavioral_score: 92}}

# Detect anomalies across visitor population
@spec detect_anomalies(keyword()) :: {:ok, [Anomaly.t()]} | {:error, term()}
PrismaticVisitorIntelligence.detect_anomalies(timeframe: :last_hour, threshold: 2.0)

# Track visitor across sessions
@spec track_visitor(Plug.Conn.t()) :: {:ok, Session.t()} | {:error, term()}
PrismaticVisitorIntelligence.track_visitor(request)

3.4 Configuration

config :prismatic_visitor_intelligence,
  # Risk scoring
  risk_weights: %{
    external_intelligence: 0.35,
    behavioral_analysis: 0.30,
    geolocation: 0.20,
    fingerprint: 0.15
  },
  classification_thresholds: %{low: 25, medium: 50, high: 75},

  # External sources
  enrichment_sources: [:abuseipdb, :greynoise, :shodan],
  enrichment_timeout: 5_000,

  # Behavioral analysis
  bot_detection_threshold: 0.5,
  behavioral_window: :timer.minutes(5),

  # Data retention
  session_retention: :timer.hours(24),
  visitor_retention: :timer.days(30)

4. Integration

4.1 Dependencies

ApplicationRelationship
Prismatic WebDashboard hosting and LiveView components
Prismatic OSINT CoreAbuseIPDB, GreyNoise, Shodan intelligence feeds
Prismatic CacheIntelligence query caching (reduces API costs)
Prismatic TelemetryVisitor metrics and performance monitoring
Prismatic StorageVisitor data persistence
Prismatic AuthDashboard access control

4.2 Dependents

ApplicationRelationship
Prismatic PerimeterThreat intelligence sharing
Prismatic SignalsVisitor threat signals feed
Prismatic Detection EngineBehavioral anomaly findings

4.3 Inter-Process Communication

The dashboard receives updates via Phoenix PubSub on the "hawkeye:visitor_update" topic. External intelligence queries are dispatched as supervised tasks to avoid blocking the profiling pipeline. Risk assessments are cached in ETS for sub-microsecond retrieval on subsequent requests from the same visitor.

4.4 External Integrations

Three real-time threat intelligence feeds: AbuseIPDB (IP reputation scoring), GreyNoise (mass scanner identification), and Shodan (infrastructure fingerprinting). All queries pass through Prismatic Resilience circuit breakers and Prismatic Cache.

5. Performance

5.1 Benchmarks

OperationLatencyNotes
Fingerprint computation< 5msServer-side hash of browser attributes
Geolocation lookup< 2msLocal MaxMind database
External enrichment (parallel)50-200msAbuseIPDB + GreyNoise + Shodan in parallel
Behavioral analysis< 10msPattern matching on session events
Total profiling pipeline< 100msEnd-to-end per request
Bot detection accuracy98.5%Validated against labeled test set

5.2 Scalability

Each visitor session is independently tracked in ETS, allowing horizontal scaling through load-balanced instances. External intelligence queries are cached for 1-24 hours depending on source, reducing API load. Behavioral analysis is stateless per session window, requiring no cross-instance coordination.

5.3 Resource Requirements

ResourceMinimumRecommended
Memory512 MB2 GB (with session cache)
CPU2 cores4 cores (behavioral analysis)
Storage500 MB5 GB (with visitor history)

6. Testing Strategy

6.1 Unit Tests

Each behavioral analyzer (mouse, scroll, click, navigation) has unit tests with known-human and known-bot behavioral samples. Risk engine tests verify score computation from predetermined signal combinations. Fingerprint tests verify consistent hashing across browser attribute variations.

6.2 Integration Tests

Full pipeline tests exercise the visitor profiling flow from HTTP request through enrichment and risk scoring. Tests use recorded visitor sessions (both human and automated) to verify end-to-end classification accuracy.

6.3 Property-Based Testing

StreamData generators produce random behavioral event sequences to verify that the risk engine always produces scores within the 0-100 range and that classification levels are consistent with score thresholds.

7. Security Considerations

7.1 Threat Model

Adversaries may attempt to evade detection by mimicking human behavioral patterns or spoofing fingerprints. Mitigations include multi-dimensional behavioral analysis (difficult to fake all dimensions simultaneously), external intelligence correlation (IP reputation is independent of browser behavior), and adaptive thresholds that adjust based on population-wide anomalies.

7.2 Access Control

Dashboard access requires dashboard_access and hawkeye_read permissions through Prismatic Auth. Visitor data is subject to configurable retention policies. No personally identifiable information beyond IP addresses is collected, and IPs are hashed after the retention period.

8. Operational Considerations

8.1 Deployment

HAWKEYE deploys as part of the main Prismatic Web application on port 4000. The browser-side JavaScript collector is served as a static asset. No additional infrastructure is required beyond the OSINT API credentials for enrichment sources.

8.2 Monitoring

Telemetry events cover visitor profiling latency ([:prismatic, :hawkeye, :profile]), bot detection rates ([:prismatic, :hawkeye, :bot_detected]), and enrichment source health ([:prismatic, :hawkeye, :enrichment]). The dashboard itself serves as the primary monitoring interface.

8.3 Troubleshooting

SymptomLikely CauseResolution
High false positive rateBot detection threshold too aggressiveAdjust bot_detection_threshold upward
Slow profilingExternal enrichment timeoutsCheck OSINT source health; verify circuit breakers
Missing risk scoresEnrichment sources all failingVerify API keys; check rate limits
Dashboard not updatingPubSub disconnectionCheck WebSocket; restart LiveView mount

9. Future Work

Planned enhancements include machine learning model integration for behavioral classification (replacing rule-based analysis), session replay for forensic investigation of high-risk visitors, integration with WAF systems for automated blocking, cross-property visitor correlation, and support for mobile application behavioral analysis.

References

  • Real-Time Monitoring -- Foundation for HAWKEYE's sub-100ms visitor profiling and live dashboard updates
  • Intelligence Synthesis -- Multi-source intelligence fusion combining behavioral signals with external threat feeds
  • Telemetry Integration -- Comprehensive event coverage for visitor profiling latency and bot detection metrics

Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL