/pvm-trace

PVM L2+ Production low usage

/pvm-trace

Overview

/pvm-trace is a production command in the PVM category of the Prismatic Platform that provides real-time execution tracing and debugging capabilities for PVM (Prismatic Virtual Machine) programs. The command attaches to a running PVM program or replays a recorded execution, providing instruction-level visibility into program behavior including register state, stack frames, heap allocations, I/O operations, and control flow decisions.

Tracing is essential for understanding the runtime behavior of compiled PVM programs, particularly when debugging failures, diagnosing performance bottlenecks, or verifying that a program behaves according to its specification. The tracer operates at the bytecode instruction level but maps each instruction back to its source IR line through the debug information embedded by the /pvm-compile command, enabling developers to reason about program behavior in terms of the original high-level description.

This command operates under the L2+ authority level and is executed by the pvm-executor agent. It is part of the platform's 216-command slash command registry, built on the AIAD (Autonomous Intelligence Agent Design) standard. The tracer is designed to have minimal performance impact on the traced program, using asynchronous event collection and buffered output to avoid introducing timing artifacts that would alter program behavior.

The command supports multiple tracing modes: full trace (every instruction), conditional trace (only when predicates match), breakpoint-based (pause at specific locations), and statistical (sampling-based performance profiling). These modes can be combined to focus tracing on specific areas of interest while maintaining manageable output volumes for programs that execute millions of instructions.

Architecture

The tracing system sits alongside the PVM runtime, observing execution without modifying it.

+-------------------+
| PVM.Runtime       |
| (executing)       |
+-------------------+
      |
      | trace events
      v
+-------------------+
| Trace Collector   |
| (async buffer)    |
+-------------------+
      |
      v
+-------------------+     +-------------------+
| Trace Processor   |---->| Source Mapper      |
| (filter/format)   |     | (IR line mapping)  |
+-------------------+     +-------------------+
      |
      v
+-------------------+
| Trace Output      |
| (terminal/file)   |
+-------------------+
ComponentResponsibility
Trace CollectorAsynchronously collects trace events from the runtime
Trace ProcessorFilters events based on predicates and formats output
Source MapperMaps bytecode offsets to source IR line numbers
Trace OutputRenders trace data to terminal, file, or structured format
Breakpoint ManagerManages breakpoint locations and pause/resume state
Statistics AggregatorCollects instruction frequency and timing statistics

Usage

Live Tracing

# Trace a running PVM program by PID
/pvm-trace --pid <runtime_pid>

# Start a program with tracing enabled
/pvm-trace --program workflow.pvmb --param domain=example.com

# Trace with source IR mapping
/pvm-trace --program workflow.pvmb --source workflow.ir

Breakpoint Debugging

# Set breakpoint at source line
/pvm-trace --program workflow.pvmb --break-at line:42

# Set breakpoint at bytecode offset
/pvm-trace --program workflow.pvmb --break-at offset:0x0020

# Set conditional breakpoint
/pvm-trace --program workflow.pvmb --break-when "r0 == :error"

# Break on specific instruction type
/pvm-trace --program workflow.pvmb --break-on CALL

Filtered Tracing

# Trace only I/O operations
/pvm-trace --program workflow.pvmb --filter io

# Trace only control flow (branches, calls, returns)
/pvm-trace --program workflow.pvmb --filter control-flow

# Trace specific register changes
/pvm-trace --program workflow.pvmb --watch r0,r1,r5

# Exclude verbose instructions
/pvm-trace --program workflow.pvmb --exclude "LOAD_CONST,NOP"

Performance Profiling

# Statistical profiling with 1% sampling
/pvm-trace --program workflow.pvmb --profile --sample-rate 0.01

# Instruction frequency analysis
/pvm-trace --program workflow.pvmb --profile --frequency

# Hot path identification
/pvm-trace --program workflow.pvmb --profile --hot-paths

# Memory allocation tracking
/pvm-trace --program workflow.pvmb --profile --memory

Replay Tracing

# Trace a recorded replay
/pvm-trace --replay /tmp/replay.log

# Step through replay
/pvm-trace --replay /tmp/replay.log --step

# Replay with search for specific state
/pvm-trace --replay /tmp/replay.log --search "r3 contains :timeout"

Options & Parameters

OptionTypeDefaultDescription
--programstringnilPath to PVM bytecode to execute with tracing
--pidstringnilPID of running PVM runtime to attach
--replaystringnilPath to replay log for offline tracing
--sourcestringnilPath to source IR for line mapping
--paramkey=value[]Input parameters when executing with tracing
--break-atstringnilBreakpoint location (line:N or offset:0xN)
--break-whenstringnilConditional breakpoint predicate
--break-onstringnilBreak on specific instruction type
--filterstringallEvent filter: all, io, control-flow, memory
--watchstringnilComma-separated register names to watch
--excludestringnilInstruction types to exclude from trace
--profilebooleanfalseEnable statistical profiling mode
--sample-ratefloat1.0Sampling rate for profiling (0.0-1.0)
--outputstringstdoutTrace output destination
--formattext | json | flamegraphtextOutput format
--stepbooleanfalseStep-by-step execution mode
--verbosebooleanfalseInclude register dumps at each instruction

Execution Flow

  1. Attachment -- The tracer attaches to the target PVM runtime, either by starting a new program with tracing hooks or by connecting to an existing runtime process via its PID.

  2. Instrumentation -- Trace hooks are installed in the instruction dispatcher. These hooks fire asynchronously for each executed instruction, pushing events to the trace collector without blocking the main execution loop.

  3. Event Collection -- The trace collector buffers events in batches, periodically flushing to the trace processor. Buffer size is tuned to balance latency against overhead.

  4. Filtering -- The trace processor applies the configured filters, discarding events that do not match the specified criteria. For conditional breakpoints, predicate evaluation occurs at this stage.

  5. Source Mapping -- Surviving events are enriched with source IR line mappings using the debug information from the bytecode. If --source is provided, the actual source lines are included in the output.

  6. Breakpoint Handling -- When a breakpoint is hit, the runtime is paused and control is transferred to the interactive debugger. The operator can inspect registers, evaluate expressions, and continue or step execution.

  7. Output Rendering -- Processed events are rendered to the configured output destination in the specified format. Text format provides human-readable output; JSON enables tool integration; flamegraph format produces profiling visualizations.

Integration Points

SystemIntegrationPurpose
/pvm-compileDebug info enables source mappingSource correlation
/pvm-executeTraces programs started by executorRuntime attachment
TelemetryTrace statistics as telemetry eventsObservability
Quality GatesTrace analysis for performance validationPerformance gates
Formal VerificationExecution traces as verification evidenceProof artifacts
NABLAProvenance tracking via execution tracesEpistemics

Best Practices

  1. Use filters in production -- Full tracing generates enormous output volumes. Always use --filter to limit tracing to relevant operations when diagnosing production issues.

  2. Prefer sampling for profiling -- For performance analysis, use --profile --sample-rate 0.01 rather than full tracing. Sampling provides statistically valid profiling data with minimal overhead.

  3. Source mapping for debugging -- Always provide --source when debugging logic errors. Bytecode-level traces are difficult to interpret without IR line correlation.

  4. Breakpoints over full trace -- When searching for a specific condition, use --break-when instead of tracing everything and searching the output. Breakpoints are far more efficient.

  5. Save traces for regression -- When debugging a resolved issue, save the trace and add it to the test suite as a regression artifact.

Error Handling

ErrorCauseResolution
{:error, :process_not_found}Target PID does not exist or is not a PVM runtimeVerify PID and that the program is running
{:error, :no_debug_info}Bytecode compiled without debug infoRecompile with --debug-info (default)
{:error, :source_mismatch}Source IR does not match compiled bytecodeUse the source file that matches the compiled version
{:error, :trace_buffer_overflow}Trace events generated faster than processedUse --filter to reduce event volume or increase buffer
{:error, :invalid_breakpoint}Breakpoint location outside program rangeVerify line number or offset against source/bytecode

Advanced Usage

Flamegraph Generation

# Generate flamegraph from profiling data
/pvm-trace --program workflow.pvmb --profile --format flamegraph --output /tmp/flame.svg

# View in browser
open /tmp/flame.svg

Remote Tracing

# Attach to PVM runtime on remote node
PVM.Trace.attach({:pvm_runtime, :"prismatic@prod-node1"},
  filter: :io,
  output: {:file, "/tmp/remote_trace.log"}
)

Differential Tracing

# Compare execution traces of two program versions
/pvm-trace --program v1.pvmb --output /tmp/trace_v1.json --format json
/pvm-trace --program v2.pvmb --output /tmp/trace_v2.json --format json
diff <(jq '.events[]' /tmp/trace_v1.json) <(jq '.events[]' /tmp/trace_v2.json)

Automated Trace Analysis

# Search trace for error patterns
/pvm-trace --replay /tmp/replay.log --search "instruction == BRANCH_ERR" --format json | \
  jq '.events[] | select(.registers.r0 == "timeout")'

Doctrine Compliance

All commands operate under the NO MERCY, NO DOUBTS doctrine:

  • NO MERCY: Zero tolerance for incomplete execution or quality violations. The tracer captures all events within its configured scope -- no events are silently dropped. Buffer overflow is reported as an error, not handled by silent discard.
  • NO DOUBTS: Full investigation before action, evidence-based results. Execution traces provide complete, verifiable evidence of program behavior. Every traced instruction includes its full context (registers, stack depth, heap usage) for unambiguous interpretation.

The tracing system directly supports the NABLA Provenance Mandatory axiom by providing an auditable chain of execution steps that trace any output value back to its input sources and computation path.


Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL