/pack-sources

Framework L2+ Production medium usage

/pack-sources

Overview

/pack-sources is a production command in the Framework category of the Prismatic Platform. It creates optimized, context-rich source archives specifically designed for consumption by AI models and Large Language Models (LLMs). The command intelligently selects, filters, and packages source code, configuration files, documentation, and metadata into compressed archives that maximize the useful context an LLM can extract while staying within token limits. Optional mycelial evolution integration enables the archives to include pattern propagation data, evolution history, and quality metrics alongside raw source code.

This command operates under the L2+ authority level and is executed by the source-archive-specialist agent. It is part of the platform's 216-command slash command registry, built on the AIAD (Autonomous Intelligence Agent Design) standard. The command recognizes that modern development workflows increasingly involve AI-assisted analysis, and that the quality of AI outputs depends heavily on the quality and organization of the input context.

In a platform with 90+ applications, 6,600+ Elixir source files, and millions of lines of code, blindly concatenating source files produces archives that exceed LLM context windows and dilute relevant information with noise. /pack-sources solves this through intelligent selection heuristics, dependency-aware ordering, and configurable filtering that produces archives tuned for specific analysis tasks.

Architecture

The source packing system operates as a pipeline that transforms a raw codebase into an optimized archive.

Pipeline Architecture

             /pack-sources
                   |
           Source Scanner
                   |
          +--------+--------+
          |        |        |
       Filter   Dependency  Metadata
       Engine   Resolver    Enricher
          |        |        |
          +--------+--------+
                   |
           Context Optimizer
                   |
          +--------+--------+
          |        |        |
       Token     Priority   Format
       Budget    Ranker     Converter
          |        |        |
          +--------+--------+
                   |
           Archive Builder
                   |
           Output (tar.gz/zip)

Core Components

ComponentResponsibilityKey Feature
Source ScannerDiscovers all files in scopeGit-tree based, 100x faster than filesystem scan
Filter EngineRemoves irrelevant files (build artifacts, deps, etc.)Configurable exclusion rules
Dependency ResolverOrders files by dependency graphEnsures LLM reads definitions before usage
Metadata EnricherAdds context annotations to source filesModule descriptions, test coverage, quality scores
Context OptimizerFits content within token budgetIntelligent truncation and summarization
Priority RankerRanks files by relevance to analysis goalRecently modified, high-complexity, or flagged files first
Format ConverterOutputs in LLM-friendly formatsMarkdown, concatenated text, or structured JSON
Archive BuilderPackages final outputCompressed archives with manifest

Usage

# Pack all sources with defaults
/pack-sources

# Pack specific application
/pack-sources --app prismatic_web

# Pack with token budget for specific LLM
/pack-sources --token-budget 128000 --model claude

# Pack only recently modified files
/pack-sources --since 7d

# Pack with mycelial evolution data included
/pack-sources --include-mycelial

# Pack specific file types only
/pack-sources --types ex,exs,md

# Pack with full metadata enrichment
/pack-sources --enrich full

# Output as single concatenated markdown file
/pack-sources --format markdown --output ./context.md

# Pack for specific analysis goal
/pack-sources --goal "security audit" --app prismatic_api

# Exclude test files
/pack-sources --exclude-tests

Practical Examples

# Prepare context for LLM-assisted security review
/pack-sources --app prismatic_perimeter --goal "security audit" --enrich full --format markdown

# Pack quality-related code for evolution analysis
/pack-sources --domain quality --include-mycelial --token-budget 200000

# Create minimal context for a quick code review
/pack-sources --app prismatic_web --since 3d --format markdown --token-budget 50000

# Pack entire platform with maximum compression for archival
/pack-sources --all --format tar.gz --compress max --output ./archives/

Options

OptionTypeDefaultDescription
--appstringallSpecific application to pack
--appsstringallComma-separated list of applications
--domainstringallPack files related to specific quality domain
--typesstringex,exsComma-separated file extensions to include
--sincedurationallInclude only files modified within period
--token-budgetinteger200000Maximum token count for output
--modelenumclaudeTarget LLM model for token estimation
--formatenumtar.gzOutput format: tar.gz, zip, markdown, json, text
--outputpath./packed-sources/Output directory or file path
--include-mycelialflagfalseInclude mycelial evolution metadata
--enrichenumbasicMetadata enrichment: none, basic, full
--goalstringnoneAnalysis goal hint for priority ranking
--exclude-testsflagfalseExclude test files from archive
--exclude-docsflagfalseExclude documentation files
--compressenumstandardCompression level: none, standard, max
--manifestflagtrueInclude file manifest with metadata
--allflagfalsePack entire platform (override app filter)
--verboseflagfalseShow detailed packing progress

Execution Flow

Phase 1: Scope Definition

The command determines which files fall within scope based on the provided filters. It uses the Git tree index (via git ls-tree) for rapid file enumeration, applying extension filters, recency filters, and application boundaries.

Phase 2: Dependency Analysis

Files within scope are analyzed for dependency relationships. In Elixir, this means parsing alias, import, use, and require directives to build a dependency graph. Files are then topologically sorted so that definitions appear before their usages in the final archive.

Phase 3: Priority Ranking

Each file receives a priority score based on multiple factors:

FactorWeightDescription
Recency0.25Recently modified files score higher
Complexity0.20Higher cyclomatic complexity scores higher
Centrality0.20Files with many dependents score higher
Goal Relevance0.20Files matching --goal keywords score higher
Quality Flags0.15Files with quality issues score higher for audit goals

Phase 4: Token Budget Allocation

The total token budget is allocated across files based on priority scores. High-priority files receive their full content allocation, while lower-priority files may be truncated or summarized. The token estimator uses model-specific tokenization rules to ensure accurate budget management.

Phase 5: Metadata Enrichment

When enrichment is enabled, each file is annotated with:

  • Module documentation (from @moduledoc)
  • Function typespecs and documentation
  • Test coverage percentage
  • Quality score and known issues
  • Mycelial propagation history (if --include-mycelial)
  • Last modification timestamp and author

Phase 6: Archive Assembly

The final archive is assembled with files in dependency-sorted, priority-ranked order. A manifest file is included at the top listing all files with their metadata, token counts, and priority scores.

Integration Points

ComponentIntegration TypeDescription
/seadfFrameworkSEADF provides source scanning infrastructure
/analyzePeerArchitecture analysis enriches metadata
/scan-myceliumData SourceMycelial patterns included in enriched archives
Git TreesInfrastructureFast file enumeration from git index
Quality GatesData SourceQuality scores used for metadata enrichment
TelemetryMonitoringPacking statistics and timing metrics
GARDENData SourceLegacy patterns available for inclusion

Best Practices

Token Budget Management

Different LLMs have different context window sizes. Set --token-budget to approximately 80% of the target model's context window to leave room for the analysis prompt and model response. For Claude Opus with a 200K context window, a budget of 150,000-160,000 tokens is appropriate.

Goal-Directed Packing

Always specify --goal when the analysis has a specific focus. The priority ranker uses goal keywords to boost relevant files. "Security audit" boosts authentication, authorization, and input validation code. "Performance optimization" boosts hot paths and database queries.

Incremental Context Building

For iterative analysis sessions, start with a small --since window and expand as needed. This prevents overwhelming the LLM with irrelevant context while ensuring recent changes are prominently featured.

Mycelial Integration

Enable --include-mycelial when the analysis involves understanding pattern propagation, evolution history, or quality improvement trends. The mycelial data adds approximately 10-15% to archive size but provides valuable context about how the codebase has evolved.

Error Handling

Error CodeDescriptionResolution
TOKEN_BUDGET_EXCEEDEDContent exceeds budget even after truncationNarrow scope with --app or --since
GIT_TREE_UNAVAILABLEGit index not accessibleEnsure working directory is a git repository
APP_NOT_FOUNDSpecified application not found in umbrellaCheck application name spelling
FORMAT_UNSUPPORTEDRequested output format not availableUse supported format: tar.gz, zip, markdown, json, text
DEPENDENCY_CYCLECircular dependency detected in file graphFiles included in arbitrary order for cycle members
ENRICHMENT_FAILUREMetadata enrichment failed for some filesFiles included without enrichment; check compilation

Advanced Usage

Custom Priority Functions

Override the default priority ranking with a custom function:

/pack-sources --priority-fn "fn file -> if String.contains?(file, 'controller'), do: 1.0, else: 0.5 end"

Diff-Based Packing

Pack only the changes between two git refs:

/pack-sources --diff main..feature-branch --format markdown

Multi-Model Output

Generate archives optimized for multiple LLMs simultaneously:

/pack-sources --app prismatic_api --models claude,gpt4,gemini --output ./multi-model/

Pipeline Integration

Use pack-sources as part of an automated analysis pipeline:

/pack-sources --app prismatic_web --format json --output - | /analyze --input-format packed-json

Doctrine Compliance

All commands operate under the NO MERCY, NO DOUBTS doctrine:

  • NO MERCY: Zero tolerance for incomplete execution or quality violations. Archives include quality metadata so analysis can identify violations.
  • NO DOUBTS: Full investigation before action, evidence-based results. Manifests provide complete provenance for every included file.
  • /seadf - Self-Evolving Autonomous Development Framework control and monitoring
  • /analyze - System architecture analysis with dependency mapping
  • /scan-mycelium - Mycelial pattern scanning across documentation and code
  • /rc1-orchestrate - Complete RC1 delivery pipeline execution with ROC optimization
  • /inject - AIAD injection coordination for pattern and agent deployment
  • /migrate - Safe migration planning with rollback strategies
  • /integrate - Cross-system integration design and implementation

Connect & Contribute

Created by Tomáš Korcak (korczis) | Open Source under GHL