Toolrun Compression

Devlish can save tokens in two ways: compiling stable workflows so the LLM stops orchestrating every step, and compressing tool output while workflows are still being discovered. The devlish-toolrun crate implements the second path.

What Toolrun Does

devlish-toolrun is a Rust command wrapper that runs ordinary shell commands, stores raw output locally, and returns compact, typed JSON to the LLM.

LLM asks for command
  -> devlish-toolrun exec -- <command>
  -> command runs normally
  -> raw stdout/stderr saved to .devlish/toolruns/
  -> model receives compact structured report
  -> raw_ref can be requested only when needed

The key design rule: raw output is preserved by reference. Compression reduces default context, it does not erase evidence.

Usage

devlish-toolrun exec -- bundle exec rspec
devlish-toolrun exec -- npm test
devlish-toolrun exec -- git status --short --branch

Each command runs normally. The raw output is saved to .devlish/toolruns/ with a timestamp. The LLM receives a compact summary instead of the full output.

Command-Specific Adapters

Toolrun includes tailored summaries for high-frequency development commands:

Command	Adapter	Summary
bundle exec rspec	RSpec	Pass/fail counts, failed examples, runtime
npm test / node --test	Node test	Test results, failures, coverage
git status	Git	Branch, staged/unstaged counts, untracked files
rg / grep	Search	Match count, file list, sample matches
(other)	Generic	Bounded output sample with line counts

The Token-Saving Ladder

Toolrun fits into a progression from raw agent loops to compiled workflows:

raw tool calls (full output in context)
  -> compressed tool calls (toolrun summaries)
  -> discovered repeatable flow (agent patterns mined from runs)
  -> reviewed Devlish source (.dvl program)
  -> compiled bytecode/WASM run (zero orchestration tokens)

At each step, the model sees less while the program does more deterministically.

How It Works

1. The LLM invokes a command through toolrun instead of a raw shell
2. Toolrun executes the command and captures stdout/stderr
3. An adapter parses the output into structured fields (counts, paths, errors)
4. The compact JSON summary goes to the LLM context
5. If the LLM needs full details, it requests the raw output by reference

Example Output

Running devlish-toolrun exec -- bundle exec rspec might return:

{
  "command": "bundle exec rspec",
  "exit_code": 1,
  "summary": {
    "examples": 142,
    "failures": 2,
    "pending": 3,
    "runtime_seconds": 4.2
  },
  "failures": [
    "spec/models/invoice_spec.rb:45 - expected 1200, got nil",
    "spec/services/validator_spec.rb:12 - undefined method 'validate'"
  ],
  "raw_ref": ".devlish/toolruns/2026-07-18T10-30-00-rspec.txt"
}

Instead of hundreds of lines of RSpec output, the LLM sees a focused summary with enough detail to act on failures.

Relationship to the Compiler

Toolrun does not depend on the Devlish compiler or VM. The boundaries are separate:

• Compiler (devlish_core): Devlish source to bytecode
• VM (devlish_vm): Deterministic bytecode execution
• Toolrun (devlish_toolrun): Host command execution and output compression

All three share the same Rust workspace but operate independently. After enough toolrun captures, repeated patterns can be promoted into reviewed .dvl workflows.