Skip to content
MCP Filesystem Ultra

Bug #10: Batch Edit Token Overhead

Status: RESOLVED in v3.14.0 Category: Token Optimization / Performance Severity: High (Cost and latency impact) Resolution Date: 2026-02-13

Problem

Section titled “Problem”

When Claude Desktop needed to edit multiple files in a workflow (e.g., refactor a function name across a codebase), each step required a separate MCP round-trip:

1. smart_search(pattern="oldName")          → 1 call
2. read_file_range(file1, lines...)          → 1 call
3. read_file_range(file2, lines...)          → 1 call
4. edit_file(file1, old, new)                → 1 call
5. edit_file(file2, old, new)                → 1 call
6. count_occurrences(file1, "newName")       → 1 call
7. count_occurrences(file2, "newName")       → 1 call
                                    Total:     7 calls

For larger refactors touching 19+ files, this exploded to 20-40 sequential MCP calls.

Root Causes

Section titled “Root Causes”
  1. No batch workflow mechanism: Each operation (search, read, edit, verify) was an independent MCP call
  2. Token overhead per call: Each round-trip adds ~300-600 tokens of serialization, schema, and reasoning overhead
  3. No data flow between operations: Results from search couldn’t feed directly into edit — Claude had to manually extract file paths and pass them
  4. Redundant file reads: The same file was read multiple times across different operations

Impact

Section titled “Impact”
WorkflowFilesMCP CallsToken Overhead
Simple rename (2 files)25-7~2,100 tokens
Namespace refactor (10 files)1022-25~9,000 tokens
Pattern migration (19 files)1940+~18,000 tokens

The overhead was purely structural — the actual useful work (search, edit, verify) was identical regardless of how many calls it took.

Solution: Pipeline Transformation System

Section titled “Solution: Pipeline Transformation System”

Introduced execute_pipeline — a single MCP tool that chains multiple operations with automatic data flow between steps.

Before (7 calls, ~2,100 token overhead)

Section titled “Before (7 calls, ~2,100 token overhead)”
smart_search(pattern="oldFunc")
read_file_range(file1, ...)
read_file_range(file2, ...)
edit_file(file1, "oldFunc", "newFunc")
edit_file(file2, "oldFunc", "newFunc")
count_occurrences(file1, "newFunc")
count_occurrences(file2, "newFunc")

After (1 call, ~400 token overhead)

Section titled “After (1 call, ~400 token overhead)”
{
  "name": "rename-function",
  "create_backup": true,
  "steps": [
    {
      "id": "find",
      "action": "search",
      "params": { "path": "src/", "pattern": "oldFunc" }
    },
    {
      "id": "edit",
      "action": "edit",
      "input_from": "find",
      "params": { "old_text": "oldFunc", "new_text": "newFunc" }
    },
    {
      "id": "verify",
      "action": "count_occurrences",
      "input_from": "find",
      "params": { "pattern": "newFunc" }
    }
  ]
}

Key Design Decisions

Section titled “Key Design Decisions”
  1. input_from data flow: Each step can reference a previous step’s results — file lists pass automatically without Claude extracting and re-passing them
  2. Server-side execution: All steps execute in the MCP server process with zero network round-trips between them
  3. In-memory passing: File contents and match results stay in memory between steps — no redundant disk reads
  4. Built-in safety: Automatic backup before destructive steps, risk assessment, and rollback on failure

Real-World Benchmark

Section titled “Real-World Benchmark”

Measured on the mcp-filesystem-go-ultra codebase (core/ directory):

Search + Read + Count (2 files)

Section titled “Search + Read + Count (2 files)”
MethodMCP CallsServer TimeToken Overhead
Individual calls5~5ms~2,100
Pipeline13.4ms~400
Savings5×~30%~5×

Search + Count across 19 files (dry-run)

Section titled “Search + Count across 19 files (dry-run)”
MethodMCP CallsToken Overhead
Individual calls~22~9,000
Pipeline1~400
Savings22×~22×

Verbose vs Compact

Section titled “Verbose vs Compact”

The pipeline supports two output modes:

  • Compact (default): OK: 3/3 steps | 2 files | 0 edits — minimal tokens, ideal for edit workflows
  • Verbose (verbose: true): Full file contents, per-file counts, complete file lists — ideal when Claude needs to inspect results

Verification

Section titled “Verification”
=== RUN   TestPipeline_Validation         --- PASS (9 sub-tests)
=== RUN   TestPipeline_SearchAndCount      --- PASS
=== RUN   TestPipeline_DryRun              --- PASS
=== RUN   TestPipeline_StopOnError         --- PASS
=== RUN   TestPipeline_DependencyChain     --- PASS
=== RUN   TestPipeline_RiskAssessment      --- PASS
=== RUN   TestPipeline_BackupAndRollback   --- PASS
=== RUN   TestPipeline_MultiEdit           --- PASS
=== RUN   TestPipeline_Copy                --- PASS
PASS  ok  0.316s

8/8 tests passing. No regressions in existing test suites.

Lessons Learned

Section titled “Lessons Learned”
  1. Round-trip overhead dominates: Server execution time is nearly identical — the cost is in serialization, context, and reasoning tokens per call
  2. Data flow eliminates redundancy: input_from removes the need for Claude to manually extract and pass file lists between operations
  3. Two output modes are essential: Compact for edits (save tokens), verbose for inspection (Claude needs the data)
  4. Test assertions must use real keys: Path normalization (NormalizePath + filepath.Walk) can produce keys different from filepath.Join — tests should use actual pipeline output keys
Section titled “Related”