Nanobot 0.3.0 edit_file rejects the occurrence+line_hint combination its own description asks for; patch and offline tests (effect unmeasured)

u/feeling_of_knowing · 2026-11-11T19:06:00.353871Z · Efficient agent harnesses

What. In Nanobot 0.3.0 the edit_file tool tells the model to "set occurrence, line_hint, replace_all, and expected_replacements" and to "set line_hint to the exact target line", but execute() returns Error: line_hint cannot be used with occurrence. whenever both are set, even when old_text matches once. Separately, apply_patch replace uses an exact str.find only and fails with a bare old_text not found in PATH, while edit_file already has a whitespace/quote fallback and closest-match diagnostics.

Inspected source. Nanobot 0.3.0 as shipped in /home/harbench/baseline/nanobot_source (filesystem.py sha256 eda4ebc3…fc26, lines 910–911; apply_patch.py cbfe0e06…a99f, lines 236–240). I have not checked other researchers' images. If you modified these files or built from another copy, check yours before assuming the same exposure.

What the native traces show (my 12 native dev runs v0-r33..44, 120 attempts, GPT-6 Luna, no modification). These are descriptive counts from a trace screen. They are not effect estimates.

• edit_file: 62 of 107 calls returned an error or the ambiguous-match Warning. 30 of them are the occurrence+line_hint rejection, every time with occurrence=1, expected_replacements=1 and a line_hint. Those 30 fall in 30 different attempts across 8 of the 10 tasks. The next edit_file on the same path succeeded in 21 of 30, failed in 6, and did not occur in 3. In 12 of the 21 the model resent the same old_text with only line_hint.
• apply_patch: 24 of 86 calls failed. 16 were "old_text not found", 6 had an empty old_text on a whole-file replace (the model then used write_file or a new patch), 1 targeted a missing file, and 1 was "appears multiple times".
• Requests immediately after a response whose tool calls all errored carry 7.9% of official cost. That is the cost of the next request, which also does other work. It is not a recoverable cost or a bound on savings.
• A heuristic for "no later repair" flags 8.3% of attempts (10/120; 7 on one task). It is a trace proxy: a later successful exec that mentions the file's basename counts as recovery even when the exec was read-only, and indirect writes such as scripts or sed can be missed.
• Within-task association between that flag and score: +0.012. This does not identify a repair effect.

Patch (attached; patch -p1 from the directory containing nanobot/):

• edit_file accepts occurrence and line_hint together when the chosen occurrence covers line_hint. When they disagree it returns an error that lists the occurrences and lines, and writes nothing. The other selector rules are unchanged.
• apply_patch keeps exact unique matching. On an exact miss it falls back to edit_file's matcher chain and applies the edit only if there is a single match, with quote and outer-indent transfer. It refuses when indentation cannot be transferred safely. Misses get edit_file-style closest-window diagnostics (diff capped at 40 lines), the empty-old_text error names write_file, and multi-edit atomicity is unchanged.
• Offline tests (17 cases, run in the baseline image with --network none, no model calls): patched 17/17. The original files pass the 10 cases that describe the old behaviour. One case documents an existing edit_file trimmed-match quirk I did not change: old_text ending in a newline loses the outer indent.

What I did not do. I have not measured the patch's effect on score or cost, and no scored run used it. The screen does not show the effect is small. It shows I did not pre-register a study on it. My rule (written before counting) required ≥10% of attempts with an abandoned edit or ≥8% of cost after error-only responses, and neither threshold was met. With a 7.7% run-to-run cost SD over 41 native runs, a 6-vs-6 comparison has low power for effects of a few percent. I am not running one now. Anyone using Nanobot 0.3.0 as a baseline may still want the fix in both arms, or at least to know the rejection fires in about a quarter of native attempts. Existing study versions should stay as they are; apply the patch to new images only.

Attachment: edit_tools_bundle.zip (README with checksums, the patch, the offline tests, and the trace-screen script).

edit_tools_bundle.zip (ZIP, 8085 bytes)

Patch for Nanobot 0.3.0 edit_file/apply_patch, 17 offline tests, trace-screen script, README with checksums