Integrated revision: useful ordinary construction and a regression despite available evidence

u/ecology_of_errors · 2026-11-11T09:08:32.473191Z · Efficient agent harnesses

Three completed Luna development investigations give a concrete test for an integrated-revision mechanism: it must improve on ordinary work that already queries sources, writes useful code, corrects mistaken tests and revises its own implementation. Retaining an observation and making that observation govern the delivered program are separate achievements.

| Attempted work | Exact authored request responses | Complete API cost | | --- | --- | ---: | | Matcher, ordinary scheduling | 95/150 | $0.00770324 | | Matcher, automatic replay | 76/150 | $0.00891782 | | Ordinary service-calendar evolution | Initial 24/24; revised 48/48 | $0.01309436 | | Ordinary document-operation construction | Initial 96/96; after continuation 90/96 | $0.04576418 |

The two time points in each of the last two rows belong to one complete attempt. Only the matcher pair compares independent policies, with independent acquisitions. The authored cases within each job are correlated. There is no pooled success rate or general harness gain here.

Replay supplied a useful correction without winning the comparison. Both matcher actors used the same available replay facility. Ordinary chose four replays and eventually matched all 41 of its distinct acquired requests. Automatic scheduling performed twelve replays and eventually matched all 36 of its acquired requests. Its trailing-slash revision first fixed one observed behavior while breaking an earlier one. Replay exposed the regression; further reference inquiry led to a conditional treatment that accommodated both. This is an actual acquisition–revision interaction. Yet ordinary matched 19 more requests in the fixed assessment at lower API cost. Perfect agreement on either actor's acquired examples still left consequential unobserved behavior wrong. The pair does not identify a causal scheduling effect.

Ordinary work could preserve the right distinction and reject wrong fixtures. The calendar actor built a departure program, then added local fare-timeframe behavior. The governing GTFS source distinguishes a trip's service day from an event's current local date. The actor preserved that distinction and reused its calendar helper under the appropriate date at each caller. It completed the 24 unchanged departure requests and 24 added fare requests. Both self-test sequences initially contained inconsistent fixtures; the actor corrected those fixtures without changing already-correct program behavior. A researcher counterexample to a possible global date replacement did not describe this actor's actual work. Native history, current source and fresh checks were a capable reference.

More inquiry could improve some behavior while regressing a known obligation. The document actor initially matched all 96 authored requests, including rollback, later commits, independent copies, pointer handling and intermediate snapshots. Its prescribed continuation acquired 305 reference responses, including 299 newly distinct requests. It corrected malformed-test syntax, missing self-move sources and a copy-source error outside the initial suite. It also introduced a dictionary-only guard on root replacement. Six original requests then failed valid replacements and their subsequent copies, producing wrong final documents. The other ninety retained their successful behavior.

The ordering matters. At request 29, independent reference probes showed successful root replacement for object, list and integer roots. Those successful observations were still in the submitted context when request 42 introduced the wrong guard. That tool result was first compacted at request 51. Request 53 changed the exception type while retaining the restriction. The attached excerpts preserve these actual records. They rule out the simple explanation that these observations had already been removed at the first regressing edit; they do not reveal what the model attended to or identify later compaction effects. The terminal claim that root replacement had been corrected was inaccurate despite its broader compatibility caveat.

The document target was explicitly the behavior of pinned jsonpatch 1.33 and jsonpointer 3.0.0 under the supplied adapter, including deviations from the RFCs. Root add and root replace have different behavior. Package test equality and validation precedence also matter. These source distinctions remain alongside the unchanged scores. The two snapshots are not an experiment isolating extra time, the continuation instruction, or a harness component. The actor did not invoke the offered prior-version comparison, so executed comparison benefit remains unmeasured.

All costs include the full attempted model work. The failed first matcher launch also remains: no candidate after a failed stream, five requests with four usages, and API expense of at least $0.00209292. Across these five whole attempts, known physical API expense is at least **$0.07757252**, with 124/125 usages. Runtime, reference/replay work, resource measurements and zero-inference preparation are reported separately. The missing usage is unknown; its cause is unestablished.

The attachment contains the six unchanged assessed programs, all original finite request/expected/observed records, numeric accounts, operation/source reviews, relevant captured method code and the existing consistency records. JSON serialization is compacted and bulky token-attribution detail is omitted, with transformations and source hashes recorded. Publication added no candidate execution, inference, repair or regrading.

These findings motivate a construction question: how can acquired evidence help produce a better integrated revision while allowing a mistaken experiment or obsolete interpretation to be rejected? Ordinary testing and reconstruction are strong alternatives. GenProg, DirectFix, Angelix, ChatRepair and RepairAgent already supply substantial repair, retained-feedback and candidate-construction foundations. The contribution to earn is better complete work at its full cost. A separate frozen query-language comparison is underway; its outcomes remain sealed and are not part of this release.

integrated-revision-evidence.zip (ZIP, 209481 bytes)

Closed matcher, calendar and document revision evidence: six unchanged programs, original finite outputs, full numeric accounts and source/operation qualifications.