Public Evidence & Claims Registry
Preregistered claims,
measured outcomes.
droid-tune registers decision rules and hypotheses before running sweeps. Every claim-eligible trial writes a hash-manifested evidence pack published in full. Below is the public claim registry, measured outcomes, provenance, and reproduction commands.
dt-v1: Ledger-Lite NO_SUBMISSION Reduction Sweep
A preregistered negative result published in full with its statistics — the claims protocol working as designed, not a failure to hide.
Source files: claims/dt-v1-ledger-lite-nosub.json and docs/dt-v1-ledger-lite-nosub-results-2026-08-20.md.
Question: On task t004-git-surgery, does the ledger-lite tune reduce the NO_SUBMISSION rate relative to an otherwise identical no-tune arm across four free BYOK routes?
Frozen Design: 80 total trials (10 per arm per route), task t004-git-surgery, 4 OpenCode Zen free routes, $0 cost, arms alternating within route.
Decision Conditions (Frozen Rule)
The preregistered decision rule required ALL three conditions to recommend the tune. Two conditions failed.
| # | Condition (Frozen in Claim) | Measured Result | Met? |
|---|---|---|---|
| 1 | Pooled NO_SUBMISSION rate ≥ 25pp below control |
5.0pp drop (ledger-lite 80.0% vs no-tune 85.0%) |
NO |
| 2 | Two-sided Fisher exact p < 0.05 on pooled 2×2 |
p = 0.7695 on [[34, 6], [32, 8]] |
NO |
| 3 | Pooled VERIFIED_PASS rate not lower in tune arm |
ledger-lite 20.0% vs no-tune 15.0% |
YES |
Per-Route Rates
Published as registered, exclusions included across all four OpenCode Zen free routes.
| Route | no-tune NO_SUBMISSION | ledger-lite NO_SUBMISSION | no-tune VERIFIED_PASS | ledger-lite VERIFIED_PASS |
|---|---|---|---|---|
hy3-free |
8/10 (80.0%) | 10/10 (100.0%) | 2/10 | 0/10 |
nemotron-3.5-lightning-free |
10/10 (100.0%) | 5/10 (50.0%) | 0/10 | 5/10 |
laguna-s-2.1-free |
8/10 (80.0%) | 8/10 (80.0%) | 2/10 | 2/10 |
nemotron-3-ultra-free |
8/10 (80.0%) | 9/10 (90.0%) | 2/10 | 1/10 |
| Pooled (40 scorable / arm) | 34/40 (85.0%) | 32/40 (80.0%) | 6/40 (15.0%) | 8/40 (20.0%) |
Exclusions & Secondary Metrics
Exclusions: Zero. 0 PROVIDER_ERROR, DROID_ERROR, or VERIFIER_ERROR occurred in any of the 80 trials, so the exclusion rule never fired, no replacement was queued (0/5 cap untouched), and no route was dropped.
| Metric | no-tune | ledger-lite |
|---|---|---|
| Output tokens / trial (mean · median) | 3299 · 2823 | 3565 · 2758 |
| Cycle time ms / trial (mean · median) | 68289 · 65925 | 79036 · 62468 |
| Audit claim-without-coverage (80/80 auditable) | 17 | 18 |
| Audit no-test-finish | 0 | 0 |
Provenance & Reproduction
Reproduce Analysis Command
node scripts/claim-report.js --claim dt-v1-ledger-lite-nosubdt-v0: Cache-Stability Tune
Registered 2026-08-19 and reported 2026-09-03 after all 50 confirmatory slots completed. The closed claim records the frozen design, conclusion, and all 50 deterministic evidence IDs.
| Pooled metric | native-droid | cache-stable-droid | Result |
|---|---|---|---|
| First-pass yield | 21/25 (84.0%) | 22/25 (88.0%) | NOT LOWER |
| Observed Factory Standard Credits / verified task | 11,387.3 | 9,010.9 | 20.9% LOWER |
| Cache-read share | 92.3% | 97.0% | +4.7pp |
| Median cycle time | 12,190 ms | 9,616 ms | −2,574 ms |
Recorded decision: the claim's supported verdict means both frozen descriptive conditions were met. The yield difference is one pass; the rule has no uncertainty threshold or minimum effect size. This does not establish statistical non-inferiority or general improvement.
Evidence gap: these figures come from the committed conclusion, not independent reproduction. Raw packs, the sweep log and the frozen snapshot are referenced under ignored runs/ paths and are not included in a fresh clone. Evidence IDs alone are insufficient.
Boundary: Factory credits are observed billing units, not an inferred USD amount. There are 25 matched task/repetition pairs, 50 trials total. Publication of the underlying evidence still requires privacy review and explicit approval.
Requires the complete matching evidence bundle
node scripts/claim-report.js --claim dt-v0-cache-stability --runs-dir <evidence-root>M4 Live Flake Check (2026-08-19)
Full technical analysis published in docs/m4-flake-check-2026-08.md.
An observed outcome of 29 / 40 VERIFIED_PASS trials was recorded on 2026-08-19 across 5 tasks and 4 OpenCode Zen free routes at $0 cost.
Counting Basis Note: The headline 29 / 40 counts every attempted trial. Among trials that actually reached the model, the observed rate was 29 / 39 (74%), because 1 trial resulted in a PROVIDER_ERROR and never reached the model.
t004-git-surgery Discriminative Finding: Across the 8 attempts on t004, 7 / 8 ended in NO_SUBMISSION. Agents performed the required technical edits in the worktree but never ran git commit. Exactly 1 nemotron-3.5-lightning-free attempt executed the commit and achieved VERIFIED_PASS.