Migrate clean-result body to the 2-content-section spec (TL;DR + Reproducibility) — #432 overhaul [weekly-W22 #1]
Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Source: 2026-W22 weekly proposal #1. Thomas dictated a wholesale clean-result body overhaul against task #432: "no separate Details section; parameters into Reproducibility; Methodology corrections not necessary; just # Human TLDR (empty) + # TLDR (LessWrong style, starts with Motivation, per-result subheadlines with one plot each, one raw-completion example per artifact + dropdown + links to all raw); Next steps not by default; remove 'why confidence is where it is'; training/eval data as a 5-example dropdown + link in the relevant section; every clean result except Motivation stands alone; never use 'byte identical'."
This must ship as ONE reviewed+tested change: the canonical verifier
scripts/verify_task_body.py enforces the CURRENT 4-section model, so changing
the docs without the verifier (or vice versa) breaks the promotion pipeline.
The full implementation plan (check-by-check verifier changes, doc edits, test rewrites, migration analysis, execution order) follows.
Plan: Migrate clean-result body to the 2-section spec (TL;DR + Reproducibility)
0. Executive summary of what I found (decision-critical)
- The verifier is a forward-only gate. It runs ONLY at
/issueStep 5 (analyzer pre-publish) and clean-result-critic pre-pass. It is NEVER run retroactively over the ~95 existinghas_clean_result: truebodies. I confirmed those legacy bodies already FAIL the current verifier (they carry retired shapes:## AI TL;DR,## Source issues,## Headline numbers, legacy-sagan-card HTML). So no migration / grandfathering work is needed — changing the verifier cannot break already-promoted bodies. This is the load-bearing finding that de-risks the whole change. - Task #432 is the target END state, not a currently-passing body. I ran it through the current verifier: it FAILs 5/19 checks (missing
## Details, noMotivation/What I ran/Resultsbullet labels, etc.). #432 already embodies most of what Thomas dictated (no Details, params in Reproducibility, findings as####subheadlines with one plot each, no "why confidence", no Next steps). The new verifier must make #432-shaped bodies PASS. - #432 still has two stale pieces the new spec must also fix: (a) the old
## Human TL;DRstub uses Headline/Takeaways/How-this-updates-me — the dictation says Human TL;DR is empty; (b) the new spec wants TL;DR to start with a### Motivation(or**Motivation:**) labeled block then findings. - "byte identical" appears in ~10 legacy bodies as a data-equivalence phrase. Thomas wants it banned going forward -> a new
audit_clean_results_body_discipline.pyPATTERN, not retroactive edits.
1. The exact new required body shape
---
<frontmatter: title, kind, goal, has_clean_result, ...>
---
# <one-sentence claim> (LOW|MODERATE|HIGH confidence)
## Human TL;DR
placeholder <- single stub token; Thomas fills in (UNCHANGED behavior)
## TL;DR
### Motivation <- REQUIRED, labeled literally "Motivation", comes FIRST
<LessWrong-style narrative: why this matters, prior tasks linked>
### <result-1 subheadline naming the finding>
<setup prose>
 <- exactly one plot per result
> **Figure.** *italic lead.* caption...
<read prose>
<one cherry-picked raw-completion example + dropdown of more + link to all raw>
### <result-2 subheadline ...>
<same shape, one plot>
...
## Reproducibility
**Parameters:** <table — ABSORBS the old Details parameters table>
**Artifacts:** <incl. training/eval data dropdown-of-5 + full-link, placed in relevant section>
**Compute:** ...
**Code:** ...
Rules distilled from the dictation:
- Two H2 sections only:
## Human TL;DR,## TL;DR,## Reproducibility. (Three H2s total; "two-section" = the two content sections beyond the human stub. Call it the 2-content-section model.) - No
## Details. Everything that was in Details (narrative, figures, samples, "why this test") moves UP into## TL;DRunder per-result###subheadlines. - No
## FigureH2 (already deprecated). - Parameters table ->
## Reproducibility(under**Parameters:**, as #432 already does). - No "Methodology corrections" section by default. If a correction is load-bearing it folds into the relevant result's prose — no dedicated H3.
- No "Next steps" by default. Keep the hard exception: raw-completions-not-uploaded still requires the re-run note, but it lives in the relevant result's prose now.
- No "Why confidence is where it is" section. The
Confidence: LEVEL — rationalesentence moves to## Reproducibility(or the end of the last TL;DR result) — verifier still cross-checks it against the title. - TL;DR starts with
### Motivationlabeled as Motivation. - One plot per result; one cherry-picked raw-completion example per artifact, a dropdown (
<details>) of more, and a link to all raw completions. - Training/eval data: a
<details>dropdown of 5 examples + a full-data link, placed in the relevant result section (#432 already does this — keep). - Every clean result except Motivation must stand alone and never use "byte identical" (or "byte-identical").
2. verify_task_body.py — exact check-by-check changes
Reference current check list (CHECKS, 17 body checks + Goal soft check = 19 results). By current name/number:
| # | Check (current) | Action | Detail |
|---|---|---|---|
| 0 | check_body_nonstub | KEEP | unchanged |
| 0b | check_no_duplicate_frontmatter | KEEP | unchanged |
| 1 | check_title_confidence | KEEP | unchanged |
| 2 | check_required_sections | CHANGE | REQUIRED_H2_SECTIONS = ["Human TL;DR", "TL;DR", "Reproducibility"] (drop "Details"). Order check follows. Remove the ## Figure-between-TL;DR-and-Details positional clause. |
| 3 | check_tldr_labels | CHANGE | New required label set. Drop What I ran, Results as required bullets. New TLDR_BULLETS_REQUIRED = ["Motivation"]. Motivation may be ### Motivation H3 or **Motivation:** bullet — relax the regex to accept an ### Motivation heading too. |
| 4 | check_figure_image | CHANGE | Image must exist inline under ## TL;DR. Keep >=1 image required. |
| 4b | check_figure_url_resolvable | KEEP (retarget) | Now scans images under ## TL;DR only. |
| 5 | check_figure_caption | KEEP/SIMPLIFY | Vacuously passes when ## Figure absent (always now). Keep minimal. |
| 6 | check_confidence_matches | CHANGE | Currently looks for Confidence: line inside ## Details. Retarget to whole-body (or Reproducibility + TL;DR). Keep title-match + >=20-char rationale. |
| 7 | check_repro_subgroups | KEEP | unchanged. |
| 8 | check_repro_url_permanence | KEEP | unchanged. |
| 9 | check_repro_sentinel_scrub | KEEP | unchanged. |
| 10 | check_cherry_picked_label | CHANGE | Retarget sample-fence scan from ## Details to ## TL;DR. |
| 11 | check_qualitative_data_link | CHANGE | Same retarget to ## TL;DR. |
| 11b | check_planned_vs_actual_denominator | CHANGE | Retarget section scan from Details to whole-body so it still fires if a corrections paragraph is kept. Document that the H3 is no longer required. |
| 12 | check_figure_h2_is_deprecated | KEEP | Still WARNs on legacy ## Figure. |
| 13 | check_details_narrative_flow | CHANGE | Retarget figure-dump + bad-H3-label heuristics to scan ## TL;DR. |
| 14 | check_mdx_safe_urls | KEEP | unchanged. |
| 15 | check_repro_committed_claims_exist | KEEP | NEW as of weekly-W22 #2; unchanged here. |
| — | check_goal_present | KEEP | unchanged (soft WARN). |
Helper changes: section_text(body, "Details") callers (6, 10, 11, 11b, 13) retarget to "TL;DR"/whole-body; _gather_figure_image_urls drop "Figure"; _iter_sample_fences/_prelude_window operate on TL;DR. New constants REQUIRED_H2_SECTIONS, TLDR_BULLETS_REQUIRED. Rewrite module docstring ("Three required H2; Details retired 2026-W22"). Net check count unchanged (retarget, not add/remove) — but note the W22 #2 committed-at-sha check already bumped CHECKS to 18 / results to 20; preserve those counts.
3. Doc edits — SPEC.md, analyzer.md, clean-result-critic.md, codex twin, CLAUDE.md
3a. .claude/skills/clean-results/SPEC.md (source of truth — edit FIRST)
Replace the 4-section enumeration with the 3-H2 / 2-content-section model. Delete the ## Details bullet + sub-rules. Rewrite ## TL;DR to: start with ### Motivation; one ### <finding> per result with one inline figure + one cherry-picked raw example + <details> dropdown + link to all raw. Move Parameters table + Confidence sentence into ## Reproducibility. Rewrite the mechanical-checks list. Rename "Story arc for Details" -> "Story arc for TL;DR results". Add byte_identical to the anti-pattern list. Add a migration note (forward-grandfathered). Make #432 the canonical exemplar.
3b. .claude/agents/analyzer.md
Rewrite "four required H2s" -> three everywhere (~lines 41, 214, 244, 258, 368). Step 6 pre/post-flight greps: change ^## Details$ -> ^## TL;DR$ + ^## Reproducibility$. Delete the "Methodology corrections as last H3 in Details" rule. Confirm Human TL;DR stub stays the bare placeholder. Rewrite TL;DR shape (Motivation first; per-result H3 with one figure + raw example + dropdown + all-raw link; drop What-I-ran/Results required bullets). Move Confidence to Reproducibility. Retitle "Story arc for Details" -> TL;DR result beats. Update anti-patterns #11/#13/#14; add "byte identical" ban.
3c. .claude/agents/clean-result-critic.md
Update mechanical pre-pass section names. Fold Lens 4 (Details narrative) rules into Lens 2 (TL;DR); drop the Confidence-after-Parameters sub-rule. Lens 8: drop the Methodology-corrections-placement half, keep the title check, rename "Mentor-facing title". Lens 12: retarget ## Details -> ## TL;DR result beats. Lens 13: drop the ### Methodology corrections H3 sub-checks; scope-shrinkage documented in result prose. Add a "byte identical" ban (Lens 6 voice). Keep lens numbers stable, retitle/mark-merged to minimize cross-file churn.
3d. .claude/agents/codex-clean-result-critic.md
Mirror every clean-result-critic change (per-lens summaries at ~156, ~172-178, ~199-256, ~273-287). Keep it a thin mirror.
3e. CLAUDE.md
§ Experiment Report Structure: "four required H2 ... / TL;DR / Details / Reproducibility" -> three; point at SPEC.md. § After Every Experiment item 8: missing-condition documentation now in result prose, not ### Methodology corrections.
3f. .claude/workflow.yaml
Update the clean-result-critic marker fields description (lens set + 2-content-section shape) and the stale verification: "...six mechanical checks" line.
4. Test updates under tests/
tests/test_verify_task_body.py: rewrite GOOD_BODY to the new shape (drop ## Goal/## Figure/## Details; TL;DR starts ### Motivation then a ### <finding> with inline image + caption + cherry-picked sample + qualitative-data link + <details>; Parameters + Confidence in Reproducibility). Fix all dependents: section-order/missing-section tests, test_missing_tldr_labels (target Motivation), cherry-picked + qualitative-data injection points (-> TL;DR), confidence tests (locate in Reproducibility), test_goal_* (GOOD_BODY no longer carries ## Goal — inject a copy). Add new tests: ### Motivation form passes; a #432-shaped body passes end-to-end; pin the stray-## Details behavior (recommend FAIL). New audit test: byte_identical fires. Also audit tests/test_verify_clean_result.py, test_verify_clean_result_body_stdin.py, test_clean_result_critic_planned_vs_actual.py (retarget Methodology-in-Details fixtures), test_task_cli_set_body_assertions.py.
5. Migration — none needed
Verifier is forward-only; the ~95 legacy bodies already FAIL it and are never re-verified, so tightening cannot regress them. Keep the legacy-sagan-card short-circuit. Leave "byte identical" in legacy bodies (forward audit-pattern only). One consequence to flag: in-flight awaiting_promotion tasks (54) with a 4-section draft will FAIL on next analyzer/critic re-run and simply get re-drafted under the new spec — acceptable (#432, #382, #237 are examples).
6. Recommended execution path
Ship as one reviewed+tested infra-kind /issue change on a worktree. Edit order (SPEC.md first): SPEC.md -> verify_task_body.py + helpers -> audit script (byte_identical) -> tests -> analyzer.md -> clean-result-critic.md -> codex twin -> CLAUDE.md -> workflow.yaml. Gate on green tests + workflow_lint.py --check-asks + ruff. End-to-end smoke: a #432-shaped body PASSes, a legacy 4-section body FAILs cleanly. Keep lens numbers stable. Code-review ensemble verifies docs<->verifier<->tests consistency.
Decision for Thomas before execution: should a stray/leftover ## Details H2 in a NEW body be a hard FAIL (forces clean migration) or an allowed extra H2? Recommend FAIL.
Target exemplar (END state to make PASS): tasks/awaiting_promotion/432/body.md (with Human TL;DR -> bare placeholder and an explicit ### Motivation label; it already uses ### Motivation).