Make nightly lesson-consolidation deterministic + monitored (de-risk /daily)
kind: infra
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
The failure-lesson consolidation pipeline is the SOLE cross-issue path that
dedupes epm:failure-lesson markers, promotes recurring lessons into
.claude/rules/gotchas.md, and prunes over-eager generalizes: yes
agent-memory entries. Today it lives ONLY inside the nightly /daily LLM
skill (claude -p /daily, crontab 23:27 PT). Two fragilities:
- It rides a flaky 44K-token LLM run. If the nightly headless
/dailyerrors or half-completes, that day's consolidation silently does not happen and nothing alerts. Every other cron job in this repo is pure Python for exactly this reason. - No health check. A silently-failed
/dailyreads as "quiet day" — the PM digest line omits silently when yesterday's file is absent.
Goal
Make the load-bearing, DETERMINISTIC parts of nightly consolidation independent of the LLM run, and add a heartbeat so a failed nightly run is visible.
Scope / surfaces
- NEW
scripts/consolidate_lessons.py(pure Python): scan the rolling-windowepm:failure-lessonmarkers across all tasks, dedupe against existing.claude/agent-memory/<agent>/entries, promote recurring /gotcha_candidate: yeslessons into.claude/rules/gotchas.md(or route a workflow-fix candidate), prune over-eagergeneralizes: yesentries. Deterministic, idempotent, fails loud, logs counts. The rolling window (default 7d) also subsumes the cross-day pattern-folding/weeklydid. - NEW
scripts/cron_lesson_consolidate.sh+ crontab registration (mirror the othercron_*.sh). - NEW
scripts/cron_daily_healthcheck.sh+ crontab line (~01:00 PT): iflogs/daily/<yesterday>.mdis missing or >25h old, Telegram-push an alert via the my-goattelegram_push.shchannel. - The LLM
/dailykeeps only its judgment parts (transcript problem-mining, held-item triage);.claude/skills/daily/SKILL.mdupdated to note the deterministic consolidation now runs inconsolidate_lessons.pyand/dailyno longer owns it. .claude/workflow.yaml:2049text updated to point the consolidation responsibility at the deterministic script, not/daily.
Constraints / invariants
- Workflow-surface only. No experiment code / configs / tasks.
- The inline
/issue-time routing (generalizes: yes→ agent-memory write,gotcha_candidate: yes→ workflow-fix candidate) is UNCHANGED — it already fires per-lesson and is robust. - Re-promotion must be idempotent (do not re-file a gotcha already present / a workflow-fix already open).
scripts/workflow_lint.py+ ruff on touched files pass.
Acceptance
- A simulated failed nightly
/daily(nologs/daily/<date>.mdwritten) triggers the heartbeat alert. consolidate_lessons.pyrun over a fixture day produces the same gotcha / memory writes the/dailyconsolidation step would have, deterministically, and is a no-op on a second run.
Activity