daily-fix: rubric-bearing JudgeCache._hash_key
kind: infra#wf-fix#wf-fix-fp:fe8ef634bf02#daily-auto-filed
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Auto-filed by the /daily 2026-07-02 backfill problem sweep (route 2 — behavior/logic change, independent review required). Emitting context: transcript mining of the 2026-07-02 sessions.
Goal
Add rubric/judge-prompt hash + judge model id to JudgeCache._hash_key so cached judgments cannot leak across rubrics
Workflow gap
- Bug observed: JudgeCache._hash_key keys on (question, completion) only; refusal-rubric judgments leaked into harmful_compliance E0 scores on #810 (64 percent of high scores shared exact reasoning/score pairs with the refusal file)
- Why it is a workflow gap: llm-judging.md rule 22 (#882) documented the keying requirement but explicitly deferred the batch_judge.py code fix as a separate follow-up that was never filed; the per-rubric cache_dir is only an accepted surrogate
- Confidence (emitter): high
Proposed change (candidate diff sketch — refine in planning)
- def _hash_key(question: str, completion: str) -> str:
- content = f"{question}\n---\n{completion}"
+ def _hash_key(question, completion, rubric_key) -> str:
+ # rubric_key = sha256 of the full judge prompt (+ judge model id)
+ content = f"{rubric_key}\n---\n{question}\n---\n{completion}"
(thread rubric_key through get()/put(); bust or namespace existing
content-keyed cache dirs so stale cross-rubric entries cannot be read)
Scope / surfaces
- Primary target:
src/explore_persona_space/eval/batch_judge.py - Grep the workflow surface for the pattern before editing and update every hit; list them in the plan.
Constraints / invariants
- Workflow-surface only — never experiment code,
configs/, ortasks/. scripts/workflow_lint.py --check-askspasses; ruff on touched files passes.- This session runs under
EPM_WORKFLOW_FIX_SESSION=1and carries aworkflow_fix_target:Provenance line — it MUST NOT auto-route any of its own subagents' workflow-fix candidates (recursion guard).
Provenance
- workflow_fix_target: src/explore_persona_space/eval/batch_judge.py
- fingerprint: fe8ef634bf02
- related_task: #810
- source: /daily 2026-07-02 backfill problem sweep (route 2)
Activity