workflow-fix: janitors fail rmtree on symlinked per-issue cache dirs
kind: infra#wf-fix#wf-fix-fp:43b535edca3f
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Auto-filed by the workflow-fix-on-bug protocol from a workflow-fix candidate raised during a chat-mode disk-full diagnosis (2026-07-03; emitting agent: orchestrator).
Goal
Handle symlinked per-issue cache dirs in the janitors: unlink the symlink and reap the target only when it resolves inside the managed eps-data cache root, instead of failing rmtree on the link.
Workflow gap
- Bug observed:
vm_disk_guard.py --applyreportedissue 761: FAILED to remove data/issue_761/g1_dl(andhf_dl) and exited 2, because those cache dirs are symlinks to/mnt/eps-data/thomasjiralerspong/eps-data/issue_761/...andshutil.rmtreerefuses symlinks — despite zero boot-disk bytes at stake. - Why it is a workflow gap: relocating a per-issue cache onto the data disk via a symlink is the natural interim pattern before the #681 cutover (and #761 already did it); every janitor that assumes cache dirs are real directories now fails loud on the healthiest configuration, and the guard's nonzero exit masks real failures.
- Confidence (emitter): high
Proposed change (candidate diff sketch — refine in planning)
clean_experiment_downloads.py / vm_disk_guard.py reap path:
+ if os.path.islink(cache_dir):
+ target = os.path.realpath(cache_dir)
+ os.unlink(cache_dir) # link itself is boot-disk-free
+ if target.startswith(EPS_DATA_CACHE_ROOT): # managed relocation only
+ shutil.rmtree(target, ignore_errors=False)
+ else: log "symlink unlinked, external target kept: <target>"
- shutil.rmtree(cache_dir) # raises on symlink today
Scope / surfaces
- Primary target:
scripts/clean_experiment_downloads.py, scripts/vm_disk_guard.py - Grep both janitors for every
rmtreecall on a per-issue cache path (grep -n 'rmtree' scripts/clean_experiment_downloads.py scripts/vm_disk_guard.py) and cover each; add a symlinked-cache test. - Sibling task #911 (widen janitor glob coverage) touches the same files — the planner should coordinate/sequence with it, but this is a distinct bug (rmtree-on-symlink failure, not glob coverage).
Constraints / invariants
- Workflow-surface only — never experiment code,
configs/, ortasks/. - NEVER delete
store/,eval_results/, or any active issue's data; never follow a symlink outside the managed eps-data cache root. scripts/workflow_lint.py --check-askspasses; ruff on touched files passes.- This session runs under
EPM_WORKFLOW_FIX_SESSION=1and carries aworkflow_fix_target:Provenance line — it MUST NOT auto-route any of its own subagents' workflow-fix candidates (recursion guard).
Provenance
- workflow_fix_target: scripts/clean_experiment_downloads.py, scripts/vm_disk_guard.py
- fingerprint: 43b535edca3f
Raised during the 2026-07-02/03 disk-full diagnosis: a vm_disk_guard.py --apply run (log: /tmp/vm_disk_guard_run.log) freed 0.00G and exited 2 on the #761 symlinked caches.
Activity