daily-fix: failover relaunch must sync new pod into pods.conf
kind: infra#wf-fix#wf-fix-fp:1e207e2050f6#daily-auto-filed
track:Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Auto-filed by /daily 2026-06-29 problem sweep. Recurring across pod-664, pod-697, pod-658.
Goal
A failover/relaunched pod is always registered in pods.conf, and the documented recovery commands work when the pod is absent.
Workflow gap
- Bug observed: GCP->RunPod failover relaunch (backend_poll.py path) bypassed
pod.py provisionsync, leaving the new pod absent frompods.conf/SSH/MCP. The raw-SSH poller then readstatus=dead(false alarm), andpod.py config --update pod-697AND--refresh-from-api pod-697BOTH failed with "pod 'pod-697' not found in pods.conf" — the documented recovery path does not work for an absent pod. - Why it is a workflow gap: the failover relaunch path + the pod-config recovery surface are workflow infra (backends/ router + pod helper).
- Confidence (emitter): medium-high; reproduced in 3 independent sessions today.
Proposed change
- The failover relaunch in
backend_poll.pycalls the pod_config sync so the new pod lands in pods.conf (the same registrationpod.py provision/resumeperform). pod.py config --update/--refresh-from-apiCREATE a missing entry from the live RunPod API instead of erroring when the pod is absent.- Poller cross-checks the live API before declaring a pod dead on a name-resolution failure.
Scope / surfaces
src/explore_persona_space/backends/backend_poll.py(failover relaunch path)scripts/pod_config.py(--update / --refresh-from-api create-missing)- Grep
pods.conf/refresh-from-apicallers before editing.
Constraints / invariants
- Workflow/infra surface only. ruff on touched files; relevant
tests/test_router*.py/ pod-config tests pass. - This session runs under EPM_WORKFLOW_FIX_SESSION=1 (recursion guard).
Provenance
- workflow_fix_target: src/explore_persona_space/backends/backend_poll.py, scripts/pod_config.py
- fingerprint: 1e207e2050f6
Sessions: 3bded713 (issue-697), f317c8a8 (issue-664), 6185d397 (issue-658).
Activity