workflow-fix: gcp poller must map PENDING → running (FLEX_START queue false-stall)
Highlight text on any card (body, plan, or an event) to anchor a comment, or leave a whole-task comment here. Mention @claude to summon a reply.
No comments yet.
Overview / Motivation
Auto-filed by the workflow-fix-on-bug protocol from a workflow-fix candidate raised on task #778 (emitting agent: orchestrator (/issue skill)).
Goal
Handle GCE status PENDING in _gcp_status_to_poll_result so a FLEX_START-queued instance is polled as status=running current_phase=pending (still waiting for capacity, keep polling) instead of the current false-stalled routing that would set-status blocked on a healthy queued instance.
Workflow gap
- Bug observed:
GcpBackend.poll(via_gcp_status_to_poll_resultinsrc/explore_persona_space/backends/gcp.py) returns{"status": "stalled", "current_phase": "unknown_pending"}for a FLEX_START-queued GCE instance whose lifecycle state isPENDING. Per/issueSKILL.md Step 6d.2's bg-Bash poll pseudocode,status=="stalled"triggerspost epm:failure v1 ... run CRON-TEARDOWN ... set-status blocked; exit. Live reproduction on task #778: instanceeps-issue-778inus-central1-bcreated 02:07 UTC, sat legitimately in the FLEX_START capacity queue atPENDING, andscripts/backend_poll.py --issue 778returned the false-stalled JSON. Following the SKILL literal would have blocked a healthy run. - Why it is a workflow gap: The reconnect probe
reconnect_or_none(same file) already treatsPENDINGas live — it only excludes_NONLIVE_INSTANCE_STATUSES = {TERMINATED, STOPPED, SUSPENDED}from reconnect. The comment at line 3521 explicitly notes "FLEX_START rungs legitimately stay PENDING past the GCLOUD_DEFAULT_TIMEOUT_SEC subprocess cap" — PENDING is a known-legitimate live state on the credit-spending lane. Yet_gcp_status_to_poll_resultmaps onlyRUNNING,PROVISIONING,STAGING,STOPPING,REPAIRING,TERMINATED,STOPPED,SUSPENDED, lettingPENDINGfall through the finalreturn _coarse_poll(status="stalled", current_phase=f"unknown_{up.lower()}")line. The reconnect-vs-poll asymmetry is the bug: reconnect thinks PENDING is live, poll thinks PENDING is stalled. - Confidence (emitter): high
Proposed change (candidate diff sketch — refine in planning)
--- a/src/explore_persona_space/backends/gcp.py
+++ b/src/explore_persona_space/backends/gcp.py
@@ _gcp_status_to_poll_result
up = status.upper()
if up == "RUNNING":
return _coarse_poll(status="running", current_phase="running")
- if up in {"PROVISIONING", "STAGING"}:
+ if up in {"PROVISIONING", "STAGING", "PENDING"}:
return _coarse_poll(status="running", current_phase=up.lower())
if up in {"STOPPING", "REPAIRING"}:
return _coarse_poll(status="stalled", current_phase=up.lower())
if up in {"TERMINATED", "STOPPED", "SUSPENDED"}:
return _terminal_dead_poll(reason=up.lower())
return _coarse_poll(status="stalled", current_phase=f"unknown_{up.lower()}")
Alternative if the planner prefers a distinct current_phase name: keep PENDING in its own branch returning status="running", current_phase="pending" so log readers can tell "FLEX_START capacity-queued" from "PROVISIONING (booting)".
Add a unit test covering:
_gcp_status_to_poll_result("PENDING")returnsstatus="running"(regression pin).- The full
GcpBackend.pollpath with a mockedgcloud describereturningstatus: PENDINGprintsstatus=runningfrombackend_poll.py, NOTstatus=stalled.
Scope / surfaces
- Primary target:
src/explore_persona_space/backends/gcp.py(the_gcp_status_to_poll_resultfunction + its docstring; the docstring's status enum list needs the addition too). - Also update:
tests/test_gcp_backend.py(add the PENDING regression test). - No changes to
reconnect_or_none— it already treats PENDING as live. - Grep for other
PENDINGreferences in the backend/router surface to confirm they already treat it as live (the audit at emit-time found:_NONLIVE_INSTANCE_STATUSESexplicitly excludes it via non-membership; line 3521 comment acknowledges the state; nothing else routes on it).
Constraints / invariants
- Workflow-surface only — never experiment code,
configs/, ortasks/. scripts/workflow_lint.py --check-askspasses; ruff on touched files passes.- Router tests (
tests/test_router*.py) still pass — this fix is on the poll path, not the launch path. - This session runs under
EPM_WORKFLOW_FIX_SESSION=1(via the auto-set Provenance line) and MUST NOT auto-route any of its own subagents' workflow-fix candidates (recursion guard,.claude/rules/workflow-fix-on-bug.md§ Recursion guard).
Provenance
- workflow_fix_target: src/explore_persona_space/backends/gcp.py
- fingerprint: 2e28249543fd
target_file: src/explore_persona_space/backends/gcp.py bug_observed: GcpBackend.poll returns status=stalled current_phase=unknown_pending for a FLEX_START-queued GCE instance in state PENDING; per Step 6d.2 that would post epm:failure v1 and set-status blocked on a healthy queued instance (live repro on task #778, eps-issue-778 in us-central1-b sat PENDING in the FLEX_START capacity queue for ~1h47m). why_workflow_gap: reconnect_or_none in the same file treats PENDING as live (only {TERMINATED, STOPPED, SUSPENDED} are non-live); the poll's _gcp_status_to_poll_result should mirror that and map PENDING to status=running current_phase=pending, but instead it falls through to unknown_pending -> stalled. proposed_change: Handle GCE status PENDING in _gcp_status_to_poll_result — map to status=running current_phase=pending (FLEX_START-queued instance is legitimately live, poller should keep polling not stall). diff_sketch: | --- a/src/explore_persona_space/backends/gcp.py +++ b/src/explore_persona_space/backends/gcp.py @@ _gcp_status_to_poll_result up = status.upper() if up == "RUNNING": return _coarse_poll(status="running", current_phase="running")
- if up in {"PROVISIONING", "STAGING"}:
- if up in {"PROVISIONING", "STAGING", "PENDING"}: return _coarse_poll(status="running", current_phase=up.lower()) if up in {"STOPPING", "REPAIRING"}: return _coarse_poll(status="stalled", current_phase=up.lower()) confidence: high related_task: #778