fix: reclaim the AgentInstance task slot after an interrupted turn

[QuentinBisson]

Fixes #2469.

The problem

agent_instance_one_active_task_idx reserves one non-terminal task per AgentInstance. The gateway terminates a task on every error path it can observe (failTask on dial failure, send error, stream error, store error), but a turn that ends without one of those paths running writes no terminal state at all.

The row then stays non-terminal, and because the index reserves the slot, every later send to that AgentInstance fails with ErrAgentInstanceTaskConflict, "AgentInstance already has an active task".

Two ways in, and the second is the ordinary one:

• The process serving the turn stops, through a rollout, an OOM kill or an eviction.
• The client disconnects mid-stream. In SendStreamingMessage the persistence call and the failTask that would record the failure both run on the request context, so the cancellation takes out the recovery along with the write. if !yield(event, nil) { return } leaves without a terminal write too. This is the mechanism [BUG] A2A client disconnect can cancel terminal task persistence and leave tasks working #2447 reports for the v1 store; the index is what turns it from a stuck task into an unusable instance.

Nothing recovers it. CancelTask dials the runtime first, and a restarted process has no record of a task it never loaded, so the cancel fails and the slot is never cleared. There is no reaper, lease or takeover in go/core/v2. The only escape is deleting the row by hand.

The change

Reserving the slot now first terminates an active task that has not reported progress within a configurable bound. It happens inside the transaction that reserves the slot, holding the incumbent with FOR UPDATE, so the takeover cannot race a task that is still running.

The interruption is appended to the task history as well as written to status, so a client renders it as part of the conversation without needing its own handling for the terminal state.

Staleness is measured on updated_at rather than status.timestamp: it is a server clock, it is always set, and every persisted event refreshes it.

AGENT_INSTANCE_TASK_STALE_AFTER configures the bound and defaults to 1h. The bound has to clear the longest quiet period of a live turn, and the constraining case is a non-streaming SendMessage, which persists nothing between submit and return. Setting it to 0 disables the takeover and restores the previous behaviour.

Scope

This makes the state recoverable. It does not stop the orphan being created, and for the disconnect trigger the prevention is to decouple persistence and the recovery write from the client's context, which is what #2447 asks for on the v1 store. The two are complementary, and that one is deliberately not in this PR.

Notes for review

• The bound is a heuristic. A stronger signal is available here and is deliberately left out: the gateway can ask the runtime whether it still knows the task, which taskForEvent already does for event folding. That would make silence a trigger rather than a verdict and allow a much tighter bound. It is easier to argue once the slot can be reclaimed at all.
• If a takeover or lease is already planned as part of the durable-task work, say so and I will close this in favour of it. The issue stands on its own either way.
• CreateAgentInstanceTask gains the bound as an explicit parameter rather than reading hidden client state, which is what moves the call sites in the tests.
• No migration. state and updated_at are already columns on agent_instance_task, and the lookup rides the existing partial unique index.

[Copilot]

Pull request overview

This PR makes AgentInstance task-slot reservation self-healing by allowing CreateAgentInstanceTask to reclaim the “one active task” slot when the incumbent task has been silent past a configurable bound, preventing permanently stuck AgentInstances after interrupted turns.

Changes:

• Add a staleness-bound takeover to CreateAgentInstanceTask that terminates a non-terminal task whose updated_at is older than AGENT_INSTANCE_TASK_STALE_AFTER (default 1h, 0 disables).
• Introduce a FOR UPDATE query to lock the active task row inside the reservation transaction to avoid races during takeover.
• Update gateway/controller wiring and tests to thread the new taskStaleAfter configuration through call sites.

Reviewed changes

Copilot reviewed 7 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
go/core/v2/a2agateway/gateway.go	Threads taskStaleAfter into task creation via the store interface and adds gateway config field.
go/core/v2/a2agateway/gateway_test.go	Updates test store + gateway construction to account for the new taskStaleAfter parameter.
go/core/internal/database/queries/agent_instance_tasks.sql	Adds LockActiveAgentInstanceTask query to lock the non-terminal task row for takeover safety.
go/core/internal/database/gen/querier.go	Regenerates sqlc interface to include LockActiveAgentInstanceTask.
go/core/internal/database/gen/agent_instance_tasks.sql.go	Regenerates sqlc implementation for LockActiveAgentInstanceTask.
go/core/internal/database/client_postgres.go	Implements stale-task termination + event recording before reserving the active-task slot.
go/core/internal/database/client_agent_instance_test.go	Adds coverage for stale-task termination reclaiming the slot; updates existing calls for new signature.
go/core/cmd/controller-v2/main.go	Adds AGENT_INSTANCE_TASK_STALE_AFTER parsing and wires it into gateway construction.
go/api/database/client.go	Updates the public DB client interface/docs to accept the staleness bound parameter.

Files not reviewed (2)

• go/core/internal/database/gen/agent_instance_tasks.sql.go: Generated file
• go/core/internal/database/gen/querier.go: Generated file

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.