Summary
In agent-client-protocol-http, a websocket subscriber that falls more than OUTBOUND_STREAM_CAPACITY (1024) notifications behind is silently dropped, and the connection closes with code 1006 and no close frame. The only log is a debug!, so at default log levels the server records nothing at all. Any agent that emits a burst of more than 1024 outbound messages faster than the socket drains loses the connection deterministically.
We hit this through Goose's desktop app: session/load on a large session replays one session/update per content item, and every load of a session beyond roughly a thousand messages killed the connection. Downstream report with full debugging history: aaif-goose/goose#10664.
Mechanism (v1.0.1, src/agent-client-protocol-http/src/connection.rs)
OutboundStream::subscribe creates a bounded mpsc::channel(OUTBOUND_STREAM_CAPACITY) per subscriber (line 76; capacity constant at line 85).OutboundStream::push fans out with try_send and treats TrySendError::Full the same as Closed: the subscriber is removed from the list (lines 61-70). The log on this path is debug!("outbound subscriber queue full; closing subscriber stream").- In
websocket_server.rs, the ws writer's outbound_rx.recv() returns None, the loop breaks with no log, and the SplitSink<WebSocket> is dropped without a close handshake. The client observes 1006 with an empty reason.
The core agent-client-protocol crate's outgoing path is unbounded end to end, so producers (send_notification) never feel backpressure and can always outrun this one bounded stage.
Reproduction
Serve any ACP agent over the HTTP transport, then from a plain ws client (Node, maxPayload raised, acking immediately) trigger a method that produces >1024 notifications in a tight loop. We used Goose's session/load on a 33,509-message session: the client receives ~1,600 notifications (~14.5 MB) in ~1.5s, then 1006. With OUTBOUND_STREAM_CAPACITY raised to 1 << 20 and nothing else changed, the same load completes: 33,711 notifications, 179 MB, 5.3s.
Suggested fix
A bigger buffer only moves the cliff. The durable fix is real backpressure: in OutboundStream::push, snapshot the sender clones, release the state mutex, then send().await to each subscriber (removing only on Closed), so a slow socket throttles the producer instead of being dropped. Holding the mutex across an awaited send would serialize or deadlock, hence the snapshot-first shape. If dropping remains a deliberate policy for lagging subscribers, it should at least log at warn! and close the websocket with a proper close frame and reason so clients can distinguish overflow from network failure.
Happy to open a PR along those lines if the approach sounds right to maintainers.
Summary
In
agent-client-protocol-http, a websocket subscriber that falls more thanOUTBOUND_STREAM_CAPACITY(1024) notifications behind is silently dropped, and the connection closes with code 1006 and no close frame. The only log is adebug!, so at default log levels the server records nothing at all. Any agent that emits a burst of more than 1024 outbound messages faster than the socket drains loses the connection deterministically.We hit this through Goose's desktop app:
session/loadon a large session replays onesession/updateper content item, and every load of a session beyond roughly a thousand messages killed the connection. Downstream report with full debugging history: aaif-goose/goose#10664.Mechanism (v1.0.1,
src/agent-client-protocol-http/src/connection.rs)OutboundStream::subscribecreates a boundedmpsc::channel(OUTBOUND_STREAM_CAPACITY)per subscriber (line 76; capacity constant at line 85).OutboundStream::pushfans out withtry_sendand treatsTrySendError::Fullthe same asClosed: the subscriber is removed from the list (lines 61-70). The log on this path isdebug!("outbound subscriber queue full; closing subscriber stream").websocket_server.rs, the ws writer'soutbound_rx.recv()returnsNone, the loop breaks with no log, and theSplitSink<WebSocket>is dropped without a close handshake. The client observes 1006 with an empty reason.The core
agent-client-protocolcrate's outgoing path is unbounded end to end, so producers (send_notification) never feel backpressure and can always outrun this one bounded stage.Reproduction
Serve any ACP agent over the HTTP transport, then from a plain
wsclient (Node,maxPayloadraised, acking immediately) trigger a method that produces >1024 notifications in a tight loop. We used Goose'ssession/loadon a 33,509-message session: the client receives ~1,600 notifications (~14.5 MB) in ~1.5s, then 1006. WithOUTBOUND_STREAM_CAPACITYraised to1 << 20and nothing else changed, the same load completes: 33,711 notifications, 179 MB, 5.3s.Suggested fix
A bigger buffer only moves the cliff. The durable fix is real backpressure: in
OutboundStream::push, snapshot the sender clones, release thestatemutex, thensend().awaitto each subscriber (removing only onClosed), so a slow socket throttles the producer instead of being dropped. Holding the mutex across an awaitedsendwould serialize or deadlock, hence the snapshot-first shape. If dropping remains a deliberate policy for lagging subscribers, it should at least log atwarn!and close the websocket with a proper close frame and reason so clients can distinguish overflow from network failure.Happy to open a PR along those lines if the approach sounds right to maintainers.