Summary
A streaming chat request can be accepted by the provider (HTTP 200) and then terminated in-band: the stream delivers an error event instead of completing. In that case the openai SDK raises a plain Error from Stream.iterator carrying only a provider code property, and @core-ai/openai's stream transform (transformStream2) lets it propagate unwrapped — it never becomes one of core-ai's typed errors, so downstream classifiers that match on RateLimitError / RetryableProviderError treat it as an unknown generic failure.
Observed
Azure OpenAI under burst load (>64 concurrent streaming calls against one deployment): 43 of 100 runs failed mid-stream with
Error: Your requests to <deployment> for <deployment> in swedencentral have exceeded rate limit.
at Stream.iterator (openai/src/core/streaming.ts:70:21)
at transformStream2 (@core-ai/openai/dist/chunk-MKAXZHMO.js:1847:20)
at @core-ai/core-ai/dist/index.js:655:5
at pump (@core-ai/core-ai/dist/index.js:530:22)
with error.code === 'rate_limit_exceeded' — name is plain Error, no HTTP status, because the request itself succeeded. Consumers classifying via instanceof RateLimitError (or the RetryableProviderError base) miss it and report a generic model failure to users instead of a transient/overloaded condition.
Proposed fix
In the openai adapter's stream transform, map in-band stream errors to core-ai's typed errors before rethrowing — at minimum:
code: 'rate_limit_exceeded' → RateLimitError
and consider a general mapping table for other in-stream provider codes (service unavailable, overloaded) so the streaming path produces the same typed errors as the non-streaming request path already does.
Repro
Fire ~100 concurrent streaming chat completions at an Azure OpenAI deployment with a modest rate limit; a subset of accepted streams terminates mid-flight with the error above.
Summary
A streaming chat request can be accepted by the provider (HTTP 200) and then terminated in-band: the stream delivers an error event instead of completing. In that case the openai SDK raises a plain
ErrorfromStream.iteratorcarrying only a providercodeproperty, and@core-ai/openai's stream transform (transformStream2) lets it propagate unwrapped — it never becomes one of core-ai's typed errors, so downstream classifiers that match onRateLimitError/RetryableProviderErrortreat it as an unknown generic failure.Observed
Azure OpenAI under burst load (>64 concurrent streaming calls against one deployment): 43 of 100 runs failed mid-stream with
with
error.code === 'rate_limit_exceeded'—nameis plainError, no HTTP status, because the request itself succeeded. Consumers classifying viainstanceof RateLimitError(or theRetryableProviderErrorbase) miss it and report a generic model failure to users instead of a transient/overloaded condition.Proposed fix
In the openai adapter's stream transform, map in-band stream errors to core-ai's typed errors before rethrowing — at minimum:
code: 'rate_limit_exceeded'→RateLimitErrorand consider a general mapping table for other in-stream provider codes (service unavailable, overloaded) so the streaming path produces the same typed errors as the non-streaming request path already does.
Repro
Fire ~100 concurrent streaming chat completions at an Azure OpenAI deployment with a modest rate limit; a subset of accepted streams terminates mid-flight with the error above.