real-time-streams - #203
Conversation
- Created docs/real-time-event-streams/overview.mdx - Created docs/real-time-event-streams/setup.mdx - Created docs/real-time-event-streams/delivery-and-reliability.mdx - Created docs/real-time-event-streams/states-and-monitoring.mdx - Updated docs.json Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams.mdx - Updated docs/real-time-event-streams/states-and-monitoring.mdx - Updated docs/real-time-event-streams/delivery-and-reliability.mdx - Updated docs/real-time-event-streams/overview.mdx - Updated docs/real-time-event-streams/setup.mdx - Updated docs.json Mintlify-Source: dashboard-editor
|
@copilot resolve the merge conflicts in this pull request |
|
|
|
||
| {/* PLACEHOLDER: Describe which changes can be made in place (event types, filters, template, auth) and which require pausing the Sync. Clarify how in-flight events are handled during edits. */} | ||
|
|
||
| ## Next steps | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card title="Delivery and reliability" icon="shield-check" href="/docs/real-time-event-streams/delivery-and-reliability"> | ||
| Retries, failed events log, and at-least-once delivery. | ||
| </Card> | ||
|
|
||
| <Card title="States and monitoring" icon="chart-line" href="/docs/real-time-event-streams/states-and-monitoring"> | ||
| Destination and Sync states, plus how to monitor health. | ||
| </Card> | ||
| </CardGroup> |
There was a problem hiding this comment.
Missing troubleshooting and FAQ
The new documentation ends without the troubleshooting and FAQ material required by the automatically discovered STREAM-310 acceptance criteria, leaving customers without consolidated recovery guidance for common configuration failures.
Source Used: Linear — Public docs
Resolve merge conflict in docs.json by taking the updated structure from main branch. The main branch restructured guides navigation, moving guides from guides-by-use-case/empower-your-team to a flatter structure with better grouping. Since the real-time-streams branch is adding documentation about real-time event streams and not modifying the guides structure, we accept the updated version from main. Co-authored-by: ryancurtis1 <176308989+ryancurtis1@users.noreply.github.com>
✅ Merge conflicts resolved. I've merged the The merge commit has been created and pushed. |
| - **Sync** — a rule that binds a set of filtered events and a Liquid payload template to a Destination. | ||
| - **Failed events log** — a 72-hour record of events that could not be delivered after all retries, retained with the destination HTTP response for debugging. | ||
|
|
||
| <Tip> |
There was a problem hiding this comment.
this is an empty callout in the docs. should add something here or remove
There was a problem hiding this comment.
@copilot can you remove this empty callout please
There was a problem hiding this comment.
Done. Removed the empty callout at line 59 of overview.mdx.
| @@ -0,0 +1,3 @@ | |||
| --- | |||
| title: "docs/real-time-event-streams" | |||
There was a problem hiding this comment.
to the greptile comments point the way this is displayed current should be updated. i don't think this page necessarily needs a bunch of content but we should have something (at least a title instead of the path)
There was a problem hiding this comment.
Done. Replaced the path-based title with "Real-Time Event Streams" and added a proper description to match the pattern of other index pages.
Remove the empty <Tip> block that contained only a placeholder comment at line 59 of docs/real-time-event-streams/overview.mdx. Co-authored-by: ryancurtis1 <176308989+ryancurtis1@users.noreply.github.com>
Replace the path-based title "docs/real-time-event-streams" with a proper customer-facing title and add a description to match the pattern of other index pages in the docs. Co-authored-by: ryancurtis1 <176308989+ryancurtis1@users.noreply.github.com>
|
|
||
| Up to **5 retries per event** with exponential backoff over approximately **12 minutes**: | ||
|
|
||
| | Attempt | Delay before attempt | |
There was a problem hiding this comment.
the code does up to 5 total attempts (1 initial + 4 retries) with exponential backoff starting at 200ms (~3s total); 429s honor Retry-After, capped at 10s.
retries run inline in the same consumer path, so we keep the window to seconds - a long in-process backoff would block the batch and hold PubSub messages. longer recovery is handled by pausing the stream and surfacing the error, not by retrying for minutes. Can we update the table to match?
|
|
||
| - **Destination** — the webhook endpoint and its authentication configuration. | ||
| - **Sync** — a rule that binds a set of filtered events and a Liquid payload template to a Destination. | ||
| - **Failed events log** — a 72-hour record of events that could not be delivered after all retries, retained with the destination HTTP response for debugging. |
There was a problem hiding this comment.
In beta the failed log is an aggregated roll-up: one row per stream + status + response code per window, with the response status code, a short error detail, and counts. no per-event entries, no headers/body, no payload samples. Can we align the description?
| | `5xx` response | Yes | Server-side error at the destination. | | ||
| | Connection timeout / network error | Yes | Includes DNS failures and TLS handshake errors. | | ||
| | `429 Too Many Requests` | Yes | Handled via backpressure. See [Rate limits](#rate-limits-and-backpressure). | | ||
| | `4xx` response (other than 429) | No | Surfaced immediately as a destination configuration error. | |
There was a problem hiding this comment.
Only 401/403 are terminal (auth). Other 4xx (400/404/422…) are retried as "unreachable" and, if they keep failing, the stream is paused (DESTINATION_UNHEALTHY) - so they are retried and do pause, the opposite of this section. Can we correct it?
| | `unhealthy` | Deliveries are partially failing but no event has exhausted its retries yet. Investigate soon. | | ||
| | `paused_rate_limit` | Temporarily paused after a `429` response from the destination. Auto-resumes when the destination recovers. | |
There was a problem hiding this comment.
In beta there's no unhealthy running state and no paused_rate_limit auto-resume.
a terminal failure - including rate-limit after retries - pauses the stream and surfaces the health reason. Can we mark these as GA-target and drop them from the beta states table?
| | 5 | 2 minutes | | ||
| | 6 | 10 minutes | | ||
|
|
||
| Retries are processed in a separate queue, so they never block new events from streaming. |
There was a problem hiding this comment.
Small correction: retries run in the same consumer path, not a separate queue.
|
|
||
| ## Delivery guarantees at a glance | ||
|
|
||
| - **At-least-once delivery.** Every outbound request includes an `X-Mixpanel-Insert-Id` header so you can deduplicate on your side. |
There was a problem hiding this comment.
This is a current gap - the X-Mixpanel-Insert-Id header isn't sent yet. i've filed a ticket and will fix it. https://linear.app/mixpanel/issue/STREAM-860/send-x-mixpanel-insert-id-header-on-webhook-deliveries
can we drop the X-Mixpanel-Insert-Id statements for now and add them back once it lands?
- Updated docs/real-time-event-streams/delivery-and-reliability.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/delivery-and-reliability.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/delivery-and-reliability.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/overview.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/overview.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/overview.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/overview.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/setup.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/setup.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/setup.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/setup.mdx - Updated docs/real-time-event-streams/delivery-and-reliability.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/delivery-and-reliability.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/states-and-monitoring.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/states-and-monitoring.mdx Mintlify-Source: dashboard-editor
- Updated docs/real-time-event-streams/states-and-monitoring.mdx Mintlify-Source: dashboard-editor
Mintlify
10 threads from 2 users in Mintlify