Repository navigation
feat: add ssf - #123731
feat: add ssf#123731ReeceJones wants to merge 2 commits into
Conversation
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
There was a problem hiding this comment.
PostHog Review · Standard · reviewed 98c422d: 2 Must fix, 12 Should fix. This pull request is large, so the review ran in 4 parts with less depth than usual.
This is the only automatic review while this pull request is a draft. Comment @posthog review to review new changes sooner. Otherwise, a review of the new changes runs when you mark it ready for review.
See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.
| except (jwt.PyJWTError, ValueError, TypeError, OverflowError) as error: | ||
| if isinstance(error, SETValidationError): | ||
| raise | ||
| raise SETValidationError("invalid_request", "The security event token is invalid.") from error |
There was a problem hiding this comment.
P1 · Preserve retryable errors from signing-key retrieval
When JWKS retrieval times out or returns a transient error, SSFRemote.select_key() raises SSFRemoteError, which inherits ValueError. This handler converts it to SETValidationError, so SSFReceiverView returns HTTP 400 instead of its intended retryable 503, causing transmitters to discard valid events; let SSFRemoteError propagate.
| ) | ||
|
|
||
| urlpatterns = [ | ||
| path(".well-known/ssf-configuration/<uuid:organization_id>", SSFDiscoveryView.as_view(), name="ssf-discovery"), |
There was a problem hiding this comment.
P1 · Register discovery at the path derived from the advertised issuer
For the advertised issuer /ssf/<organization_id>, SSF §7.2 requires discovery at /.well-known/ssf-configuration/ssf/<organization_id>. This route omits /ssf, so standards-compliant receivers, including SSFRemote.discovery_url, receive a 404 and cannot discover PostHog automatically. Register the issuer-derived path and align the advertised discovery URL.
| if ( | ||
| stream.connection | ||
| and stream.connection.direction == SSFConnection.Direction.INCOMING | ||
| and stream.remote_stream_id |
There was a problem hiding this comment.
P2 · Recover the upstream stream before deleting failed setup
When the transmitter creates a stream but its response is lost, the local stream retains an empty remote_stream_id. Deleting that failed stream or its connection skips the remote DELETE and removes the local recovery marker, leaving the upstream stream delivering to a nonexistent receiver; recover its ID before completing deletion.
| SSFRemote.request( | ||
| "DELETE", | ||
| SSFProvisioning.endpoint(stream.connection, "configuration_endpoint"), | ||
| token=stream.connection.remote_token, | ||
| params={"stream_id": stream.remote_stream_id}, | ||
| ) |
There was a problem hiding this comment.
P2 · Treat an absent upstream stream as successful deletion
When an upstream DELETE succeeds but its response is lost, SSFRemote.request() raises and preserves the local stream. If the transmitter returns HTTP 404 on retry, that also raises before stream.delete(), permanently blocking stream and connection removal through the API; treat an already absent upstream stream as successful cleanup.
| values.streams?.some((stream) => stream.verification_pending || stream.is_busy) | ||
| ) { | ||
| actions.loadStreams() | ||
| } | ||
| }, 2000) |
There was a problem hiding this comment.
P2 · Back off verification polling after failed stream reads
When a polling request fails during verification, the loader retains the previous streams with verification_pending set. This interval retries every two seconds indefinitely because only a successful response clears that stale flag, repeatedly hitting an unhealthy backend and generating error notifications. Back off failed reads and honor Retry-After.
| SSFService._queue(config, stream, event_type, subject, payload, txn=transaction_id) | ||
| count += 1 |
There was a problem hiding this comment.
P2 · Isolate queue backpressure between outgoing streams
When one subscribed receiver stops acknowledging events and reaches the queue limit, _queue() raises inside the transaction covering every stream. This rolls back deliveries already queued for healthy receivers and prevents subsequent deliveries to them on every retry. Isolate each stream's queue failure and retain retryable work for the blocked receiver.
| SSFService.poll_stream( | ||
| self.config, | ||
| self.stream(kwargs["stream_id"]), | ||
| max_events=data["maxEvents"], | ||
| return_immediately=data["returnImmediately"], | ||
| acknowledged=[*data["ack"], *data["setErrs"]], |
There was a problem hiding this comment.
P2 · Avoid blocking web workers for default polling requests
With an empty queue and the default returnImmediately=false, this synchronous call occupies a web worker for five seconds. poll_stream also reacquires database locks every 250 ms while waiting. Concurrent receivers consume worker capacity and database connections without delivering events, delaying other requests. Use asynchronous waiting or immediate responses.
| eventTypes: stream.events_requested.filter((event): event is SSFOutgoingEventTypeEnumApi => | ||
| Object.values(SSFOutgoingEventTypeEnumApi).some((preset) => preset === event) | ||
| ), |
There was a problem hiding this comment.
P2 · Preserve control-event subscriptions when editing streams
Editing a stream that subscribes to SSF verification or stream-updated events filters those subscriptions out of eventTypes. Saving even a description-only edit then sends the filtered array as events_requested, and the backend replaces the original subscriptions. Preserve event types omitted from the editor, or omit events_requested when unchanged.
| key = SSFRemote._cache_key("discovery", url) | ||
| metadata = None if force else cache.get(key) | ||
| if metadata is None: | ||
| document = SSFRemote.request("GET", url) | ||
| if not isinstance(document, dict) or document.get("issuer") != connection.issuer: | ||
| raise SSFRemoteError("The discovered issuer does not exactly match the trusted issuer URL.") |
There was a problem hiding this comment.
P2 · Validate the expected issuer when reusing cached discovery metadata
When two connections reuse transmitter_url with different issuers, the second discover call returns the first connection's cached metadata without checking its issuer. refresh_connection then trusts that transmitter's JWKS for the second identity, bypassing the cold-request trust check. Include the expected issuer in the cache key or validate every cached result.
| if event_type == VERIFICATION_EVENT: | ||
| expected_state = stream.verification_state | ||
| if payload.get("state") != expected_state and ("state" in payload or expected_state is not None): | ||
| raise SETValidationError( | ||
| "invalid_state", "The verification state does not match the expected value." | ||
| ) |
There was a problem hiding this comment.
P2 · Distinguish unsolicited verification events from pending challenges
After provisioning completes its verification challenge, verification_state retains the confirmed nonce. A later transmitter-initiated verification must omit state under SSF §8.1.4.2, but this comparison rejects it with invalid_state, breaking subsequent health checks. Track whether a challenge remains pending and handle unsolicited verification separately.
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the |
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the |
Problem
Reviewers need smaller diffs to assess this SSF implementation.
Changes
The implementation now lives in seven draft PRs, listed in review order:
Parts 1–4 form the backend stack. Parts 5–7 form a second stack based on #132008, keeping both stacks below five layers.
This original draft remains open as a reference. The original note was: “going to split this pr into a bigger stack, just using this for now while dev'ing”.
How did you test this code?
The split preserves the exact tree at
98c422d. Commit hooks, Python syntax and whitespace checks passed. Runtime tests and visual verification remain pending.Test rationale: The split preserves existing tests and assertions.
Release status
Docs update
No documentation behavior changed during this split.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Agent: Codex, GPT-6. Original implementation by ReeceJones; the agent redistributed unchanged files.
Skills: /stacking-prs, /writing-pr-descriptions, /verifying-changes, /running-ci-preflight. Each layer lists its additional applicable skills.
Rebase conflicts with current master and full validation remain outstanding before review readiness.