Repository navigation
fix(signals): stop repository selection treating app events as vendor work - #111226
posthog[bot] wants to merge 3 commits into
Conversation
… work Repository selection and research now treat a project's own events, errors, and logs as its own application, and ask for explicit evidence before they call the behavior vendor-owned. Research can name the repository that holds its code evidence. The research activity then replaces the selection agent's guess with that repository when it is connected, and clears the guess when it is not. Pins and corrections by people or scouts stay. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: e0cddaa9-2fac-4e82-8296-54df7caa2338
|
Merging to
After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here |
|
Risk: No findings This increment replaces the forgeable reason-prefix marker with a server-set Sentinel reviewed |
🤖 CI report
|
HostHog preview —
|
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. 🧰 Additional context used📚 Code guidelines (8)No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (4)
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review. 📝 WalkthroughWalkthroughReport research can identify a code repository other than the initially selected repository. The report activity reconciles that repository with connected repositories, then persists and returns the updated selection. The repository-selection prompt distinguishes application ownership from vendor ownership and clarifies when to return a null result. Priority: ⬇️ Low Merge Risk: 🔵 Low · up to A stale repository cache can temporarily clear a valid selection and delay report processing, but autostart is disabled and the repository can be rediscovered. Security Architecture ReviewSecurity architecture risk: 🟡 Moderate · up to A repository correction can leave an older reviewer identity active for the new destination. Cached connection data can also incorrectly clear a valid destination. Project-level access checks and existing execution limits contain the exposure, but destination and authorization state are not consistently updated together. Retained concerns
Security review detailsSecurity Blast Radius
Security Findings and Attack Paths
Trust Boundaries and Controls
Resilience and Maintainability Implications
Hardening Proposals
🚥 Pre-merge checks | ✅ 1✅ Passed checks (1 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 2
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (1)
products/signals/backend/temporal/agentic/report.py-748-748 (1)
748-748: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winPreserve the selection when repository connectivity is unavailable.
resolve_team_github_integrationexcludes unavailable integrations and returnsNone.list_team_connected_repositoriesconverts that state to[], so the reconciliation branch treats an unavailable lookup as a confirmed non-member and clears the existing selection. Return a distinct unavailable state and preserve the selection for it. Continue clearing the selection only when repository membership was successfully checked and the repository is absent.
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
- Review profile: QUIET
- Plan: Enterprise
- Run ID:
116e18b0-f968-4faa-a8e5-16fad47666fb
📒 Files selected for processing (6)
products/signals/backend/report_generation/AGENTS.mdproducts/signals/backend/report_generation/research.pyproducts/signals/backend/temporal/agentic/report.pyproducts/signals/backend/test/test_agentic_report_activity.pyproducts/tasks/backend/logic/repo_selection/agent.pyproducts/tasks/backend/tests/test_repo_selection_prompt.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.
…n reconcile A selection that research already wrote now stays on later runs, because only a selection agent guess yields. An empty connected-repository list means the installation is unavailable, so the guess stays instead of being cleared. The selection prompt now counts a match in a candidate that holds the vendor's own service as evidence for that candidate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: e0cddaa9-2fac-4e82-8296-54df7caa2338
A new stamphog review started for this PR — the fresh verdict replaces this approval.
|
Re the minor note on unavailable connectivity: fixed in bdf55c5. An empty connected-repository list now keeps the selection agent's guess, because the selection agent only ran with candidates, so an empty list means the installation is unavailable. A new test case covers it. |
|
There was a problem hiding this comment.
Note
Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.
🟡 Other comments (2)
products/signals/backend/temporal/agentic/report.py-752-754 (1)
752-754: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winDo not clear a repository guess based on a stale cache.
The selection agent refreshes the cache when needed, but reconciliation later reads it with
allow_refresh=False. If that nonempty snapshot omits the repository found by research, reconciliation can clear the selection, disable autostart, and say GitHub cannot reach the repository. Use a fresh membership check before clearing; preserve the selection when freshness is unknown.products/signals/backend/temporal/agentic/report.py-745-751 (1)
745-751: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winTrack selection provenance outside
reason.The selection agent supplies
reasonas model output, but the guard treats any reason with the research prefix as proof that research wrote the selection. If the agent returns that prefix and research finds a different repository,_reconcile_repo_selectioncan return the guess unchanged withautostart_eligible=True. Add an explicit provenance field toRepoSelectionResultand skip reconciliation only for a research-produced selection.
ℹ️ Review info
⚙️ Run configuration
- Configuration used: Repository: PostHog/posthog/.coderabbit.yaml
- Review profile: QUIET
- Plan: Enterprise
- Run ID:
253686c1-e5cd-4e31-a4d8-d7b789f3d070
📒 Files selected for processing (3)
products/signals/backend/temporal/agentic/report.pyproducts/signals/backend/test/test_agentic_report_activity.pyproducts/tasks/backend/logic/repo_selection/agent.py
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.
The reconcile guard read the selection reason prefix, which is model output and can be forged. A new `from_research` field on `RepoSelectionResult` now marks a research write. It is stripped from the selection prompt schema and reset after the selection agent runs, so the model cannot set it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Generated-By: PostHog Desktop Task-Id: e0cddaa9-2fac-4e82-8296-54df7caa2338
A new stamphog review started for this PR — the fresh verdict replaces this approval.
|
CodeRabbit notes: the provenance note is fixed in 71af1e0 ( |
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the |
|
This PR was closed due to lack of activity. Feel free to reopen if it's still relevant. |
A new stamphog review started for this PR — the fresh verdict replaces this approval.
There was a problem hiding this comment.
Not approved yet — waiting on the conditions below.
Re-add the stamphog label to request another review once you have addressed this.
The prerequisites gate refused this pull request because it has merge conflicts. The other gates passed: no deny-list categories matched, and the size and tier checks were within limits. Because the conflicts need to be resolved first, stamphog can't proceed with it. To move forward, the author can resolve the conflicts by rebasing or merging the base branch and pushing the update, or ask a human reviewer to take a look.
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✗ | merge conflicts present |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 115L, 4F substantive, 205L/7F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1c-medium (205L, 7F, single-area, fix) |
| stamphog 2.7.0 | .stamphog/policy.yml @ unknown · reviewed head 71af1e0 |
There was a problem hiding this comment.
PostHog Review · Standard · reviewed 71af1e0: 1 Must fix, 1 Should fix.
See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.
| autostart_eligible=False, | ||
| from_research=True, |
There was a problem hiding this comment.
P1 · Block reviewer-based autostart after a research correction
When a corrected report has a reviewer who meets the autonomy threshold, maybe_autostart_implementation_task selects that reviewer regardless of repository_autostart_eligible; this flag only guards the reviewer-less fallback. The report therefore opens a PR in the research-selected repository without a person starting it. Gate automatic starts on from_research too.
| await _persist_agentic_report_artefacts( | ||
| input.team_id, | ||
| input.report_id, | ||
| result, | ||
| input.repo_selection, | ||
| repo_selection, |
There was a problem hiding this comment.
P2 · Refresh reviewers when research changes the repository
When re-research changes repositories while confirming all existing findings, _persist_agentic_report_artefacts skips the new reviewer list because has_new_finding is false. The report retains the old repository's reviewers even after resolving the new owners, misrouting review and autostart identity. Refresh pipeline-selected reviewers whenever the repository changes.
|
This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the |
|
This PR was closed due to lack of activity. Feel free to reopen if it's still relevant. |
Problem
nullfor "billing" or "internal ops" questions, with no ownership check. A project's ownsubscription_*event read as a vendor billing question.Origin
Changes
nullcase.reasonmust name the ownership evidence.code_repository. When research found the code elsewhere, the activity replaces the selection before it resolves reviewers and persists artefacts.autostart_eligible=Falserepository=None), so the next run selects againNote
A reconciled selection never autostarts a PR, the same as a reviewer's wrong-repo correction. A person starts the run from the report.
How did you test this code?
products/tasks/backend/tests/test_repo_selection_prompt.py,products/signals/backend/test/test_agentic_report_activity.py, andproducts/signals/backend/test/test_research_prompt.pylocally against the dev stack database.ruff check,ruff format, andhogli lint:tachon the changed files.Test rationale:
test_run_agentic_report_activity_reconciles_the_selection_with_researchcovers the regression where research finds the code in another repository and the report keeps the earlier guess. The two replace and clear cases fail with the reconcile step removed. The closest existing tests (..._keeps_a_scout_repository_correction,..._keeps_reviewer_selection_written_mid_run) only cover corrections written by others.test_prompt_asks_for_ownership_evidence_before_a_vendor_verdictguards the ownership section and its place before thenullrules, next to the existing prompt-section tests.Release status
Automatic notifications
Docs update
products/signals/backend/report_generation/AGENTS.mdto describe the reconcile step.🤖 Agent context
Autonomy: Fully autonomous
Agent: PostHog Desktop (Claude Code), claude-opus-5-5
/writing-pr-descriptions.Created with PostHog Desktop from this inbox report.
🤖 Generated with Claude Code