Skip to content

fix: require standalone demand for standalone cache warming - #108313

Open
vdekrijger wants to merge 6 commits into
codex/cache-warming-view-contextfrom
codex/cache-warming-standalone-demand-policy
Open

vdekrijger wants to merge 6 commits into
codex/cache-warming-view-contextfrom
codex/cache-warming-standalone-demand-policy

Conversation

@vdekrijger

@vdekrijger vdekrijger commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Viewing a dashboard can keep an unused standalone insight query eligible for cache warming, even when nobody requests that standalone variant.

Changes

  • Require recent attributed standalone views before warming a standalone candidate, behind default-off cache-warming-standalone-demand.
  • Keep dashboard eligibility on its existing access rule. Unattributed history does not establish standalone demand.
  • Reuse the seven-day demand window and the recorder's one-minute coalescing allowance.
  • Find that demand with one query: the distinct insights with a sourced view outside a dashboard inside the window. No view rows are loaded into the scheduler.

Depends on #106556. Deploy recording first and collect a full seven days of history. Check web, shared and agent/API caller coverage before enabling the flag. MCP insight-query recording remains a separate follow-up.

Turning the flag off restores the previous selector. This PR does not change Redis registration, the 500-candidate batch, warming frequency or failure backoff.

How did you test this code?

Tests cover dashboard-only demand, a subsequent standalone view, flag-off fallback, unattributed history, source handling and the seven-day cutoff. The policy and tests are moved from #106556 without intentional behavior changes. Current-head CI and human review remain required. No flag enablement or production savings are claimed.

After the merge with #106556 and master, the warming test file passes locally and mypy is clean on warming.py. A new test covers the shared-only selector with the flag on: it needs an enabled sharing configuration and a standalone view inside the three-day shared window.

The focused run completed all test assertions, then Python segfaulted during shutdown. A clean process exit is not claimed.

Release status

  • No feature flag controls this change
  • This change is behind a feature flag and is not available to users
  • This change makes a previously flagged feature available to everyone

Automatic notifications

  • Publish to changelog?

Docs update

Internal rollout instructions are above. No visible UI change.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Agent: Codex, GPT-6

Skills: posthog-pr-finish, brainstorming, writing-pr-descriptions. Tools: Git, gh, Flox, pytest. Extracted the existing policy into a separate deployable PR. No customer data or operational metrics are included. No new third-party review or manual UI validation is claimed.

@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

🤖 CI report

⚠️ Trunk lane — backend Python lane

This PR is assigned to the backend Python lane. It runs backend Python tests and may merge in parallel with PRs in other lanes.

✅ Duplication (Python) — clean

New Python code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Duplication (TypeScript) — clean

New TypeScript code duplication introduced by this branch. Fails at 70+ tokens in app code, or 150+ tokens when both copies live in test files. Advisory while the gate proves itself: extract a shared helper instead of copying.

✅ Playwright — all passed

All tests passed.

View test results →

✅ Django migration risk — migration analysis complete

We've analyzed your migrations for potential risks.

Summary: 1 Safe | 1 Needs Review | 0 Blocked

⚠️ Needs Review

May have performance impact

product_analytics.0017_remove_legacy_insight_view_uniqueness
  └─ #1 ⚠️ RunPython: RunPython data migration needs review for performance
  └─ #2 ⚠️ RemoveConstraint: Unscored Django operation: RemoveConstraint (needs manual review)

✅ Safe

Brief or no lock, backwards compatible

product_analytics.0016_insight_viewed_context_uniqueness
  └─ #1 ✅ SeparateDatabaseAndState
     Wrapper operation - see nested operations for risk: CreateIndexConcurrently
     database_operations: CreateIndexConcurrently
     └─ #2 ✅ CreateIndexConcurrently
        PostHog concurrent-index helper: idempotent (timeout disabling + invalid-leftover recovery)
        model: None, index: insightviewed_context_unique
  └─ #3 ✅ DropIndexConcurrently
     PostHog concurrent-index helper: idempotent (timeout disabling + invalid-leftover recovery)
     model: None, index: posthog_insightviewed_null_team_user_unique

📚 How to Deploy These Changes Safely

RunPython:

Use batching for large data migrations:

  • Use .iterator() to avoid loading all rows into memory
  • Use .bulk_update() instead of saving individual objects
  • Batch size: 1,000-10,000 rows per batch
  • Add pauses between batches
  • Consider background jobs for very large updates (millions of rows)

See the migration safety guide

Last updated: 2026-11-13 08:44 UTC (f2e097c)

@greptile-apps

greptile-apps Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Retrigger

[Medium risk] Changes cache warming logic for standalone insights.

The selector appears safe to merge once the repository’s module-level import requirement is satisfied.

Reviews (1) · Last reviewed commit: "fix: require standalone demand for stand..."

@coderabbitai

coderabbitai Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Walkthrough

When the team’s cache-warming-standalone-demand feature flag is enabled, standalone insight warming candidates come from recent standalone view contexts with a nonempty source. Otherwise, selection uses the existing insightviewed timestamp cutoff. Tests cover recency boundaries, dashboard attribution, feature-flag behavior, and unattributed views.

Merge Risk: Low · up to e5efa

The warming change has no established behavior defect, but its new tests need two small Python-guideline corrections before merge.

Security architecture risk: Low · up to e5efa

The new rule narrows standalone warming eligibility without adding a public endpoint or expanding access to insights. Disabling the flag restores the previous selection rule. Incomplete view recording could cause eligible queries to miss warming after enablement, so caller coverage matters before rollout.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The changed decision is scoped to existing Redis candidates for a team, not arbitrary insight IDs supplied directly to the warming task. The selector checks team ownership and retains the shared-only restriction.

Trust Boundaries and Controls

  • observed — Source attribution is a selection condition, not independent proof of a viewer’s authority: the recorder accepts a supplied nonempty source after checking insight and dashboard ownership. The changed selector neither exposes that recorder nor removes its team constraints.

Resilience and Maintainability Implications

  • observed — Repeated context writes use an upsert with a one-minute update condition. The selector applies the corresponding recency allowance, while the existing scheduler and warming task retain their separate execution and error handling.

Hardening Proposals

  • proposed — Before enabling the flag, verify attributed recording for each intended saved-insight caller and monitor standalone selection counts during rollout; restore the legacy selector if legitimate demand is under-recorded.
Pre-merge checks: Passed 1

Check name Status Explanation
Description check Passed The description states the problem (dashboard views keep unused standalone insight queries eligible for cache warming) and the change (a default-off flag requiring recent attributed standalone views),…
Full details: Description check

Explanation

The description states the problem (dashboard views keep unused standalone insight queries eligible for cache warming) and the change (a default-off flag requiring recent attributed standalone views), and it lists tests, dependencies, and rollout guidance. It is well-scoped and mostly complete. It exceeds the 150-word target, and the release status section keeps checkbox syntax, which the template says to avoid. The Problem section does not name who is affected or cite a request or issue, and it has no 'Closes' line or issue reference. The 'Automatic notifications' section is not in the template. These are non-critical gaps, so the check still passes.

Finishing Touches

Fix failing CI checks

  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Comment thread posthog/caching/test/test_warming.py Resolved

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
posthog/caching/test/test_warming.py (1)

61-61: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Move record_insight_view_context to the module imports.

The test already imports the same facade module at module scope. The local import does not avoid a circular, heavy, or optional dependency.

Suggested fix
-from products.product_analytics.backend.facade.api import record_insight_views
+from products.product_analytics.backend.facade.api import record_insight_view_context, record_insight_views
...
-        from products.product_analytics.backend.facade.api import record_insight_view_context

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: PostHog/posthog/.coderabbit.yaml

Review profile: QUIET

Plan: Enterprise

Run ID: 7bcea4e5-16ad-4423-b4eb-bfe7ee2086a4

📥 Commits

Reviewing files that changed from the base of the PR and between 577e5ea and e5efa3d.

📒 Files selected for processing (2)
  • posthog/caching/test/test_warming.py
  • posthog/caching/warming.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

Comment thread posthog/caching/test/test_warming.py Resolved
@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.

@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

@scheduled-actions-posthog

Copy link
Copy Markdown
Contributor

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.

…g-standalone-demand-policy

Adapts the standalone demand filter to the facade read function, which
now returns view rows instead of a queryset.
…g-standalone-demand-policy

The standalone demand filter now asks the view history only for views
inside the warming window.
@vdekrijger
vdekrijger marked this pull request as ready for review November 13, 2026 07:55
@stamphog
stamphog Bot requested a review from a team November 13, 2026 07:55
@parameterai

parameterai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Risk: No findings

This change makes standalone cache warming require recently recorded standalone insight views, behind the default-off cache-warming-standalone-demand flag, and pulls in master's dashboard-placement, tile-override, and time-of-day read-history helpers that the selector now calls. I traced every new read path for team scoping and found no security issue: all insight lookups are scoped by insight__team_id, dashboard/tile lookups consume only the team's own Redis freshness identifiers with documented exceptions, and the unattributed-history exclusion in insights_viewed_standalone is exact because source is non-nullable with an empty-string default.

Sentinel reviewed f2e097c · Review settings

@veria-ai

veria-ai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

PR overview

No security issues were found in this pull request.

Reviews

@posthog posthog Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PostHog Review · Standard · reviewed c2f57d1: 1 Should fix.

See this review and its findings in PostHog Review. Comment @posthog review --deep for a Deep review. A Deep review takes longer and looks deeper than a Standard review.

Comment thread posthog/caching/warming.py Resolved
Asks the view history for the distinct insights viewed outside a dashboard
instead of loading every recent view row into the scheduler. Also moves a
test import to module level and annotates the new test signatures.
@vdekrijger vdekrijger added the stamphog Request AI approval (no full review) label Nov 13, 2026

@stamphog stamphog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

Contained, flag-controlled cache-selection change with regression coverage; no public API, persistence, or authorization behavior changes. The actionable import nit is fixed, and no concrete unresolved substantive concerns or blocking reactions are present.

  • Author wrote 0% of the modified lines and has 10 merged PRs in these paths (familiarity MODERATE).
  • 👍 on the PR from greptile-apps[bot].
Gate mechanics and policy version
Gate Result
prerequisites ✓ all clear
deny-list ✓ no deny categories matched
size ✓ 49L, 3F substantive, 126L/4F incl. docs/generated/snapshots — within ceiling
tier ✓ T1-agent / T1c-medium (126L, 4F, two-areas, fix)
stamphog 2.8.0 .stamphog/policy.yml @ f2e097c · reviewed head f2e097c

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stamphog Request AI approval (no full review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant