Skip to content

chore(signals): make the billing scout lean on the official billing and cost skills #138868

Open

Description

@andrewm4894

Problem

signals-scout-billing (#137999) watches an organization's own PostHog bill. It finds a usage step or a spend ramp, prices it, and hands the root cause to another scout. It makes little use of the official skills that already cover the same ground.

  • It names one official skill. SKILL.md ends with a pointer to posthog:understanding-billing-usage, to read "when a drilldown needs more than the routing table". Nothing else is named.
  • It does not know posthog:managing-self-driving-spend (feat(signals): add a skill for managing self-driving spend #124825). For Self-driving credits, the handoff table in references/usage-types.md says only "The Signals inbox — which includes this scout. Surface the loop once, plainly". Every other meter gets an owner and a drilldown. This one gets neither, so a report on Self-driving spend states the projection and stops. The spend skill already breaks that spend down by signal source, scout and day, and ranks the controls that lower it.
  • It reaches the AI cost skills only by accident. posthog:exploring-llm-costs and posthog:analyzing-expensive-users are named in understanding-billing-usage/references/usage-type-routing.md, not in the scout. The scout's own row for PostHog AI credits says no sibling scout owns the meter and names no skill.
  • It carries a second copy of the usage type table. signals-scout-billing/references/usage-types.md has a vocabulary table and a handoff table keyed by usage type. understanding-billing-usage/references/usage-type-routing.md has a routing table keyed by the same usage types, with the follow-up skill for each. A new meter must be added in both, and the two can drift.
  • It has no discovery step. The skill does not tell the scout to look for a matching skill before it writes its own drilldown.

signals-scout-ai-observability already shows the pattern this scout lacks: a "Deep-dive skills (baked into the sandbox ...)" line and a lens table that names the skill to open for each lens.

Goal

The billing scout owns only what is specific to a scheduled billing watcher. For everything else it reads the official skill and follows it. When an official skill already answers the question a finding raises, the scout acts as a thin wrapper: it supplies the trigger, the pricing and the report, and the skill supplies the investigation.

One fact lives in one place. A change to a usage type, a spend control or a cost query is made once.

Proposal

1. Decide what the scout owns

Keep in the scout:

  • Access gates and the no-access and volume-only modes.
  • Cadence, cursor and scratchpad memory rules, including what must never be stored there.
  • The two lanes: step detection and the trajectory read.
  • The pricing mechanics: period boundaries, the difference-of-two-prices rule, which projection field to quote, and "may, not will" for limits.
  • Report framing and dedupe.

Move out, or replace with a pointer:

  • The usage type vocabulary and the per-meter follow-up, where understanding-billing-usage already has it.
  • Any drilldown query an official skill already carries.

2. Add a skill map to the handoff table

Give each row in "Where each usage type hands off" a third column: the official skill to read when the scout needs more than the one-line hint. Start from the list in usage-type-routing.md. At least:

Meter group Skill to lean on
Self-driving credits posthog:managing-self-driving-spend
PostHog AI credits, AI observability events posthog:exploring-llm-costs, posthog:analyzing-expensive-users
Events, identified events posthog:investigate-metric, posthog:exploring-autocapture-events
Warehouse synced rows posthog:auditing-warehouse-source-health, posthog:tuning-incremental-sync-config
Logs and traces posthog:investigating-logs
Replay Vision credits posthog:exploring-replay-vision-observations
Anything else posthog:understanding-billing-usage and its routing reference

The scout still hands the root cause to the owning scout. The skill is for the cases where no scout owns the meter, or where a short read makes the report useful on its own.

3. Wrapper mode for meters with no owning scout

Self-driving credits and PostHog AI credits have no sibling scout. For these, the scout runs the read-only part of the matching skill and puts the result in the report:

  • Self-driving credits: the breakdown by signal source, scout and day from managing-self-driving-spend, and its ranked list of controls as recommendations.
  • PostHog AI credits: the breakdown from exploring-llm-costs.

The scout never applies a control. managing-self-driving-spend changes settings only after a person confirms each one, and a scheduled run has no person to ask. The report lists the recommended changes and names the skill, so the reader can run it and confirm.

4. Remove the duplicate table

Pick one source for the usage type list. Two options:

  • A. understanding-billing-usage/references/usage-type-routing.md is the source. The scout's reference keeps only the billing-service field mechanics and a short table of scout handoffs, and tells the scout to read the routing reference for the rest.
  • B. Keep both files, and add a test that fails when the set of usage types differs between them.

A is the better end state. B is a cheap guard if A needs a harness change first.

5. Add the reverse links

  • understanding-billing-usage/references/usage-type-routing.md names posthog:managing-self-driving-spend for the Self-driving meters. Today it sends the reader to posthog:inbox-exploration only.
  • managing-self-driving-spend says the billing scout can raise the spend finding that leads to it.

6. Add a discovery rule

One line in the scout's "Seams" section: before the scout writes a drilldown that is not in its own skill, it checks for an official skill that covers it and reads that skill.

Open questions

  • Which official skills are available inside a scout run today, and by what route (baked into the sandbox, or loaded through the MCP)? The scout's skill says understanding-billing-usage is baked in. This needs a check for the others before the map is written.
  • Does a scout run have the scopes for inbox-usage-get and billing-limits-get, which managing-self-driving-spend starts from? If not, wrapper mode for Self-driving credits needs a grant first.
  • inbox-usage-get reports recorded usage, which includes exempt work and is not an invoice. The report must not mix that number with the billed figure from the billing tools. Which one leads?
  • Does every pointer need a fallback line for a team that has edited or removed its copy of the scout skill, or for a run where the skill is not available?
  • Should the same audit run across the other canonical scouts? chore(signals): refresh canonical scouts against current harness capabilities #103304 refreshed them against harness capabilities, not against the official skills.

Acceptance

  • The billing scout's skill names every official skill it depends on, per meter group.
  • A finding on Self-driving credits produces a report with a breakdown by source and scout, and a list of recommended controls.
  • The usage type list has one source, or a test that keeps two in step.
  • No drilldown query in the scout repeats one in an official skill.

Related: #137999 (the scout), #124825 (the spend skill), #103304 (canonical scout refresh).

Activity

  1. posthog commented on Nov 19, 2026

    @posthog
    Contributor

    PostHog self-driving picked up this issue. Status: Research is done.

    Somebody may already be working on it, so ask before you start a fix.

    Report (team access only): https://us.posthog.com/project/2/inbox/reports/463160c1-cf8b-452d-9420-57e1d416ed41?link_source=github_comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions