Skip to content

feat(research): ship the study category set to the web UI - #1400

Open
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/research-webui-category-preset
Open

feat(research): ship the study category set to the web UI#1400
TimeToBuildBob wants to merge 2 commits into
ActivityWatch:masterfrom
TimeToBuildBob:feat/research-webui-category-preset

Conversation

@TimeToBuildBob

Copy link
Copy Markdown
Contributor

Problem

The Research Edition watcher rewrites app to a study category before the event is stored. aw-webui categorises client-side using its own default regexes and never sees the watcher's map. So the Categories panel reads Uncategorized while Top Applications shows the correct study categories. The data is right; the UI disagrees with it.

This is what the Lund/IIIEE study reported on v0.14.0b3-research — twice, on 13 and 17 August. It reads as "the build is broken" to a researcher, and no amount of watcher-side fixing changes it.

#1397 makes the stored data correct. It does not make the Categories panel agree, because that panel never consults the watcher config.

What this adds

ActivityWatch/aw-webui#936 adds the consumer: a build may supply a preset category set via AW_PRESET_CATEGORY_SETS, activated by default only when the user has no stored categorization (so existing users keep their own categories). This PR wires the producer.

scripts/emit_research_category_preset.py derives the preset from the same CATEGORY_MAP and APP_CATEGORY_MAP that patch_research_edition_config.py injects into the watcher. One source of truth — the UI's categories cannot drift from the watcher's, which is the failure this whole class of bug comes from.

The release workflow emits it beside the existing patch step, for both the Qt and Tauri research paths.

Design notes

  • Rules match the category name anchored end to end (^Video\ Streaming$), because by the time aw-webui sees an event, app is the category name.
  • Names are regex-escaped. Several contain &, / or -Sensitive / Excluded, Shopping - Groceries & Food, AI Chatbots & Assistants. An unescaped name is a silently broken rule, which would look exactly like the bug being fixed.
  • Output is sorted and compact, so the same map always yields a byte-identical preset and the value survives $GITHUB_ENV (which terminates at a newline).
  • 18 labels: Matthias's 17 study categories plus the Excluded residual.

Verification

5 tests: preset covers exactly the watcher map's categories; every rule matches its own name and neither an unrelated app nor a superstring; metacharacter escaping is asserted explicitly; output is stable across runs; serialisation is newline-free and round-trips.

Deliberately in its own test file so it does not conflict with #1398, which rewrites the patch-script tests.

Dependencies and ordering

Depends on ActivityWatch/aw-webui#936 plus the aw-webui submodule bump that carries it. Until then the variable is simply unread and builds are unchanged — so this is safe to merge in any order.

Related: #1397 (app map), #1398 (keeps the patch script working past aw-watcher-window#137), aw-webui#936 (the consumer).

The Research Edition watcher rewrites `app` to a study category before the
event is stored. aw-webui, however, categorises client-side using its own
default regexes and never sees the watcher's map -- so the Categories panel
reads "Uncategorized" while Top Applications shows the correct categories.
The data is right and the UI disagrees with it.

That is exactly what the Lund study reported on v0.14.0b3-research, twice. The
app-map fix in ActivityWatch#1397 makes the stored data correct; it does not make the
Categories panel agree, because that panel never consults the watcher config.

aw-webui#936 adds the missing half: a build can supply a preset category set
via AW_PRESET_CATEGORY_SETS, activated by default only when the user has no
stored categorization. This wires the producer.

scripts/emit_research_category_preset.py derives the preset from the same
CATEGORY_MAP and APP_CATEGORY_MAP the watcher patch uses, so the UI's
categories cannot drift from the watcher's. Rules match the category name
anchored end to end, because by the time aw-webui sees an event, `app` IS the
category name. Names are regex-escaped -- several contain '&', '/' or '-'.

The release workflow emits it next to the existing patch step, for both the Qt
and Tauri research paths, and exports it as a single-line value into
$GITHUB_ENV for the build.

Depends on ActivityWatch/aw-webui#936 and the aw-webui submodule bump that
carries it; until then the variable is simply unread and builds are unchanged.
@greptile-apps

greptile-apps Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR generates an aw-webui category preset from the Research Edition watcher’s existing category maps and exports it at build time for both Qt and Tauri packages.

  • Adds deterministic, compact preset generation with anchored and escaped regular expressions.
  • Exports AW_PRESET_CATEGORY_SETS before both Research Edition build paths.
  • Adds tests for map coverage, matching behavior, escaping, determinism, and serialization.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking defects identified.

Both Research Edition build paths export the preset before building, and the generator consistently derives valid, escaped, deterministic rules from the current watcher maps.

Important Files Changed

Filename Overview
.github/workflows/release.yml Exports the generated category-set JSON through GITHUB_ENV immediately before both Research Edition build paths.
scripts/emit_research_category_preset.py Loads the watcher category maps and emits a sorted, compact preset containing escaped, whole-value regex rules.
scripts/tests/test_emit_research_category_preset.py Verifies map coverage, regex behavior, metacharacter escaping, deterministic generation, and newline-free JSON serialization.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Maps["CATEGORY_MAP + APP_CATEGORY_MAP"] --> Emitter["emit_research_category_preset.py"]
  Emitter --> Env["AW_PRESET_CATEGORY_SETS"]
  Env --> Qt["Qt Research build"]
  Env --> Tauri["Tauri Research build"]
  Qt --> WebUI["aw-webui category preset"]
  Tauri --> WebUI
Loading

Reviews (1): Last reviewed commit: "feat(research): ship the study category ..." | Re-trigger Greptile

@TimeToBuildBob

Copy link
Copy Markdown
Contributor Author

Automated placeholder: work was recorded for this monitoring item but the worker finished before posting the thread update. See evidence below for what changed — the delivery guard is NOT closing the loop, just preventing silent orphaning.

Evidence: commit: 028e5590e1 chore(task): record the full RE merge chain and the real acceptance check

Session: cc02824a-f516-5bc1-85ce-70ebd0ab74f2

The preset regexes were built with Python's `re.escape`, which escapes space
as `\ ` and ampersand as `\&`. Both are valid in Python and both are
*invalid identity escapes* in JavaScript unicode-mode regex: `new RegExp(r,
"u")` throws `Invalid escape` on 14 of the 18 study categories, including
every name containing " & ".

They compile today only because aw-webui builds the regex without the `u`
flag. Depending on that leniency is the wrong bet -- adding the flag later
would break all 14 at once, and the symptom would be the Categories panel
showing nothing, which is exactly the bug this preset exists to fix and
therefore the hardest kind to attribute.

Escape only the metacharacters both engines agree on outside a character class.
For the current 18 names that is a no-op (none contain a metacharacter), so the
emitted patterns become plain readable literals like `^Sensitive / Excluded$`,
while a future name containing `.` or `(` is still escaped rather than
silently becoming a wildcard.

Verified in node: 18/18 compile and match with no flag and with the `u` flag,
and still reject superstrings. Mirrors the cross-engine class Greptile caught on
the consumer side in ActivityWatch/aw-webui#936, where JS-only regexes were
reaching the Python query engine.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant