feat(learning): browse learned workloads by project and operation

docs/handoffs/learning-browse.md

@@ -0,0 +1,39 @@
+# Learning browse handoff
+
+## Result
+
+Learned workloads now opens on **Projects**, busiest first. Projects with two workloads or fewer share one **Other projects** row. Choosing a project shows one row per operation (tests, build, lint and so on) plus **All operations**. Choosing an operation opens the existing paged group list with selection, details and merge. The phone dashboard has a breadcrumb; the native app has a Back button and breadcrumb buttons. Search always covers every project and shows a flat list.
+
+This builds on the repository grouping and catalog API from the learning grouping change (`docs/handoffs/learning-grouping.md`). That branch is this one's base.
+
+## What changed
+
+- `remote/static/profiles.js`, `remote/static/app.css`, `remote/static/index.html`: the three levels, breadcrumb, focus and announcements, failed-load cleanup.
+- `native/AgentLearning.swift`, `native/AgentQueue.swift`: the same levels in `LearningStore`, the hierarchy transport, and a submitted query kept apart from the search box.
+- `internal/backend/profiles_learning.go`, `learning.go`:
+  - Group project names are compared in NFC.
+  - The CLI normalises its string flags, because Foundation's `Process` passes them decomposed.
+  - Search also matches member project names, ranked after label and id matches.
+- `internal/preview`: fixture projects and the hierarchy flags for screenshots and browser tests.
+
+## Behaviour worth knowing
+
+- When a merge folds a project into Other projects, its groups load directly. Back and the project crumb then return to Projects, because that project has no operations level.
+- A page past the end after a merge re-reads the last page once.
+- The overview lists at most 100 projects. Search reaches groups behind that limit.
+
+## Known limits
+
+- The `folded` flag is set only on the way into the operations level. If a project regains its own row while you stay on its groups, Back still goes to Projects. If a merge folds it while you are on a group list, the first Back lands on its All operations list. Each costs one extra click; no data is wrong.
+- The HTTP handler checks the 120-character limit before NFC normalisation. A direct API client sending a long decomposed name gets a 400. The web client is not affected.
+- No browser test covers the case where the re-read of the last page is also empty.
+
+## Evidence not covered
+
+- The flaky "Cannot establish process identity" failures described in the grouping handoff were not seen in this branch's runs and remain undiagnosed.
+
+## Next steps
+
+1. Merge the measurement fix (#78), then the grouping change (#79), and retarget this PR to `main` so hosted CI runs.
+2. After merge, open Learned workloads on the phone dashboard and the native app and check the project counts against `agent-heavy profiles list --hierarchy --json`.
+3. Optional later work: tool and memory filters, a `--kind` flag, and a look at the 64 to 66 GiB `rewrite` peaks before switching to Adaptive mode.

docs/learning.html

@@ -1,4 +1,4 @@
-<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>learning</title><style>:root{color-scheme:light dark;font:16px/1.65 system-ui,sans-serif}body{max-width:76ch;margin:32px auto;padding:0 20px;overflow-wrap:anywhere}h1{line-height:1.2;font-size:1.8rem}h2{font-size:1.25rem;margin-top:2em}li{margin:.7em 0}code{font-size:.9em}pre{overflow:auto;padding:16px;border:1px solid #8886;border-radius:8px}img{max-width:100%;display:block;margin:20px auto;border-radius:8px}a{color:light-dark(#006b60,#78d7bf)}@media print{body{max-width:none;margin:0}}</style></head><body><main><h1>Inspect and correct workload learning</h1><h2>Use</h2><p>Choose <strong>Learned workloads</strong> from the native app&#x27;s gear menu or the phone dashboard navigation. Search by name, select a workload, and inspect its measured peaks, successful and incomplete run counts, variants and recent numeric measurements. A job&#x27;s detail view links to its current learning profile.</p><p>Give a profile a useful name and save minimum RAM or CPU reservations when you know a job needs more headroom. Blank minimums return to automatic estimation. Sharing learning combines the original families under the selected target profile, keeping the larger reservations from both profiles. Separate a family when it should learn independently. Reset measurements only when the old evidence is obsolete; reset removes those measurements permanently while preserving your corrections.</p><p>Related workloads now share learning automatically within the same verified git repository, whatever subdirectory or linked worktree they ran in. Grouping needs a recognized tool (Go, JavaScript, Python, shell, Cargo, Swift or Make). Each workload also has an operation: tests, browser tests, native tests, race tests, full tests, build, lint, type check, dependencies, benchmark or combined checks, and <code>other</code> for mixed batches and anything unknown. Workloads with <code>other</code> share one group per repository and tool, so one heavy script raises the conservative estimate for the rest of that group. Simple <code>env</code> and <code>fnm exec</code> wrappers, <code>*.sh</code> scripts and inline <code>sh -c</code> or <code>bash -c</code> programs are recognized for new jobs. Opaque scripts can use their safe task descriptions. Unrecognized tools and workloads outside a verified git repository remain independent. Project names alone never establish a repository match. Only the category is stored; command arguments and inline program text are never retained. Automatic groups retain the largest measured peaks rather than averaging dissimilar runs down.</p><p>Group estimates deliberately have no exact-variant successful-run count: they describe the conservative envelope a new variant can borrow. Each actual job still has its own prediction and confidence. Measured history, the group estimate and operator minimums are displayed separately. Run history and variant peaks can include multiple machines; each run identifies its machine with an opaque hash, while the group estimate uses the current machine only.</p><p>The CLI has the same inspection and correction behavior:</p><pre><code>agent-heavy profiles list --json
+<!doctype html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><title>learning</title><style>:root{color-scheme:light dark;font:16px/1.65 system-ui,sans-serif}body{max-width:76ch;margin:32px auto;padding:0 20px;overflow-wrap:anywhere}h1{line-height:1.2;font-size:1.8rem}h2{font-size:1.25rem;margin-top:2em}li{margin:.7em 0}code{font-size:.9em}pre{overflow:auto;padding:16px;border:1px solid #8886;border-radius:8px}img{max-width:100%;display:block;margin:20px auto;border-radius:8px}a{color:light-dark(#006b60,#78d7bf)}@media print{body{max-width:none;margin:0}}</style></head><body><main><h1>Inspect and correct workload learning</h1><h2>Use</h2><p>Choose <strong>Learned workloads</strong> from the native app&#x27;s gear menu or the phone dashboard navigation. Search by name, select a workload, and inspect its measured peaks, successful and incomplete run counts, variants and recent numeric measurements. A job&#x27;s detail view links to its current learning profile.</p><p>The list opens on <strong>Projects</strong>, busiest first, each with its workload, group and run counts. Projects with two workloads or fewer share one <strong>Other projects</strong> row, and a note appears when more projects exist than the overview can list, saying how many are not shown; searching a project or group name finds them. Choose a project to see what it ran, one row per operation such as tests, build or lint with the same counts, plus <strong>All operations</strong>. Choose an operation to reach the group list, which has the selection checkboxes, group details and paging described below. In the phone dashboard a breadcrumb such as <strong>All projects / Sample app / Tests</strong> goes back to any earlier level; the native app has the same trail, with earlier parts as buttons, and a <strong>Back</strong> button. Search always covers every project, matching group names and the names of the projects behind them, and shows a flat list with groups whose own name matches before those that match only through a project; clear the box and search again to return to the level you were on. Returning from a group&#x27;s details lands on the level and page it came from. Counts reload when you go back up a level or refresh, a project that a merge folded into <strong>Other projects</strong> still lists its groups and Back from them returns to the projects, a page past the end after a merge shows the last page, and a project or operation with nothing left shows an empty state with a way back.</p><p>Give a profile a useful name and save minimum RAM or CPU reservations when you know a job needs more headroom. Blank minimums return to automatic estimation. Sharing learning combines the original families under the selected target profile, keeping the larger reservations from both profiles. Separate a family when it should learn independently. Reset measurements only when the old evidence is obsolete; reset removes those measurements permanently while preserving your corrections.</p><p>Related workloads now share learning automatically within the same verified git repository, whatever subdirectory or linked worktree they ran in. Grouping needs a recognized tool (Go, JavaScript, Python, shell, Cargo, Swift or Make). Each workload also has an operation: tests, browser tests, native tests, race tests, full tests, build, lint, type check, dependencies, benchmark or combined checks, and <code>other</code> for mixed batches and anything unknown. Workloads with <code>other</code> share one group per repository and tool, so one heavy script raises the conservative estimate for the rest of that group. Simple <code>env</code> and <code>fnm exec</code> wrappers, <code>*.sh</code> scripts and inline <code>sh -c</code> or <code>bash -c</code> programs are recognized for new jobs. Opaque scripts can use their safe task descriptions. Unrecognized tools and workloads outside a verified git repository remain independent. Project names alone never establish a repository match. Only the category is stored; command arguments and inline program text are never retained. Automatic groups retain the largest measured peaks rather than averaging dissimilar runs down.</p><p>Group estimates deliberately have no exact-variant successful-run count: they describe the conservative envelope a new variant can borrow. Each actual job still has its own prediction and confidence. Measured history, the group estimate and operator minimums are displayed separately. Run history and variant peaks can include multiple machines; each run identifies its machine with an opaque hash, while the group estimate uses the current machine only.</p><p>The CLI has the same inspection and correction behavior:</p><pre><code>agent-heavy profiles list --json
 agent-heavy profiles list --query Tests --offset 0 --limit 100 --json
 agent-heavy profiles list --project 'Agent Queue' --operation tests --json
 agent-heavy profiles list --other-projects --json

docs/learning.md

@@ -4,6 +4,8 @@
 
 Choose **Learned workloads** from the native app's gear menu or the phone dashboard navigation. Search by name, select a workload, and inspect its measured peaks, successful and incomplete run counts, variants and recent numeric measurements. A job's detail view links to its current learning profile.
 
+The list opens on **Projects**, busiest first, each with its workload, group and run counts. Projects with two workloads or fewer share one **Other projects** row, and a note appears when more projects exist than the overview can list, saying how many are not shown; searching a project or group name finds them. Choose a project to see what it ran, one row per operation such as tests, build or lint with the same counts, plus **All operations**. Choose an operation to reach the group list, which has the selection checkboxes, group details and paging described below. In the phone dashboard a breadcrumb such as **All projects / Sample app / Tests** goes back to any earlier level; the native app has the same trail, with earlier parts as buttons, and a **Back** button. Search always covers every project, matching group names and the names of the projects behind them, and shows a flat list with groups whose own name matches before those that match only through a project; clear the box and search again to return to the level you were on. Returning from a group's details lands on the level and page it came from. Counts reload when you go back up a level or refresh, a project that a merge folded into **Other projects** still lists its groups and Back from them returns to the projects, a page past the end after a merge shows the last page, and a project or operation with nothing left shows an empty state with a way back.
+
 Open **Group members** to compare each workload's successful and incomplete run counts, peaks and last run. Expand a member to load its own variants and recent measurements, including runs too old to appear in the group's recent list. Search and page within the group to find members beyond the first 256.
 
 For a mismatch, choose **Ungroup workload…** inside its expanded details and confirm. All of that workload's variants and retained measurements move together into an independent profile. Its minimum reservations are preserved. The separated workload and remaining group stay excluded from automatic grouping, so another learning pass cannot undo your correction. Use **Back to group** to continue reviewing. You can explicitly share their learning again later.

internal/backend/learning.go

@@ -11,6 +11,8 @@ import (
 	"strconv"
 	"strings"
 	"unicode/utf8"
+
+	"golang.org/x/text/unicode/norm"
 )
 
 const maxLearningInput = 16384
@@ -395,6 +397,7 @@ func LearningCLI(gate *Gate, args []string, input io.Reader, output, errors io.W
 					}
 					value = args[i]
 				}
+				value = norm.NFC.String(value)
 				switch key {
 				case "--family":
 					if e := profileID(value); e != nil {
@@ -447,6 +450,7 @@ func LearningCLI(gate *Gate, args []string, input io.Reader, output, errors io.W
 					}
 					value = args[i]
 				}
+				value = norm.NFC.String(value)
 				switch key {
 				case "--offset", "--limit":
 					n, e := strconv.Atoi(value)

internal/backend/profiles_classification_test.go

@@ -13,6 +13,8 @@ import (
 	"strings"
 	"testing"
 	"time"
+
+	"golang.org/x/text/unicode/norm"
 )
 
 func runGit(t *testing.T, directory string, args ...string) {
@@ -837,9 +839,46 @@ func TestProfileCatalogFiltersAndHierarchy(t *testing.T) {
 	if r, e := store.CatalogFiltered(0, 100, "build", ProfileFilter{Project: "big"}); e != nil || metric(r["total"]) != 1 {
 		t.Fatal("Search and filter do not combine", r, e)
 	}
+	// Search matches the member project name, case-insensitively, so groups of a project missing from the truncated list stay reachable.
+	if r, e := store.CatalogFiltered(0, 100, "SINGLE", ProfileFilter{}); e != nil || metric(r["total"]) != 1 || r["profiles"].([]Object)[0]["project"] != "single" || strings.Contains(strings.ToLower(text(r["profiles"].([]Object)[0]["label"])), "single") {
+		t.Fatal("Search does not match the project name", r, e)
+	}
 	if limited, _ := store.Hierarchy(1); len(limited["projects"].([]Object)) != 1 || metric(limited["total"]) != 1 {
 		t.Fatal("The hierarchy limit counts the other-projects row", limited)
 	}
+	// Foundation's Process passes arguments decomposed (NFD); a precomposed stored name must still be found, and the reverse.
+	add("cafe one", "caf\u00e9", "Go tests", "go", "cafe")
+	add("cafe two", "caf\u00e9", "Go tests", "go", "cafe")
+	add("cafe three", "caf\u00e9", "Go build", "go", "cafe")
+	if r, e := store.CatalogFiltered(0, 100, "", ProfileFilter{Project: "caf\u00e9"}); e != nil || metric(r["total"]) == 0 {
+		t.Fatalf("Project was not found: %v %v", r, e)
+	}
+	if r, e := store.CatalogFiltered(0, 100, "CAFE\u0301", ProfileFilter{}); e != nil || metric(r["total"]) == 0 {
+		t.Fatal("A decomposed search does not match a precomposed project", r, e)
+	}
+	// The same project stored in both forms is one project: one row, one count, and no stray group in the other projects.
+	add("nfc one", "caf\u00e9", "Go tests", "go", "cafe-nfc")
+	add("nfd one", "cafe\u0301", "Go tests", "go", "cafe-nfd")
+	add("nfd two", "cafe\u0301", "Go build", "go", "cafe-nfd")
+	hierarchy, _ = store.Hierarchy(100)
+	rows = hierarchy["projects"].([]Object)
+	cafeRows := 0
+	for _, row := range rows {
+		if norm.NFC.String(text(row["project"])) == "caf\u00e9" {
+			cafeRows++
+			if row["project"] != "caf\u00e9" || metric(row["groupCount"]) != metric(list(ProfileFilter{Project: "caf\u00e9"})["total"]) {
+				t.Fatal("The project row does not match the project filter", row)
+			}
+		}
+	}
+	if cafeRows != 1 {
+		t.Fatalf("A project stored in two forms has %d rows", cafeRows)
+	}
+	for _, p := range list(ProfileFilter{OtherProjects: true})["profiles"].([]Object) {
+		if norm.NFC.String(text(p["project"])) == "caf\u00e9" {
+			t.Fatal("A named project also appears under other projects", p)
+		}
+	}
 	for _, bad := range []ProfileFilter{{Operation: "typo"}, {Project: strings.Repeat("x", 121)}, {Project: "a\nb"}, {Project: " pad"}, {Project: "big", OtherProjects: true}} {
 		if _, e := store.CatalogFiltered(0, 100, "", bad); e == nil {
 			t.Errorf("Invalid filter accepted: %+v", bad)
@@ -852,6 +891,21 @@ func TestProfileCatalogFiltersAndHierarchy(t *testing.T) {
 	}
 }
 
+func TestProfileSearchRanksOwnLabelBeforeMemberProject(t *testing.T) {
+	store := fixtureProfileStore(t)
+	// Sorted by label alone, the project-only match ("Aardvark") would come first.
+	observeClassified(t, store, "project only", "zebra", "Aardvark", "go", "aardvark")
+	observeClassified(t, store, "label match", "other", "Zebra crossing", "go", "crossing")
+	r, e := store.CatalogFiltered(0, 100, "zebra", ProfileFilter{})
+	if e != nil {
+		t.Fatal(e)
+	}
+	entries := r["profiles"].([]Object)
+	if len(entries) != 2 || text(entries[0]["label"]) != "Zebra crossing" || text(entries[1]["label"]) != "Aardvark" {
+		t.Fatal("A label match does not sort before a project-only match", entries)
+	}
+}
+
 func TestProfileCLICatalogFilters(t *testing.T) {
 	g := testGate(t)
 	store, e := NewProfileStore(g.State, false)
@@ -896,6 +950,30 @@ func TestProfileCLICatalogFilters(t *testing.T) {
 	}
 }
 
+func TestProfileCLINormalizesDecomposedFilters(t *testing.T) {
+	g := testGate(t)
+	store, e := NewProfileStore(g.State, false)
+	if e != nil {
+		t.Fatal(e)
+	}
+	run := func(args ...string) (int, Object) {
+		var output, stderr bytes.Buffer
+		code := LearningCLI(g, append([]string{"list", "--json"}, args...), strings.NewReader(""), &output, &stderr)
+		result := Object{}
+		_ = json.Unmarshal(output.Bytes(), &result)
+		return code, result
+	}
+	// Foundation hands the project over decomposed; the stored name is precomposed.
+	observeClassified(t, store, "accent", "caf\u00e9", "Go tests", "go", "accent")
+	if code, r := run("--project", "cafe\u0301"); code != 0 || metric(r["total"]) != 1 {
+		t.Fatal("A decomposed --project missed a precomposed project", code, r)
+	}
+	// Decomposed text is normalized before the length check: 80 characters are 160 runes in NFD.
+	if code, r := run("--project", strings.Repeat("e\u0301", 80)); code != 0 || metric(r["total"]) != 0 {
+		t.Fatal("A long decomposed project name was rejected", code, r)
+	}
+}
+
 func TestProfilePreviewMatchesAppliedCatalog(t *testing.T) {
 	g := testGate(t)
 	store, e := NewProfileStore(g.State, false)