Repository navigation
sync: merge upstream main through 52b52f7 - #31
Merged
Merged
Conversation
A path-scoped resolution outside the project root returned an empty chain. The runtime replaces the active chain on success, so an out-of-root Read, Write, Edit, or BrowserPreview call silently dropped the project instructions until a later in-project resolution. The root path itself had the same effect because dirname(root) lies outside the root. Resolve both cases to the root chain. Instruction files remain confined to the project root, the shared 32 KiB budget is unchanged, and no instructions are fabricated when the root has no instruction file. fixes vastsa#968
Three costs made the host slower the more history a workspace held. Each was measured before it was changed, and again with the change ablated. A transcript append synced the device inside the single `Mutex<AppState>` that serialises the RPC surface, so one line's device latency -- 1787 us measured on this volume (100/1000/5000 appends, stdev/median < 2%) -- serialised every other request. The line still reaches the kernel synchronously; what moved is the device flush. It is owed by the request that caused it: handlers record the files they wrote in a task-local, and `commit_writes_of` flushes them with the lock released, before that request answers. A request whose flush cannot finish fails instead of answering success, and a caller outside a request still flushes inline, so the contract is unchanged: a successful answer means the line is on the device. The same commit moves checkpoint writes into that scope and discards a checkpoint a later decision overtook. Under the four-core load harness, appends went from 140/s at p50 11.64 ms to 343/s at p50 2.03 ms. The workspace resolvers read a whole transcript to learn one project path, from four call sites, one of them twice per call. The same index row answers both questions: 202x faster at 21 MB, and constant in history instead of linear. Every search statement ranks by `created_at` and cuts with a LIMIT, and no index carried that column, so the planner materialised and sorted its whole match set before the first row. The ordering indexes arrive the way `idx_turns_ended_at` arrives: an idempotent `CREATE INDEX IF NOT EXISTS` in `boot_maintenance` plus the schema declaration, no row rewritten and no migration backup owed. Ablated, the uncapped LIKE branch goes from 0.021 ms to 30.46 ms for the first 20 of 40k matches, the per-session snippet lookup from 0.028 ms to 0.183 ms, and hoisting `snippet()` out of the FTS row loop takes the trigram branch from 6.39 ms to 3.52 ms. That last replacement had to repeat `MATCH ?1` in its subquery: an FTS5 auxiliary function reads the phrase list of the MATCH that positioned its cursor, so reaching a document by rowid alone returns the head of the document instead of the window around the hit. The faster form was rejected after a row-for-row comparison found 0 of 20 snippets identical; the shipped form is 20 of 20. Two presence questions stopped parsing whole transcripts to answer themselves and now sit at the file read instead of above it (1.7x and 1.38x). The desktop stopped walking a search query once per emitted page: page `offset / 30` cost `offset / 30 + 1` host calls, each paying the same whole-table aggregation because `OFFSET` does not change it. Six pages of 40k messages came to 11 calls and 27.4 ms against 6 calls and 19.0 ms for the same rows. A hover-card existence check asked for a whole transcript to test a boolean; a one-message window answers it 323x faster and carries 66 KB instead of 21 MB. No output, ordering or protocol shape changes, and no durability regression: a request that cannot flush its own lines answers with an error, which the transcripts guard by test. `update_message` keeps its whole-file rewrite, because read-path windows are physical line coordinates and appending the replacement would open a search jump to a replaced message on the tail of the session instead of its neighbourhood; appending it is 396x cheaper and was rejected for that reason.
An updated message used to rewrite the whole transcript through a temporary file: read O(H), write O(H), fdatasync, rename. The reader decides a message by the last occurrence of its id, so when that occurrence is already the file's last line the replacement can simply be appended -- same id, same content, same position -- and only the tail is read. A session holding hundreds of MB paid that copy for every updated message; 4.5 ms at 1.3 MB, 27 ms at 21 MB, about 1.3 s at 1 GB by the same arithmetic. The layout cache is invalidated only on the rewrite path now: an append only grows the file, which the incremental refresh already handles, and the unconditional invalidation made the next read of a long session scan all of it again. The change was first written with a `(id, session_id)` index for those lookups, and the next commit removes it again: `messages.id` is already UNIQUE, so EXPLAIN QUERY PLAN reports `SEARCH messages USING INDEX sqlite_autoindex_messages_1 (id=?)` for the same statement with or without it, and the extra index only cost write amplification. The replacement still rewrites the whole file when the target's last copy is not the file's last line, which is what the next commit addresses.
…ranscript Whether a message is still a provisional, streaming assistant row is asked twice per turn, and it was answered by walking the transcript backwards 64 message lines at a time until the id turned up. The cost is the distance from the tail: on a 104 MB session, 4.6 ms when the row is last and 42.0 ms when it sits 4096 lines back. Every window that missed it opened the file again, and the guard also refreshed the transcript's line layout, so a rewrite by an earlier turn turned into a full rescan of the file on the next checkpoint. The `messages` index row already mirrors the transcript's last copy of an id: it is inserted with the row, re-stamped by the append that lands the terminal snapshot, and recomputed from the file by a rebuild. So the answer is a column in that row. A NULL column is a row written before it existed: those still answer from the transcript, which is why a database that has not been rebuilt behaves exactly as it did before, and the column arrives through the same idempotent boot maintenance as the ordering indexes -- no schema version bump, no migration backup, no rewritten rows. The index is a cache, so the one place it can be wrong is the place it matters most: a device that lost a settled row's bytes can leave the index claiming the reply is finished while the transcript still holds the provisional row, and the checkpoint is then the durable copy of that reply. Boot recovery therefore decides from the transcript, re-stamps the index to match it, and appends the row back when the transcript lost it entirely -- the two windows the old file read covered, and the only paths that could have regressed. Both are covered by tests that fail when the reconciliation is removed. Measured on the same 118 MB fixture, host pinned to 4 cores: distance to tail checkpoint terminal row 0 lines 4.42 -> 4.64 ms 5.04 -> 5.13 ms 64 lines 4.49 -> 3.95 ms 248.6 -> 228.7 ms 512 lines 32.60 -> 4.16 ms 264.0 -> 245.7 ms 4096 lines 42.03 -> 3.97 ms 280.9 -> 260.7 ms The guard no longer depends on the distance from the tail, and a whole turn costs 20.4 ms on a 10.5 MB session against 22.6 ms on a 127.7 MB one. The terminal row's own cost is deliberately untouched here: when the provisional row is no longer the file's last line, `transcripts::update_message` still rewrites the whole transcript. That is the next hot path, and this change does not claim it. Also drops `idx_messages_id`. EXPLAIN QUERY PLAN reports `SEARCH messages USING INDEX sqlite_autoindex_messages_1 (id=?)` for the same lookups with or without it, because `messages.id` is already UNIQUE, so the extra index only bought write amplification. Tests: the guard answers from the index even with the transcript file deleted, and from the transcript when the column is NULL; the column tracks the transcript's last copy across a replace; a database that predates the column is upgraded in place; and the cached fact is checked against the transcript after every step of a randomized append/settle/retry/recover sequence.
A windowed read merged every compaction line in the transcript, on the stated assumption that the chain is small. A real 406 MB session falsified it: 44 compactions, 2.0 MB. `session.get` with `messageLimit: 1` returned 2014 KB, of which 2012 KB was that chain, and cost 85-103 ms of parse and re-serialization per read -- inside the single `Mutex<AppState>` that serialises the RPC surface, so it stalled every other session's requests with it. A window now carries the divider in force at it plus the dividers inside it. The read with no message window -- the model-context read -- keeps the whole chain, so nothing that reconstructs a prompt changes. Measured on that session, host pinned to 4 cores, warm layout: session.get(messageLimit=1) 2014 KB -> 67 KB 93.6/85.5 ms -> 1.4/1.7 ms session.get(messageLimit=50) 2338 KB -> 391 KB 101.7/103.2 ms -> 6.3/15.9 ms response bytes per read 2.78 MB -> 0.28 MB reads in 2.5 s 28 -> 1505 p50 74.03 ms -> 1.51 ms The first read after a restart still pays the layout scan of a 406 MB transcript (531 ms for the first read of that run); that is a separate cost, not this one, and it is the next hot path rather than a claim of this commit. Guards that stay green include the compaction lifecycle, the snippet centering and the replaced-message window tests.
The first read of a session in a process built the transcript's line layout by scanning every line: 616.6 ms for a real 406 MB transcript, taken inside the single `Mutex<AppState>` that serialises the RPC surface, so a second window queued behind it waited 609.2 ms. Every later read of that session was 1-6 ms. The layout is derived data, so it is now cached beside the transcript after a scan and read back by the next process instead of scanned again. It is validated against the file it claims to describe -- file identity, file length, the number of offsets, their ascending order, and the first offset of each kind pointing at a line of that kind -- and anything that does not add up, a missing cache, a file that shrank, or a platform with no file identity falls back to the scan that always happened. The read path, the window coordinates and the RPC contract are untouched: the layout a cached process uses is the one a scan produces. Measured on three fixtures with the page cache dropped before every run and the host pinned to 4 cores: transcript scan cached window queued behind the first read 10.5 MB 16.3 ms 3.6 ms 15.6 -> 3.1 ms 127.8 MB 223.5 ms 4.8 ms 223.4 -> 4.1 ms 406.3 MB 616.6 ms 12.7 ms 609.2 -> 12.6 ms Ablation: removing the cache file returns the full cost, 12.7 ms -> 616.6 ms on the 406 MB transcript. The window returned with and without the cache was compared by message id on all three fixtures and is identical. Caching is best effort -- a store that cannot run costs the next process a scan and nothing else -- and a cache below 2 MB of transcript is not written at all.
The legacy Edit shape (`old_string` / `new_string` without `tag` / `ops`)
located the unique match by byte offset but then emitted a whole-line
`PUT first.=last:` with `new_string` as the body. `PUT N.=M` replaces whole
lines, so when `old_string` matched only part of a line the unmatched prefix
of the first line and suffix of the last line were silently deleted while the
tool still reported success (`let x = foo;` with `= foo;` -> `= bar;` became
`= bar;`). Counting `old_string.split('\n')` lines also overshot by one when
`old_string` ended in a newline, so deleting `foo\n` wiped the next line too.
Build the exact substring result instead, trim the unchanged leading and
trailing lines, and lower the difference to a single line-anchored op
(`PUT a.=b:`, `CUT a.=b`, or a `PUT >N:` insertion) that still goes through
`hashline::apply_edit` with the live tag. Only changed lines are anchored, so
the provenance gate sees exactly what changed. Exact whole-line matches produce
the same file as before; an identical replacement still fails with
`EDIT_NO_CHANGE`, and not-found / ambiguous matches still fail with
`EDIT_LEGACY_MATCH_FAILED`.
Refs vastsa#1106
The OpenAI Realtime adapter required an HTTPS base URL and the Live WebSocket transport accepted only wss, so a Realtime-compatible server on the user's own machine (http://127.0.0.1:8010/v1) could not be used. ADR 0304 already lets endpoints the user typed reach loopback and LAN over plain http under the default relaxed network mode; the Realtime path was the one user-supplied endpoint still hard-coded to TLS. Map an http base URL to ws and let the existing public-network guard decide, from networkPolicy.mode, whether that plaintext hop is allowed. Third-party Live endpoints (Gemini) keep wss only, base URLs with credentials, a query or a fragment stay refused, and a plain ws endpoint on a proxied route fails closed rather than being tunneled in the clear. The scheme rules move to a small Electron-free module so they can be tested directly. Refs vastsa#1318
The line-anchored edit format preserves terminal newline state, so the legacy substring adapter cannot express a terminal-newline-only change. Pin the result in a regression test and make the compatibility contract explicit.
…al-line fix(tools): preserve unmatched text in partial-line legacy edits
Exercise the production transport against a loopback WebSocket and keep the proxied plaintext refusal covered. Add a plain-HTTP fixture mode so the full isolated call journey can validate the user-endpoint policy.
…k-http fix(live-voice): let Realtime use a plain-HTTP user endpoint
Transcript content is the source of truth for indexed messages. Sync a line before committing its SQLite row so a failed flush cannot leave a retry with a phantom index. This integration keeps the contributor commits and merges the latest main, including both migration index changes.
…-hot-paths perf(host-core): reduce history costs in search and session reads
…tion-root-fallback
Exercise the production instruction resolver through the real sidecar Read flow so nested rules are replaced by the project root chain for an external attachment. Record the fixture-backed coverage in both E2E plans.
…oot-fallback-17942650 fix(agent-runtime): keep project instructions for out-of-root paths
A panel window is created hidden and shown only after its page loads. A load that fails out loud already ends the open request with its own error, but a load that never settles -- a renderer that crashed while loading, a page stuck before its load event -- left the request running forever: the user's click did nothing, with neither window nor error. The load is now raced against a seconds-scale budget `panelReadyWithin` (next to the close-settle budget it mirrors). A load that succeeds in time shows the window as before; a load that fails on its own keeps its original error; a hang ends the request as PANEL_LOAD_TIMEOUT, which the page that asked for the panel displays like any row action error, while the open path's own catch destroys the hidden window.
The window contract still asserts a load happens and a failed load destroys the hidden window; it now reads the budgeted race instead of a bare awaiting loadURL, which is how the open path loads since the previous commit.
execute_tool_with_path_access called the synchronous Read, Glob, Grep, Write, and Edit bodies inline on the async Tokio worker that handled the request. The host runs the default multi-thread runtime (one worker per core), the ignore walker never yields, and the rg fast path blocks until its child exits. A few large Globs therefore occupied every worker, and cheap RPCs such as session.list waited seconds behind them; the tools.execute budget from vastsa#1208 cannot help because a timer only fires at an await point. Run those five bodies through tokio::task::spawn_blocking with owned inputs. HashlineStore is an Arc<Mutex<_>> handle, so moving a clone onto the blocking thread is cheap. Bash keeps its existing async path. Tool results, error codes, and the wire contract are unchanged; the existing read and mutation class limits also bound the blocking threads. A panicking body now returns INTERNAL instead of dropping the response. The test-only rg override is thread-local, so it is carried onto the blocking thread for the duration of the body. Refs vastsa#1071
System mode left sidecar provider calls and host-core marketplace downloads outside Electron's OS and PAC route. An ephemeral authenticated loopback relay resolves each destination through Electron while keeping relay credentials out of process environments. Preserve Direct and Custom routing and document the runtime contract and current regression coverage.
…ools-off-workers perf(host-core): run sync tool bodies off the tokio workers
Record the 15-second initial-load failure behavior in the plugin spec and E2E scenario so a hanging page has a bounded, visible result and its hidden window is cleaned up. Pin the documented default in the regression test to prevent silent drift.
…-failure fix(plugins): fail a panel that never becomes ready (vastsa#998 item 5)
Expose pi-ai's TypeSafe classifier as an optional Agent tool with explicit settings opt-in. Keep the API key in Host secure storage and document its data flow and acceptance scenario.
The summary is outside the UTF-16 fix and duplicates the shared root path added by other ready maintenance PRs. Drop it so the PR stays scoped and subsequent conflict resolution only covers the E2E plan entries.
Exercise the settings secret lifecycle, Agent-only session key injection, runtime tool discovery, and TypeSafe request handling with local fixtures. Keep the release gate reproducible without calling a real provider.
The summary is outside the delegate recovery fix and duplicates the shared root path added by other ready maintenance PRs. Drop it so the PR stays scoped and later conflict resolution covers only the E2E plan entries.
The summary is outside the quit-time persistence fix and duplicates the shared root path added by other ready maintenance PRs. Drop it so the PR stays scoped and later conflict resolution covers only the E2E plan entries.
perf(host-core): replace session updated_at index with composite index
The latest main contains the completed session index migration and its regression coverage. Revalidate the UTF-16 fix against that integration baseline.
Reuse the shared password visibility strings under the config sync catalog namespace so the settings view resolves them in every locale.
fix(host-core): preserve BOM-marked UTF-16 text through Read and Edit
The latest main contains the session index and UTF-16 fixes. Refresh this PR and preserve both E2E scenarios in the combined delivery document. # Conflicts: # docs/spec/06-delivery/04-e2e-test-plan.md
…recovery fix(agent-runtime): isolate delegate mutation recovery and failure results
…hive-landing # Conflicts: # docs/spec/06-delivery/04-e2e-test-plan.md
feat(jev): integrate TypeSafe classifier
…istence fix(desktop): finish regenerate archival before host shutdown
Choosing the image model a ChatGPT (Codex) account offers under Settings > Models > Image generation model saved nothing. The picker row composed its options from the stored candidate list plus the image model a signed-in vendor account answers with, while the settings page validated the choice against the stored list alone. That list never contains a vendor account's image model — it is not one of the provider's chat models — so the choice was refused and the handler returned silently: no toast, no write, a menu that looked inert. The offer and the acceptance now read one list, composed by imageGenerationPickerCandidates, and one membership rule decides candidate identity for both the picker and the chat-model exclusion. Nothing else changes: an explicitly cleared candidate list is still not resurrected, a missing list still falls back to the legacy single binding, and a signed-out account or another vendor still offers no image model.
fix(settings): accept the image model a signed-in vendor account offers
The renderer extension shipped without reaching the documentation that decides what an author can discover. The permission matrix had no renderer.extension row, the manifest schema never named renderer, rendererActions or rendererCallMethods, and the built-in development guide mentioned no slot at all, so a plugin that draws in the composer or on a message's action bar had no documented way to exist. The matrix gains the permission with its risk tier and its display copy, the manifest schema gains the three fields with the slot contract and the whitelist vocabulary, and the zero-to-one guide and the built-in skill guide each gain a renderer slots section. The zh-CN mirrors follow.
…contract docs(plugins): document the renderer slot contract
The Jev card invited users to paste a TypeSafe key with no way to know whether it worked, and the classifier sat next to the AI service list as a special section instead of being one of the services. Jev is now offered by "Add service" in its own Classifiers group, on the add path only, because it owns no provider row and no model list: changing an existing row's service can never turn it into a classifier. Adding it is one action with two steps. The key is checked against the same TypeSafe System One address and classifier the Agent's `JevClassify` tool calls, and only a key that answered is stored and turns the classifier on, so an enabled Jev always has a key the Agent can spend. A refused key writes nothing and reports TypeSafe's status, and closing the dialog while the check is in flight cancels the whole action, so an abandoned dialog cannot leave a credential behind. The credential is an API key; this integration has no OAuth path. The card becomes the state of the integration rather than a second place to enter a key: the stored-key status, the Agent-mode switch, and the actions that add, replace or remove the key. Removing the key still turns Jev off before it is deleted.
feat(settings): add Jev from the service chooser with a checked key
The Jev card sat on the model configuration page of every install, offering "Add key" to users who had never added TypeSafe Jev at all. Jev is one of the services "Add service" provides, so an install without it now has nothing on that page: the card appears once a key is stored, and it leaves again when the key is removed, after the switch that removal already turned off. Two states keep the card: a Host that cannot answer the secret lookup, and an enabled setting that contradicts a missing key. Hiding a configured install would be the worse failure. The status read stays with the card, so the dialog's report of a stored key is what makes the card appear; the card-visibility test covers both directions.
…added fix(settings): show the Jev card only once Jev has been added
Merge vastsa/PI-Desktop main into the fork, keeping upstream logic and all fork-only features. Conflicts: 3 file(s): apps/desktop/electron/main/runtime/session-launch.ts and packages/agent-runtime/src/sidecar.ts (fork's maxSubagentDepth kept next to upstream's new infiniteProviderRetry / jevApiKey fields), packages/agent-runtime/src/runtime.ts (fork-only imports kept ahead of upstream's mutation-recovery import; scopeDelegateTools now takes both the fork's skillIds and upstream's mutationOwner, tracks both scopes per tool call and clears both, and the fork's early return for unscoped delegates is dropped because upstream always wraps for mutation ownership).
Upstream's new delegate-recovery tests call scopeDelegateTools(tools, definition, owner). The merge had inserted the fork's skillIds parameter before mutationOwner, so those tests passed the owner as skillIds and left the mutation owner undefined, which made delegate failures count against the parent (CI run 37712237427). Keep upstream's order and append the fork's skillIds as an optional trailing parameter; the fork's call site passes both. Co-authored-by: dash0-app[bot] <270606358+dash0-app[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sync vastsa/PI-Desktop main (aecdc3a..52b52f7, 104 commits: v0.17.0, the Jev service and TypeSafe classifier integration, delegate mutation recovery and failure results, BOM-marked UTF-16 Read/Edit, regenerate archival before host shutdown, paged large tool output, the plugin renderer slot contract docs, cloud sync hidden in packaged builds, a CI workflow addition) into the fork, keeping all fork-only features.
The merge commit (
7cd532e9) was pushed by the repository owner because the GitHub App used by Agent0 may not push changes to.github/workflows/. Its tree (a5c18044) is byte-identical to the resolution prepared and self-checked in the sandbox; the three resolved files were taken from the temporary payload branchsync-resolution-20261008, which is deleted once this lands.Conflicts
apps/desktop/electron/main/runtime/session-launch.tsandpackages/agent-runtime/src/sidecar.ts: the fork'smaxSubagentDepthoption kept next to upstream's newinfiniteProviderRetry/jevApiKeyfields. Taking upstream's side of the hunk also restoresinfiniteProviderRetry: settings.infiniteProviderRetry === true, a line an earlier fork conflict resolution had dropped fromsession-launch.ts(the setting never reached the sidecar in the fork since late September).packages/agent-runtime/src/runtime.ts: fork-only imports kept ahead of upstream's new./mutation-recovery.jsimport.scopeDelegateToolswas changed on both sides — the fork addedskillIds(delegate skill scoping), upstream addedmutationOwner(fix(agent-runtime): isolate delegate mutation recovery and failure results vastsa/PI-Desktop#1446) — so it now takes both, records the mutation owner, the permission scope and the skill scope per tool call and clears all three infinally; the fork's early return for unscoped delegates is dropped because upstream always wraps delegate tools for mutation ownership. The call site passesskillIds, delegationId.Validation
Local:
git diff --check;node scripts/check-pr-base-main.mjs; FORK-STANDARD C7 self-check on the merge — every fork line-level diff is unchanged except the two intentional resolutions above; the system-prompt diff against upstream is exactly the two registered hooks (maxSubagentDepthguard andwithNestedDelegationGuidancewrapper) and the wrapped upstream sentence is still present; a scan for lines added identically by both sides in both-sides files (the pattern behind the duplicatemcpServerIdin #30) found none. No local typecheck (nonode_modulesin the sync worktree; per C3 the TypeScript gate is left to PR CI). Upstream CI conclusions could not be read from the sandbox (no credentials for vastsa/PI-Desktop), so the tip was taken as the target per C1. Full gates run in PR CI. E2E skipped for sync merges by repository owner's standing instruction.