Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docs/BRIDGE_PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,28 @@ on decode and back on encode.
code 1 before content I/O. All revision comparisons use the opened handle;
Linux verifies that handle stays inside the workspace, and observed changes
during a read still fail the read-conflict check.
- `write_file` creates a missing path with create-only semantics. To replace an
existing regular text file, send `expected_revision` and `replace_token` from
the **same complete byte-mode `read_file` result** (`complete: true`). The
token is signed for this executor, path, and revision, and is absent from
partial, line-mode, binary, and unsupported reads. A legacy `path`/`content`
call against an existing file fails with guidance. A stale revision fails
before replacement. Since a complete byte result is limited to 4096 bytes,
use `patch_file` for larger files. Hosted dev sessions advertise
`write_file_preconditions: true`; the hosted server rejects coding sessions
that request `write_file` without this flag with a clear upgrade error.
Local and hosted calls execute through the same host validator. Tokens
cannot be carried to another executor/session.
The host stages content in a sibling file and syncs it first. New files are
committed with an atomic hard link that fails if another creator won the
race. Replacements recheck the bounded prior image and path identity just
before same-directory rename. On filesystems where rename over a file is
unsupported or blocked (including some Windows sharing modes), the call
fails and leaves the old file intact. Concurrent writers that bypass this
host and change the path in the interval between final check and rename are
outside the host's compare-and-swap guarantee; use an external workspace
lock when such writers are present. Successful replacements report prior
and new revisions for audit.
- Tool output is bounded. Shell and test output includes `[exit N]`; file reads
include explicit range and continuation metadata. The host sends the same
result shape to local and cloud brains.
Expand Down
10 changes: 10 additions & 0 deletions docs/CONTRACTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,16 @@ byte paging with `sha256: null` and `validation_scope: "returned_range"`.
Opened-handle containment, symlink checks, observed read-conflict checks, and
the output budget still apply.

`write_file` creates new paths exclusively. Existing-file replacement requires
both `expected_revision` and the executor-scoped `replace_token` emitted only
by a complete byte-mode read of that path. Partial reads cannot supply the
token. The host checks the same digest-backed revision over at most 4096 bytes
after staging the replacement; larger files use `patch_file`. The old file is
never opened with truncation. The final same-directory rename is atomic where
supported, and a rename failure preserves the old file. The check and rename
are separate filesystem operations, so non-cooperating external writers need
workspace locking for strict serializability.

---

## 2. ATS trading contracts (Spec 1, Gate 1.0) · frozen
Expand Down
4 changes: 4 additions & 0 deletions src/commands/code_support.ts
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@ export function writeDiffLines(exec: ToolExecutor, args: Record<string, unknown>
// this check execution would fall through to renderDiff and fabricate a
// clean "(new +N)" preview for a write that never actually happens.
if (snap.reason === "unsafe") return [];
// Legacy overwrite calls fail at execution. Do not render their proposed
// content as if the file had been changed before that refusal is shown.
const hasProof = typeof args["expected_revision"] === "string" && typeof args["replace_token"] === "string";
if (snap.existed !== hasProof) return [];

if (snap.existed && snap.text === null) {
const what = snap.reason === "binary" ? "binary" : "large file";
Expand Down
1 change: 1 addition & 0 deletions src/core/brain_cloud.ts
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,7 @@ export class CloudBrain implements Brain {
capabilities: this.opts.localToolCapabilities ?? TOOLS,
readFileRanges: true,
readFileRevisions: process.platform === "linux",
writeFilePreconditions: true,
maxUvt: this.opts.maxUvt,
protocolVersion: DEV_PROTOCOL_VERSION,
}),
Expand Down
4 changes: 2 additions & 2 deletions src/core/brain_ollama.ts
Original file line number Diff line number Diff line change
Expand Up @@ -334,10 +334,10 @@ function parseArgs(raw: string): Record<string, unknown> {
*/
export function ollamaToolSchemas(tools: readonly ToolName[] = TOOLS): readonly ToolSchema[] {
const summaries: Readonly<Record<ToolName, string>> = {
read_file: "Read a bounded byte range (offset/max_bytes) or line range (start_line/max_lines). On supported Linux filesystems, pass the returned revision as expected_revision on each continuation and restart if stale; other systems return revision_unsupported for guarded reads. An initial unguarded read can return a SHA-256 digest for patch_file. Do not combine range modes.",
read_file: "Read a bounded byte range (offset/max_bytes) or line range (start_line/max_lines). A complete byte read includes a replace_token for write_file. On supported Linux filesystems, pass the revision as expected_revision on each continuation and restart if stale; other systems return revision_unsupported for guarded reads. An initial unguarded read can return a SHA-256 digest for patch_file. Do not combine range modes.",
list_directory: "List one bounded page of a workspace directory. Use the returned cursor for the next page.",
patch_file: "Replace one exact range in a file using its read_file SHA-256 digest. A stale digest fails without writing.",
write_file: "Write or overwrite a workspace file.",
write_file: "Create a new workspace file, or replace an existing small text file only with expected_revision and replace_token from one complete read_file byte result. Partial reads cannot authorize replacement; use patch_file for larger files.",
run_shell: "Run a shell command in the workspace.",
run_tests: "Run the project's test command.",
repo_search: "Grep the repository for a string.",
Expand Down
8 changes: 8 additions & 0 deletions src/core/brain_protocol.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,14 @@ export interface ReadFileArgs {
expected_revision?: string;
}

/** Existing-file writes need both fields from one complete read_file byte result. */
export interface WriteFileArgs {
path: string;
content: string;
expected_revision?: string;
replace_token?: string;
}

export interface WorkflowDoneFrame {
type: "workflow_done";
synthesis: string;
Expand Down
4 changes: 4 additions & 0 deletions src/core/envelope.ts
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ export interface DevSessionWireRequest {
read_file_ranges?: boolean;
/** Advertise expected_revision support independently of older range support. */
read_file_revisions?: boolean;
/** Host refuses legacy blind replacements of existing files. */
write_file_preconditions?: boolean;
max_uvt?: number;
repo?: Record<string, unknown>;
protocol_version: number;
Expand All @@ -78,6 +80,7 @@ export interface BuildDevSessionArgs {
capabilities: readonly string[];
readFileRanges?: boolean;
readFileRevisions?: boolean;
writeFilePreconditions?: boolean;
maxUvt?: number;
repo?: Record<string, unknown>;
protocolVersion: number;
Expand All @@ -95,6 +98,7 @@ export function buildDevSessionRequest(args: BuildDevSessionArgs): DevSessionWir
if (args.maxUvt && args.maxUvt > 0) req.max_uvt = args.maxUvt;
if (args.readFileRanges) req.read_file_ranges = true;
if (args.readFileRevisions) req.read_file_revisions = true;
if (args.writeFilePreconditions) req.write_file_preconditions = true;
if (args.repo) req.repo = args.repo;
return req;
}
Expand Down
105 changes: 95 additions & 10 deletions src/core/tool_executor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@
// regardless of which side originally ran it.

import { spawn, spawnSync } from "node:child_process";
import { createHash, randomUUID } from "node:crypto";
import { chmodSync, closeSync, constants as fsConstants, existsSync, fstatSync, fsyncSync, lstatSync, mkdirSync, openSync, opendirSync, readFileSync, readlinkSync, readdirSync, readSync, realpathSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
import { createHash, createHmac, randomBytes, randomUUID, timingSafeEqual } from "node:crypto";
import { chmodSync, closeSync, constants as fsConstants, existsSync, fstatSync, fsyncSync, linkSync, lstatSync, mkdirSync, openSync, opendirSync, readFileSync, readlinkSync, readdirSync, readSync, realpathSync, renameSync, statSync, unlinkSync, writeFileSync } from "node:fs";
import type { BigIntStats, Dirent } from "node:fs";
import { dirname, relative, resolve, sep } from "node:path";
import { StringDecoder } from "node:string_decoder";
Expand All @@ -32,6 +32,9 @@ const DEFAULT_TEST_CMD = "";
const SNAPSHOT_MAX_BYTES = 1024 * 1024;
const SEARCH_MAX_HITS = 40;
const FILE_PATCH_MAX_BYTES = 16 * 1024 * 1024;
// A complete byte read is at most 4096 bytes. Replacement checks only this
// bounded prior image, never a whole large file.
const FILE_REPLACE_MAX_BYTES = 4096;
function fileRevision(stat: BigIntStats, digest: string | null = null): string {
const fields = [stat.dev, stat.ino, stat.size, stat.mtimeNs, stat.ctimeNs, stat.birthtimeNs];
return "r1_" + createHash("sha256").update([...fields.map(String), digest ?? ""].join(":"), "utf8").digest("base64url");
Expand Down Expand Up @@ -87,6 +90,7 @@ export interface FileSnapshot {

export class ToolExecutor {
private readonly root: string;
private readonly replacementKey = randomBytes(32);
private readonly mode: ToolExecutionContext["mode"];
/**
* Built on first use by armCommitGuard(), never in the constructor. See the
Expand Down Expand Up @@ -415,7 +419,7 @@ export class ToolExecutor {
case "patch_file":
return this.patchFile(args);
case "write_file":
return this.writeFile(String(args["path"] ?? ""), String(args["content"] ?? ""));
return this.writeFile(args);
case "run_shell":
case "run_tests":
// Shell-backed tools became async so a timeout or Ctrl+C can reap the
Expand Down Expand Up @@ -602,7 +606,10 @@ export class ToolExecutor {
try { content = decoder.decode(bytes.subarray(0, end)); }
catch { end--; continue; }
const nextOffset = offset + end;
output = JSON.stringify({ path, sha256: digest, revision, offset, range_end: nextOffset,
output = JSON.stringify({ path, sha256: digest, revision,
...(offset === 0 && nextOffset === size && size <= FILE_REPLACE_MAX_BYTES
? { replace_token: this.replacementToken(abs, revision) } : {}),
offset, range_end: nextOffset,
next_offset: nextOffset < size ? nextOffset : null, size,
complete: offset === 0 && nextOffset === size, truncated: nextOffset < size,
starts_mid_line: startsMidLine,
Expand Down Expand Up @@ -785,19 +792,97 @@ export class ToolExecutor {
}
}

private writeFile(path: string, content: string): ToolResult {
private replacementToken(abs: string, revision: string): string {
return "w1_" + createHmac("sha256", this.replacementKey).update(abs).update("\0").update(revision).digest("base64url");
}

private writeFile(args: Record<string, string | number>): ToolResult {
const path = String(args["path"]);
const content = String(args["content"]);
const expected = args["expected_revision"] as string | undefined;
const token = args["replace_token"] as string | undefined;
const abs = this.safe(path);
// Never turn an intended replacement into creation, or accept a partial
// read's public revision without its complete-read proof.
if ((expected === undefined) !== (token === undefined)) throw new Error("replacement requires expected_revision and replace_token from one complete read_file result");
if (expected !== undefined) {
const actual = Buffer.from(this.replacementToken(abs, expected));
const supplied = Buffer.from(token!);
if (actual.length !== supplied.length || !timingSafeEqual(actual, supplied)) {
throw new Error("invalid replace_token: read the complete file before replacement");
}
}
if (expected === undefined && existsSync(abs)) {
throw new Error("file already exists: read the complete file and pass expected_revision with replace_token; use patch_file for larger files");
}
if (expected !== undefined && !existsSync(abs)) {
throw new Error(`stale_revision: ${path}; replacement target no longer exists`);
}
mkdirSync(dirname(abs), { recursive: true });
// Re-validate immediately before opening, then refuse a symlink final component.
const verified = this.safe(path);
this.safe(path);
const staged = resolve(dirname(abs), `.aether-write-${randomUUID()}.tmp`);
const noFollow = fsConstants.O_NOFOLLOW ?? 0;
const fd = openSync(verified, fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_TRUNC | noFollow, 0o600);
let fd: number | undefined;
try {
// Stage in the same directory so the final rename/link is atomic on
// filesystems that provide those operations. The old file stays intact
// if writing, syncing, or changing permissions fails.
fd = openSync(staged, fsConstants.O_WRONLY | fsConstants.O_CREAT | fsConstants.O_EXCL | noFollow, 0o600);
writeFileSync(fd, content, "utf8");
fsyncSync(fd);
closeSync(fd); fd = undefined;
if (expected === undefined) {
this.safe(path);
linkSync(staged, abs); // atomic create-only; EEXIST preserves a racing creator
return { output: `[created ${path} · ${Buffer.byteLength(content)} bytes · revision new]`, exitCode: 0 };
}
// Validate the named file after staging, immediately before rename.
// The digest uses at most 4096 bytes, the largest complete read_file
// response; large targets must use patch_file instead.
const current = this.replacementRevision(path, abs);
if (current.revision !== expected) throw new Error(`stale_revision: ${path}; read again before replacement`);
chmodSync(staged, current.mode);
this.safe(path);
const confirmed = this.replacementRevision(path, abs);
if (confirmed.revision !== expected) throw new Error(`stale_revision: ${path}; read again before replacement`);
renameSync(staged, abs);
const newStat = statSync(abs, { bigint: true });
const newDigest = createHash("sha256").update(content, "utf8").digest("hex");
return { output: `[replaced ${path} · prior_revision ${expected} · new_revision ${fileRevision(newStat, newDigest)} · ${Buffer.byteLength(content)} bytes]`, exitCode: 0 };
} finally {
closeSync(fd);
if (fd !== undefined) closeSync(fd);
if (existsSync(staged)) unlinkSync(staged);
}
return { output: `[wrote ${path} · ${Buffer.byteLength(content)} bytes]`, exitCode: 0 };
}

private replacementRevision(path: string, abs: string): { revision: string; mode: number } {
this.safe(path);
const named = lstatSync(abs, { bigint: true });
if (!named.isFile() || named.isSymbolicLink()) throw new Error("replacement target must be a regular file");
if (named.size > BigInt(FILE_REPLACE_MAX_BYTES)) throw new Error("replacement target exceeds a complete read_file page; use patch_file");
const fd = openSync(abs, fsConstants.O_RDONLY | (fsConstants.O_NOFOLLOW ?? 0));
try {
const before = fstatSync(fd, { bigint: true });
if (!before.isFile() || before.dev !== named.dev || before.ino !== named.ino) throw new Error("replacement target changed while opening");
if (process.platform === "linux") {
const opened = readlinkSync(`/proc/self/fd/${fd}`).replace(/ \(deleted\)$/, "");
if (opened !== this.root && !opened.startsWith(this.root + sep)) throw new Error("replacement target opened outside workspace");
}
const bytes = Buffer.alloc(Number(before.size));
for (let at = 0; at < bytes.length;) {
const n = readSync(fd, bytes, at, bytes.length - at, at);
if (n === 0) throw new Error("replacement target changed during check");
at += n;
}
const after = fstatSync(fd, { bigint: true });
const finalName = lstatSync(abs, { bigint: true });
if (after.dev !== before.dev || after.ino !== before.ino || after.size !== before.size
|| after.mtimeNs !== before.mtimeNs || after.ctimeNs !== before.ctimeNs
|| finalName.dev !== before.dev || finalName.ino !== before.ino || finalName.isSymbolicLink()) {
throw new Error("replacement target changed during check");
}
return { revision: fileRevision(before, createHash("sha256").update(bytes).digest("hex")), mode: Number(before.mode) };
} finally { closeSync(fd); }
}

/**
Expand Down
5 changes: 4 additions & 1 deletion src/core/tool_registry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ export const TOOL_DEFINITIONS: Readonly<Record<ToolName, ToolDefinition>> = {
},
write_file: {
sideEffect: "write",
args: { path: stringArg(4096), content: stringArg(1024 * 1024, true, true) },
args: { path: stringArg(4096), content: stringArg(1024 * 1024, true, true), expected_revision: stringArg(128, false), replace_token: stringArg(128, false) },
},
run_shell: {
sideEffect: "shell",
Expand Down Expand Up @@ -126,6 +126,9 @@ export function validateToolCall(name: string, rawArgs: unknown): ToolValidation
&& (validated["offset"] !== undefined || validated["max_bytes"] !== undefined)) {
return { ok: false, error: "line and byte ranges cannot be combined" };
}
if (name === "write_file" && (validated["expected_revision"] === undefined) !== (validated["replace_token"] === undefined)) {
return { ok: false, error: "replacement requires expected_revision and replace_token together" };
}
return { ok: true, name: name as ToolName, args: validated };
}

Expand Down
1 change: 1 addition & 0 deletions test/brain_cloud_dev.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ test("dev session: create carries effort + capabilities; tool_call surfaces and
assert.ok((createBody["capabilities"] as string[]).includes("patch_file"));
assert.equal(createBody["read_file_ranges"], true);
assert.equal(createBody["read_file_revisions"], process.platform === "linux" ? true : undefined);
assert.equal(createBody["write_file_preconditions"], true);

const tc = out.find((e) => e.type === "tool_call");
assert.ok(tc && tc.type === "tool_call");
Expand Down
2 changes: 1 addition & 1 deletion test/bridge.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ test("ToolExecutor writes then reads a file in the workspace", () => {
const ex = new ToolExecutor(dir);
const w = ex.execute("write_file", { path: "a.txt", content: "hello" });
assert.equal(w.exitCode, 0);
assert.match(w.output, /\[wrote a\.txt/);
assert.match(w.output, /\[created a\.txt/);
assert.equal(readFileSync(join(dir, "a.txt"), "utf8"), "hello");
const r = ex.execute("read_file", { path: "a.txt" });
assert.equal(JSON.parse(r.output).content, "hello");
Expand Down
2 changes: 2 additions & 0 deletions test/envelope.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ test("dev-session request advertises range reads only for a capable host", () =>
assert.equal(buildDevSessionRequest({ ...base, readFileRanges: true }).read_file_ranges, true);
assert.equal(buildDevSessionRequest(base).read_file_revisions, undefined);
assert.equal(buildDevSessionRequest({ ...base, readFileRevisions: true }).read_file_revisions, true);
assert.equal(buildDevSessionRequest(base).write_file_preconditions, undefined);
assert.equal(buildDevSessionRequest({ ...base, writeFilePreconditions: true }).write_file_preconditions, true);
});

test("buildChatRequest nulls empty model/agent and defaults to auto", () => {
Expand Down
5 changes: 5 additions & 0 deletions test/filesystem_tools.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -149,4 +149,9 @@ test("local model, hosted capabilities, and headless allowlist advertise the new
assert.ok(schemas.includes(name));
assert.ok((EXEC_V1_TOOLS as readonly string[]).includes(name));
}
const write = ollamaToolSchemas().find((schema) => schema.function.name === "write_file")!;
const parameters = write.function.parameters as { properties: Record<string, unknown> };
assert.ok(parameters.properties["expected_revision"]);
assert.ok(parameters.properties["replace_token"]);
assert.match(write.function.description ?? "", /complete read_file/);
});
Loading
Loading