Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .changeset/fix-memory-status-default-quotes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
"opencode-swarm-plugin": patch
"swarm-mail": patch
---

## 🐝 Memories Were Whispering Into a Room With No Door

> "Program testing can be used to show the presence of bugs, but never to show their absence." — Edsger W. Dijkstra

**What changed:**
- `packages/swarm-mail/src/db/schema/memory.ts`: the Drizzle column default for `status` carried literal quote characters — `.default("'active'")` instead of `.default("active")`. `store()` never sets `status` explicitly, so every memory was written with the 8-character string `'active'` (quotes included, as data) instead of the correct 6-character `active`.
- `search()` and `ftsSearch()` filter on `status = 'active'`, so every corrupted row silently became permanently unsearchable — while `getStats()` kept counting the rows and reporting a perfectly healthy corpus. Store succeeds, search returns nothing, stats lies. Indistinguishable from an empty index.
- Added a regression test that reads the *raw* `status` column value straight off `store()`'s output, not just round-trip search behavior — the existing widened `statusFilter` (`m.status = 'active' OR m.status = '''active'''`) already masks this bug on the read side, so only a direct column read catches the write-side corruption.

**Why it matters:**

Found in the wild: this silently disabled semantic memory across a full multi-agent session. Every agent's "no prior results" looked exactly like a legitimately empty index — no error, no warning, nothing to notice, right up until someone checked the raw column against the stats count by hand.

**Migration:** none needed. The existing `statusFilter` widening (already on `main`, unchanged by this patch) keeps pre-existing corrupted rows searchable — this fix only stops *new* writes from being corrupted going forward. The widening can be dropped in a future major version once old installs have aged out.

**Backward compatible:** yes — no data is lost or requires backfill; previously-corrupted rows stay reachable through the existing filter, they just stop multiplying.
Comment on lines +19 to +21

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

fd -t f . packages/swarm-mail | sort | while IFS= read -r file; do
  if rg -q 'drizzle|migrat|CREATE TABLE|memories|status' "$file"; then
    printf '\n### %s\n' "$file"
    rg -n -C 4 'drizzle|migrat|CREATE TABLE|memories|status' "$file"
  fi
done

Repository: joelhooks/swarm-tools

Length of output: 50379


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Find relevant schema/migration/store files\n'
fd -t f 'memory.ts|migrations.ts|store.ts$|migrations' packages/swamp-mail packages/swarm-mail 2>/dev/null || true

printf '\nMemory schema around statusFilter/default\n'
sed -n '70,95p' packages/swarm-mail/src/db/schema/memory.ts 2>/dev/null || true

printf '\nMemory store read filter around statusFilter\n'
sed -n '220,240p' packages/swarm-mail/src/memory/store.ts 2>/dev/null || true

printf '\nMemory migrations definitions\n'
sed -n '1,220p' packages/swarm-mail/src/memory/migrations.ts 2>/dev/null || true

printf '\nMemory schema exports and imports\n'
rg -n "memory|create.*memory|schema_|schemaVersion|version.*memory|statusFilter" packages/swarm-mail/src -g '*.{ts,tsx,js,jsx}' | head -n 200

Repository: joelhooks/swarm-tools

Length of output: 31806


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf 'Memory migrations remaining definitions\n'
sed -n '220,420p' packages/swarm-mail/src/memory/migrations.ts

printf '\nMemory migrations array exports\n'
rg -n "memoryMigrations|memorySchemaOverhaulLibSQL|memoryMigrationsLibSQL|version.*1[0-2][0-9]?|status" packages/swarm-mail/src/memory/migrations.ts packages/swarm-mail/src/memory -g '*.ts' | head -n 200

printf '\nMemory store write/insert status usages\n'
rg -n -C 4 "status|statusFilter|status.*active|new\(|insert|storeMemory|MemoryAdapter|default" packages/swarm-mail/src/memory/store.ts packages/swarm-mail/src/memory/adapter.ts packages/swarm-mail/src/memory -g '*.ts' | head -n 260

Repository: joelhooks/swarm-tools

Length of output: 35382


Require migration guidance for the memories.status default fix.

default("active") does not change the default in existing SQLite memories columns or update rows where status already stores '''active'''. Either add an ALTER/fix migration or update this changeset to document the required deployment/manual update step instead of saying “Migration: none needed.”

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.changeset/fix-memory-status-default-quotes.md around lines 19 - 21, Update
the changeset’s migration guidance for the memories.status default fix: remove
the claim that no migration is needed and document the required
deployment/manual database update step, or add the corresponding ALTER/fix
migration. Ensure it addresses both existing SQLite column defaults and rows
containing the corrupted '''active''' value.


```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Specify a language for the fenced block.

markdownlint-cli2 reports MD040 for this fence. Use text for the ASCII-art block.

Proposed fix
-```
+```text
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
```
🧰 Tools
🪛 markdownlint-cli2 (0.23.2)

[warning] 23-23: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.changeset/fix-memory-status-default-quotes.md at line 23, Update the fenced
code block in fix-memory-status-default-quotes.md to specify the text language
by changing its opening fence to use text, while preserving the ASCII-art
content and closing fence.

Source: Linters/SAST tools

,-.__,-.
( o o ) "getStats() says I'm fine."
\ ^ / "search() says I don't exist."
`-----' turns out: I was written with my own name in quotes.
```
2 changes: 1 addition & 1 deletion packages/swarm-mail/src/db/schema/memory.ts
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ export const memories = sqliteTable("memories", {
last_accessed: text("last_accessed").default("(datetime('now'))"),
// Fact categorization
category: text("category"), // relationship, milestone, status, preference, context
status: text("status").default("'active'"), // active, superseded
status: text("status").default("active"), // active, superseded
});

/**
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,22 @@
*/

import { createClient } from "@libsql/client";
import { beforeEach, describe, expect, test } from "bun:test";
import {
afterAll,
beforeAll,
beforeEach,
describe,
expect,
test,
} from "bun:test";
import { sql } from "drizzle-orm";
import type { SwarmDb } from "../db/client.js";
import { createDrizzleClient } from "../db/drizzle.js";
import {
createInMemorySwarmMailLibSQL,
toSwarmDb,
} from "../libsql.convenience.js";
import type { SwarmMailAdapter } from "../types/adapter.js";
import { type Memory, createMemoryStore } from "./store.js";

/**
Expand Down Expand Up @@ -297,7 +311,7 @@ describe("Memory Store (Drizzle) - Vector Search", () => {
collection: mem.collection,
createdAt: new Date(),
},
mem.embedding
mem.embedding,
);
}
});
Expand Down Expand Up @@ -359,3 +373,123 @@ describe("Memory Store (Drizzle) - Vector Search", () => {
expect(results).toEqual([]);
});
});

describe("Memory Store (Drizzle) - Status Corruption Regression", () => {
// Regression test for a bug where the Drizzle schema's column default
// (db/schema/memory.ts `status: text("status").default("'active'")`) contained
// literal quote characters. Drizzle's SQLite dialect binds `.default()` as a
// client-side insert parameter whenever a column is omitted from `.values()`,
// so every memory silently got a `status` of the 8-char string `'active'`
// instead of `active` - independent of what the table's raw SQL DDL default said.
// search()/ftsSearch() filtered on `status = 'active'`, which never matched,
// so every memory became permanently unsearchable while getStats() still
// reported a healthy count. Store succeeded, search returned empty: silent
// write-only corruption with stats showing everything was fine.
let swarmMail: SwarmMailAdapter;
let db: SwarmDb;
let store: ReturnType<typeof createMemoryStore>;

beforeAll(async () => {
swarmMail = await createInMemorySwarmMailLibSQL(
"status-corruption-regression",
);
const dbAdapter = await swarmMail.getDatabase();
db = toSwarmDb(dbAdapter);
store = createMemoryStore(db);
});

afterAll(async () => {
await swarmMail.close();
});

test("store() writes a clean 'active' status, not a quoted literal", async () => {
const memory: Memory = {
id: "mem-status-1",
content: "Some content",
metadata: {},
collection: "default",
createdAt: new Date(),
};

await store.store(memory, mockEmbedding(1));

const rows = await db.all<{ status: string }>(
sql`SELECT status FROM memories WHERE id = ${memory.id}`,
);
expect(rows[0]?.status).toBe("active");
});

test("a freshly stored memory is retrievable via search() (end-to-end)", async () => {
const memory: Memory = {
id: "mem-status-2",
content: "Findable content about durable fixes",
metadata: {},
collection: "default",
createdAt: new Date(),
};
const embedding = mockEmbedding(7);

await store.store(memory, embedding);

const results = await store.search(embedding, { threshold: 0 });
expect(results.map((r) => r.memory.id)).toContain("mem-status-2");
});

test("a freshly stored memory is retrievable via ftsSearch() (end-to-end)", async () => {
const memory: Memory = {
id: "mem-status-3",
content: "unique-fts-marker-zzyzx durable fixes",
metadata: {},
collection: "default",
createdAt: new Date(),
};

await store.store(memory, mockEmbedding(8));

const results = await store.ftsSearch("unique-fts-marker-zzyzx");
expect(results.map((r) => r.memory.id)).toContain("mem-status-3");
});

test("search() still finds legacy-corrupted rows (status = literal 'active')", async () => {
// Simulates a row written by an unpatched install before this fix, so
// memories already in production databases remain searchable.
const memory: Memory = {
id: "mem-legacy",
content: "Legacy corrupted row",
metadata: {},
collection: "default",
createdAt: new Date(),
};
await store.store(memory, mockEmbedding(3));

// Force the legacy-corrupted value directly, bypassing the (now-fixed) default.
await db.run(
sql`UPDATE memories SET status = '''active''' WHERE id = ${memory.id}`,
);

const results = await store.search(mockEmbedding(3), { threshold: 0 });
expect(results.map((r) => r.memory.id)).toContain("mem-legacy");
});
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

test("ftsSearch() still finds legacy-corrupted rows (status = literal 'active')", async () => {
// Mirrors the search() legacy-status test above: ftsSearch() has its own
// end-to-end query path and status filter, so it needs its own coverage
// of the corrupted-status case rather than inheriting search()'s.
const memory: Memory = {
id: "mem-legacy-fts",
content: "unique-fts-marker-legacy durable fixes",
metadata: {},
collection: "default",
createdAt: new Date(),
};
await store.store(memory, mockEmbedding(9));

// Force the legacy-corrupted value directly, bypassing the (now-fixed) default.
await db.run(
sql`UPDATE memories SET status = '''active''' WHERE id = ${memory.id}`,
);

const results = await store.ftsSearch("unique-fts-marker-legacy");
expect(results.map((r) => r.memory.id)).toContain("mem-legacy-fts");
});
});