Skip to content

Latest commit

 

History

4 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

@torkbot/harness

Build durable, restart-safe agent lanes on Sledge.

Harness turns typed application stimuli into canonical model conversations. It owns per-lane scheduling, bounded input collection, retryable model turns, durable tool execution, and atomic conversation commits. Your application owns what stimuli mean, when they should become a turn, how they become messages, which prompts and tools are available, and how a model is invoked.

Use Harness when an agent loop must survive crashes and retries without losing input, running concurrent turns for one lane, repeating committed conversation output, or leaving tool batches half-finished.

Warning

Harness is experimental. It is a product-shaped testbed for durable agent-loop primitives, not a stable production release. Its API, canonical message contract, durable events, projections, and storage layout may change between revisions, and an upgrade may require an intentional ledger reset instead of a migration. Pin exact revisions and use Harness only where you control the compatibility posture of durable data.

Install

npm install @torkbot/harness @torkbot/sledge typebox

Harness requires Node.js 24 or newer.

Quick start

Define the opaque input accepted by a lane, adapt it into canonical messages, and compose the resulting modules into one Sledge model:

import {
  type AgentModel,
  type AgentTool,
  defineAgentTurnLedger,
} from "@torkbot/harness/agent-turn-ledger";
import { defineLaneLedger } from "@torkbot/harness/lane-ledger";
import { composeLedgerModels } from "@torkbot/sledge/ledger";
import {
  NodeRuntimeScheduler,
  SystemRuntimeClock,
} from "@torkbot/sledge/runtime/node-runtime";
import { createTursoLedger } from "@torkbot/sledge/turso-ledger";
import { Type } from "typebox";

const ChatStimulus = Type.Object(
  {
    text: Type.String(),
    timestamp: Type.Number(),
  },
  { additionalProperties: false },
);

const lanes = defineLaneLedger({
  moduleId: "chat.lanes",
  stimulusSchema: ChatStimulus,
});

const clock = new SystemRuntimeClock();

const model: AgentModel = {
  async complete(input) {
    // Adapt your provider here. It must return one canonical
    // AgentAssistantMessage and propagate input.signal.
    return provider.complete(input);
  },
};

const turns = defineAgentTurnLedger({
  moduleId: "chat.turns",
  lanes,
  toolLedger: {
    events: {},
    queries: {},
  },
  prepareTurn: ({ stimuli }, turn) => {
    const latest = stimuli.at(-1);

    if (latest === undefined) {
      throw new Error("turn preparation requires pending stimuli");
    }

    return turn.consumeThrough(latest.consumptionToken, [
      {
        role: "user",
        content: stimuli.map(({ stimulus }) => stimulus.text).join("\n"),
        timestamp: latest.stimulus.timestamp,
      },
    ]);
  },
  prepareModelInput: async ({ laneId, signal }) => {
    signal.throwIfAborted();

    return {
      systemPrompt: `You are the assistant for ${laneId}.`,
    };
  },
  model,
  clock,
  toolExecutionTimeoutMs: 30_000,
  logger: applicationLogger,
});

const ledgerModel = composeLedgerModels(lanes, turns);

await using ledger = await createTursoLedger({
  databaseUrl: "./assistant.sqlite",
  model: ledgerModel,
  timing: {
    clock,
  },
});

await ledger.emit(
  lanes.events.inputAppended,
  {
    laneId: "customer-42",
    stimulus: {
      text: "Where is my order?",
      timestamp: clock.nowMs(),
    },
  },
  {
    dedupeKey: "message:msg-123",
  },
);

await using workers = await ledger.startWorkers({
  scheduler: new NodeRuntimeScheduler(),
});

Appending inputAppended durably schedules the affected lane. Workers recover its pending stimuli, prepare the current model input, invoke the model, and atomically commit the new conversation messages with the consumed input prefix.

Opening a ledger is passive. Start workers only in the process that owns agent execution.

How Harness is structured

Harness provides two composable Sledge modules.

defineLaneLedger(...)

The lane module owns the durable input stream:

  • ordered, schema-validated stimuli partitioned by lane ID;
  • a stable turn ID derived from the oldest pending stimulus;
  • attempt-scoped consumption tokens for selecting a pending prefix;
  • atomic commitment or durable ignoring of that prefix.

The application appends lanes.events.inputAppended. The lane module preserves the stimulus without interpreting it. Its turnCandidate query exposes the current ordered backlog to composing modules.

Consumption tokens are process-local capabilities. Callers can select only a token offered by the current attempt; they cannot inspect its cursor, fabricate one, or reuse it after a retry. Harness persists the underlying cursor only when the corresponding durable event commits.

defineAgentTurnLedger(...)

The turn module composes with a lane module and owns agent execution:

  • partitioned, coalesced lane wakes;
  • optional bounded turn admission;
  • dynamic prompt and tool preparation;
  • retryable, cancellation-aware model invocation;
  • durable tool batches and independently executable calls;
  • canonical conversation validation and recovery;
  • atomic commitment of assistant output and consumed lane input.

Its application-supplied configuration is:

Field Responsibility
moduleId Uniquely identifies the Sledge module.
lanes Supplies the typed lane capabilities to consume.
toolLedger Declares the event and query capabilities available to tools.
prepareTurn Converts an admitted pending prefix into canonical messages.
prepareModelInput Resolves the current system prompt, tools, and cache hints.
model Performs one canonical assistant completion.
clock Timestamps engine-produced tool results.
toolExecutionTimeoutMs Bounds each tool handler attempt.
logger Records recoverable retries and terminal tool failures.
admitTurn and maxStimulusWaitMs Optionally control bounded input collection.

Omit both admission fields to admit every candidate immediately. Supplying admitTurn requires maxStimulusWaitMs.

Turn admission and windowing

Admission decides whether the current pending stimuli should become model input now. Harness calls it whenever pending stimuli are reconsidered because new input arrived, a deferred deadline became due, or a completed tool batch is ready while stimuli are also pending.

const turns = defineAgentTurnLedger({
  // ...
  maxStimulusWaitMs: 2_000,
  admitTurn: ({ stimuli }, choice) => {
    const first = stimuli[0];

    if (isOutOfBand(first.stimulus)) {
      return choice.ignoreThrough(first.consumptionToken);
    }

    if (stimuli.length < 4) {
      return choice.defer();
    }

    return choice.admit();
  },
  prepareTurn: ({ stimuli }, turn) => {
    const latest = stimuli.at(-1);

    if (latest === undefined) {
      throw new Error("turn preparation requires pending stimuli");
    }

    return turn.consumeThrough(
      latest.consumptionToken,
      renderMessages(stimuli),
    );
  },
});

Every admission attempt must return one value created by its supplied choice port:

Choice Effect
choice.admit() Calls prepareTurn for the current candidate.
choice.defer() Stops this branch and schedules reconsideration within the bounded window.
choice.ignoreThrough(token) Durably removes the selected prefix from future turn consideration without invoking the model.

Deferral uses a fixed deadline:

deadline = oldest pending stimulus time + maxStimulusWaitMs

New input wakes the lane and runs admission again, so a growing backlog can promote itself to immediate work. New input, retries, and restarts never move the deadline later. Once the deadline is due, Harness overrides another deferral and calls prepareTurn fresh.

prepareTurn runs only for admitted work. It must return turn.consumeThrough(token, messages), which makes both the consumed prefix and its non-empty canonical message representation mandatory. The supplied port and tokens are valid only for that attempt.

The admission trigger distinguishes ordinary input from a tool continuation:

type TurnAdmissionTrigger =
  | { readonly kind: "stimuli" }
  | { readonly kind: "tool-continuation" };

A tool continuation may proceed while newly pending stimuli remain deferred. The next wake reconsiders those same stimuli against their original deadline.

Model inputs and conversation state

prepareModelInput runs immediately before every model attempt and again when Harness resolves tools requested by that turn. It receives:

  • the lane ID and stable turn ID;
  • the exact canonical conversation for this input snapshot;
  • cancellation tied to the active durable work lease.

It returns the current system prompt, executable tools, and optional provider affinity hints. System prompts and tool definitions are not durable conversation messages; Harness resolves them again after a retry or restart.

Harness defaults the affinity key to the lane ID and cache retention to "short". Applications can override either for provider routing and prompt caching. The model adapter remains responsible for determining whether a request can actually reuse provider state.

Committed turns store only the exact messages added by that successful turn. The conversationPrefix query reconstructs a lane's canonical conversation in commit order.

If model preparation or completion fails, Harness commits neither the prepared messages nor the lane cursor. Retried work reads fresh conversation and lane state, including stimuli that arrived during the failed attempt.

Tools

Tools combine a model-visible definition with an executable handler:

const RememberParameters = Type.Object(
  {
    fact: Type.String(),
  },
  { additionalProperties: false },
);

const rememberTool: AgentTool<typeof RememberParameters> = {
  name: "remember",
  description: "Persist one fact for the current assistant.",
  parameters: RememberParameters,
  async execute(context, parameters) {
    context.signal.throwIfAborted();

    await context.ledger.emit(
      memory.events.factRemembered,
      {
        fact: parameters.fact,
        idempotencyKey: context.idempotencyKey,
      },
      {
        dedupeKey: context.idempotencyKey,
      },
    );

    return {
      content: [
        {
          type: "text",
          text: "Fact remembered.",
        },
      ],
      isError: false,
    };
  },
};

Declare every event and query a tool may use when defining the turn module:

const turns = defineAgentTurnLedger({
  // ...
  toolLedger: {
    events: {
      factRemembered: memory.events.factRemembered,
    },
    queries: {
      factsForLane: memory.queries.factsForLane,
    },
  },
  prepareModelInput: async () => {
    return {
      tools: [rememberTool],
    };
  },
});

Handlers receive an attempt-scoped ledger port, not the public Sledge ledger. Harness rejects undeclared capabilities and operations started after the handler settles.

Tool event emissions commit immediately before their promises resolve. They remain durable if later handler work fails or times out, so handlers must use the stable idempotencyKey, event dedupe keys, or domain-level idempotency for retryable side effects.

An assistant response requesting tools commits before any handler runs. Each call then becomes independent at-least-once work. Harness:

  • validates arguments against the current tool schema;
  • supplies cancellation and a stable per-call idempotency key;
  • enforces toolExecutionTimeoutMs;
  • records missing tools, invalid input, thrown errors, invalid output, and timeout as canonical error results;
  • preserves result order from the assistant's original call batch;
  • wakes the lane as soon as every result is durable.

Turn commits and tool results are accepted only from their engine-owned Sledge queue attempts. Event causation and authenticated work provenance prevent another module or public emitter from forging progress.

While a call remains pending, its tool name must continue to resolve to a compatible logical capability after restart. Harness may report strictly additive tool names on a result for providers that support deferred tool loading; incompatible changes simply omit that optimization.

Durability and concurrency guarantees

  • Lane input, ignored prefixes, committed messages, open tool batches, and tool results survive process restart.
  • Work for one lane is partitioned so concurrent wakes never run concurrent turns for that lane.
  • Repeated wakes coalesce by lane ID without postponing an earlier scheduled wake.
  • Successful turns atomically append conversation output and consume the selected lane prefix.
  • Failed turns leave both conversation and pending input unchanged.
  • Tool calls are at-least-once and independently retryable.
  • One unresolved tool batch per lane is enforced by durable state.
  • Worker cancellation propagates into model and tool AbortSignals.

These guarantees depend on Sledge's single-owner ledger contract. Compose all Harness and application modules into one model and open one live Sledge ledger owner for a database handle.

Composition

Harness modules retain ordinary Sledge event and query capabilities. Application modules can observe committed turns and append new stimuli without coupling their domain state to Harness internals:

const model = composeLedgerModels(lanes, turns, memory, applicationDelivery);

This keeps four concerns separate:

  1. lanes own durable ordered input;
  2. Harness owns agent turns and tools;
  3. domain modules own application facts and projections;
  4. delivery modules decide which output creates new lane stimuli.

Experiments

The repository experiments are executable design tests, not just feature showcases. They test whether Harness's technical guarantees create useful conditions for the durable, emergent behavior described in TorkBot's VISION.md. An experiment can succeed mechanically and still expose an alignment failure such as scripted behavior, lost intent, hidden runtime state, or authority leakage.

Durable multi-persona discussion

experiments/discussion runs several independent personas in a shared channel. Every committed response is durably broadcast as new input to the other lanes; there is no central conversational turn scheduler.

Technical objectives:

  • prove the minimal three-module composition: lanes own ordered input, Harness owns model turns, and an application module owns cross-lane delivery;
  • exercise concurrent lane execution, deduplicated durable broadcasts, conversation history, cancellation, and model-session cleanup;
  • show that committed output can create follow-up stimuli and reach quiescence through Sledge work rather than application polling.

Alignment objectives:

  • test whether distinct agents can sustain a coherent discussion from their own context and incoming stimuli without a scripted turn order or response graph;
  • keep the runtime responsible for continuity and delivery without making it decide what an agent should say, which preserves room for emergent action;
  • reveal when the default dynamics produce useful exchange versus noise, repetition, premature silence, or runaway interaction.

Run it with the credentials configured for the Pi coding agent:

node experiments/discussion/run.ts

Project Saffron Arc

experiments/saffron-arc runs a six-party acquisition negotiation. Participants receive role-specific private dossiers, communicate in a shared room and direct messages, and use typed tools to register proposals, signatures, and impasse. The negotiation ends only through a natural irreversible domain event: matching authorized signatures on one term sheet or a declared impasse.

Technical objectives:

  • stress composed lane, turn, conversation, and negotiation modules across six concurrent agents with different prompts, tools, and private context;
  • exercise audience-scoped delivery, typed tool effects, proposal deduplication, terminal-state fencing, pause/resume cancellation, prompt-cache telemetry, and deterministic structured scoring;
  • prove that a retained ledger can drive the same projections in passive replay without starting workers or repeating model and tool calls;
  • make runtime state legible through a control room built from durable events, including activity, messages, proposals, terminal state, usage, and replay position.

Alignment objectives:

  • Emergent Action: put tension in asymmetric objectives, information, and authority, then let participants discover strategy and trade-offs instead of prescribing a negotiation procedure;
  • Continuity of Intent: preserve each participant's unresolved mandate as new messages, proposals, private disclosures, pauses, and resumptions change the situation;
  • Truthful Presence: derive the control room, terminal outcome, and score from durable facts rather than inferred progress or an LLM judge;
  • Authority and Safety: keep private dossiers and direct messages scoped to their intended participants, and require domain-authorized actions for binding outcomes;
  • Productive Default Path: use typed capabilities, durable consequences, and a mechanically scored terminal state to create gravity toward meaningful action without forcing agreement.

Retained runs also provide event-grounded evidence for improving the scenario and runtime. That supports learning from use, but the experiment does not claim that Harness itself learns from a run or that one negotiation validates the full TorkBot vision.

Run Saffron Arc and retain its ledger:

node experiments/saffron-arc/run.ts ./saffron-arc.sqlite

Replay a retained run without starting workers or making model calls:

node experiments/saffron-arc/replay.ts ./saffron-arc.sqlite

Development

npm install
node --run lint
node --run typecheck
node --run test

The tests cover lane-prefix authority, durable recovery, admission deadlines, backlog promotion, ignored input, model retries, tool batches, tool capability provenance, cancellation, replay, and lifecycle races.

About

A durable, resumable, model-agnostic agent harness using pi-ai and built on the @torkbot/sledge ledger.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages