Skip to content

Add skills as slash commands - #711

Draft
hanna-paasivirta wants to merge 14 commits into
mainfrom
slash-command-preset
Draft

hanna-paasivirta wants to merge 14 commits into
mainfrom
slash-command-preset

Conversation

@hanna-paasivirta

@hanna-paasivirta hanna-paasivirta commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Short Description

Adds skills to global chat: /design, /qa and /diagnose. A user can run one as a slash command, or the planner can load one when the task matches. Skills can also be written for the job code agent, which the planner can hand over by name or the job agent can load itself.

Decisions

  • Skills stay in the conversation beyond the turn they're invoked on, and it's fine for them to stay from then on. There is no skill mode that ends: the instructions just stay in history, which is also how Claude Code handles skills. In testing, later unrelated turns were answered normally.
  • Apollo owns the history. Lightning stores what Apollo returns and sends it back unchanged.
  • The client recognises the slash command and names it in skill. Apollo never parses commands out of the message.
  • Both the user and the model can invoke a skill: by slash command, or through a load_skill tool. The planner loads a skill when the task matches, not only when the user's wording does, so a failed run attached to "hi" is diagnosed with /diagnose.
  • Skills use Anthropic's standard Agent Skills format, one folder per skill (services/global_chat/skills/<name>/SKILL.md). Only SKILL.md is read for now, but the folder is there for reference files later.
  • All skills live in one flat folder in global_chat, the service that loads them. A skill can be written for the job code agent (metadata: {agent: job_code} in its frontmatter). The planner can name it on a job agent call, so fixed instructions reach the job agent word for word instead of being rewritten on every call, and the job agent can also load it itself, e.g. when a user asks straight from a step page whether the step is ready to go live. Each agent sees only its own skills.
  • /qa splits this way: its per-step review lives in qa-code, which a future skill (say /migrate) can reuse. The planner reads the whole workflow first and settles the contracts between steps, then has every step reviewed and fixed in one parallel pass, then checks the result. The planner's message carries only what the job agent can't know (contracts, that other steps are being fixed at the same time, and what the user asked, such as "leave comments alone") and overrides the skill where they conflict.
  • The workflow agent can't load skills yet: it has no tool loop. We'll add that when a skill is written for it.
  • Lightning keeps its own list of skill names for the slash menu for now. It needs a way to sync with Apollo's list, which we'll add once this work is validated, to avoid complexity we may not need and get this out for testing sooner.

Implementation Details

  • The client sends skill: {name} on the turn the user types the slash command. That turn skips the router and goes to the planner, with the skill's instructions ahead of the message.
  • The planner has a new load_skill tool, so it can pick up a skill when the user didn't type the command. Its prompt now mentions skills alongside tools.
  • Skills are kept in the returned history, so a skill like /design keeps working over several turns. While a skill is in the history, every turn goes to the planner, because the other agents never saw it.
  • Planner turns now get the [pg:...] page prefix in history, like the other routes, so all routes return history in the same shape.
  • call_job_code_agent has an optional skill parameter, listing only job-agent skills. Code puts the named skill ahead of the planner's message. Job-agent skills can't be invoked by slash command or loaded by the planner.
  • job_chat takes a skills list in its payload, which global_chat sends on both the planner and the router's direct route (leaving out a skill the planner already attached). job_chat offers them through its own load_skill tool, which continues its tool loop the way inspect_job_code does, and its per-turn reminder mentions skills when it has any.
  • The response meta.skills lists every skill a turn used: invoked, loaded by the planner, attached to a job agent call, or loaded by the job agent itself. Each becomes a skill:<name> tag on the Langfuse trace (checked on a real traced turn), so traces can be filtered by skill.
  • Adds tests for invoking and loading skills, keeping them in history, routing follow-up turns, attaching job-agent skills, job_chat loading its own, and reporting which were used.
  • Adds four acceptance specs in global_chat/tests/acceptance/skills/ for the core skill behaviours, rather than one per skill: the job agent reviewing a step it's handed as done, the same step with one specific change (no full review), /design still consulting on its second turn, and /qa end to end (cross-step fix, comments left alone, adaptor pinned to the tested version). All four pass. The acceptance output now shows the skills used next to the agent path.

Required Lightning changes

These are in OpenFn/lightning#5236.

  • Store the history Apollo returns exactly as received (new apollo_history column), and send it back as history on the next request. Don't rebuild it from messages, or the skill instructions and page prefixes are lost. Old sessions can still fall back to rebuilding.
  • Read history from the final complete event of workflow_chat and global_chat too, not only job_chat. It's there for all three.
  • Send skill: {name} only on the turn the user invokes the slash command, not on later turns.
  • Add design to Lightning's skill list. Don't add qa-code; it's for the job code agent, not users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant