Skip to content

fix(backends): send the CLI/API contracts these backends actually have - #243

Merged
Yifan Yang (Yif-Yang) merged 1 commit into
microsoft:mainfrom
Yif-Yang:fix/backend-cli-contracts
Aug 21, 2026
Merged

fix(backends): send the CLI/API contracts these backends actually have#243
Yifan Yang (Yif-Yang) merged 1 commit into
microsoft:mainfrom
Yif-Yang:fix/backend-cli-contracts

Conversation

@Yif-Yang

Copy link
Copy Markdown
Contributor

Follow-up to #170 and #190. Both PRs moved in the right direction, and their test scaffolding is kept — but each shipped a parameter its target does not accept, so as merged neither backend does what it says. This corrects both against the real contracts.

Copilot — skillopt_sleep/backend.py

--allowed-tools is not a GitHub Copilot CLI option:

$ copilot --allowed-tools=Bash -p hi
error: unknown option '--allowed-tools=Bash'

Both Copilot call paths (_call and the task-replay path) sent it, so both aborted before doing any work.

Two independent axes were also conflated:

  • --allow-all-tools waives the interactive approval prompt. The CLI's own help says it is "required for non-interactive mode" — dropping it breaks headless runs. This is a permission flag, not a scoping flag.
  • --available-tools is the visibility axis: "Only these tools will be available to the model." This is where scoping belongs.

So: keep --allow-all-tools, scope with --available-tools (default bash, override via COPILOT_AVAILABLE_TOOLS).

The selector is case-sensitive, which is easy to get wrong. Verified live against Copilot CLI 1.0.80:

argv result
--available-tools bash "toolName":"bash" — tool runs
--available-tools Bash tool never appears — silently blocked
--available-tools write bash blocked — scoping demonstrably works

That last row is the control: the flag really does narrow the surface, rather than being accepted and ignored.

MiniMax — skillopt/model/minimax_backend.py

chat_template_kwargs.enable_thinking is a Qwen/HuggingFace-serving convention. It appears nowhere in MiniMax's OpenAI-compatible reference, so the endpoint ignores it and thinking stays at the server default no matter what the flag says.

This predates #190 — the field was already on main; #190 built model-aware logic on top of it, so the routing worked but the value never reached the model.

The documented control is the top-level thinking object:

{"thinking": {"type": "adaptive"}}
{"thinking": {"type": "disabled"}}

Also per the docs, the M2.x family accepts {"type": "disabled"} but keeps thinking on regardless. Sending disabled there would record a request that misrepresents what the model actually did, so M2.x is sent adaptive. This preserves #190's intent (M2.7 keeps always-on thinking) without a value that lies. Unknown deployments default to adaptive, matching the documented API default of thinking-on-when-omitted.

Reference: https://platform.minimax.io/docs/api-reference/text-openai-api

Tests

Both suites now assert the exact wire payload / argv rather than the shape the code happens to produce, plus explicit regression guards (test_nonexistent_allowed_tools_flag_is_never_sent, test_unsupported_chat_template_kwargs_is_never_sent).

  • Full suite: 1389 passed, 10 skipped, 269 subtests
  • Negative control: reverting each fix fails exactly the 9 tests that assert it, and nothing else — the tests exercise the new behavior rather than passing incidentally.

Two backends were emitting parameters that their target does not accept, so
both silently did nothing (or aborted) instead of what the code intended.

Copilot (skillopt_sleep/backend.py)
-----------------------------------
`--allowed-tools` is not a GitHub Copilot CLI option; the CLI exits with
`error: unknown option '--allowed-tools'` before doing any work, so both
Copilot call paths were dead. Two independent axes were also conflated:

* `--allow-all-tools` waives the interactive approval prompt and the CLI's
  own help calls it "required for non-interactive mode" -- removing it breaks
  headless runs, so it is not the flag to scope on.
* `--available-tools` is the visibility axis: "Only these tools will be
  available to the model". That is where scoping belongs.

Keep the former, scope with the latter. The selector is also case-sensitive:
verified against Copilot CLI 1.0.80 that `--available-tools=bash` lets the
tool run while `--available-tools=Bash` blocks it, so the default is
lowercase `bash` (override: COPILOT_AVAILABLE_TOOLS).

MiniMax (skillopt/model/minimax_backend.py)
-------------------------------------------
`chat_template_kwargs.enable_thinking` is a Qwen/HuggingFace-serving
convention that appears nowhere in MiniMax's OpenAI-compatible reference, so
the endpoint ignored it and thinking stayed at the server default regardless
of the configured flag. Send the documented top-level field instead:
`{"thinking": {"type": "adaptive" | "disabled"}}`.

M2.x accepts `{"type": "disabled"}` but keeps thinking on anyway, so it is
sent `adaptive` rather than a value that misrepresents what the model does.
Unknown deployments default to `adaptive`, matching the documented API
default of thinking-on-when-omitted.

Tests assert the exact wire payload / argv, and each new test fails when its
fix is reverted.
@Yif-Yang
Yifan Yang (Yif-Yang) merged commit 10fd848 into microsoft:main Aug 21, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant