inferCapabilities is untested in the honesty direction — llama-cpp embedding GGUFs now advertise cache.checkpoint

[sroussey]

inferAdvertisesRegistered.test.ts (new in 0.3.47) asserts registered ⊆ inferred and unions the inferred sets across every fixture id for a provider. That is the right shape for "is this capability reachable at all" and deliberately not for "is it honest for this model". capabilityHonesty.ts does not fill the gap: it tests declared-capability ↔ registry lookup, which an over-advertised branch passes trivially — the run-fn is registered, it just cannot serve that model.

Live instance introduced by the same change: LlamaCpp_Capabilities.ts:41-50 (the embedding branch) now emits cache.checkpoint and session.dispose. Path:

inferLlamaCppCapabilities("nomic-embed-text-v1.5.Q4_K_M.gguf")  → cache.checkpoint
modelMeetsRequires(model, CacheCheckpointTask.requires)  (AiTask.ts:80-85) → true
LlamaCpp_CacheCheckpoint_Stream (LlamaCpp_CacheCheckpoint.ts:77-84)
  → new LlamaChatSession(...) + setChatHistory() on a model with no chat wrapper

Before, that model failed selection with a capability message; now it fails inside the provider. The generative branch alone satisfies the test (both fixtures pass at :155-160), so this was not required.

This is the repo's own "advertises true while every call throws" anti-pattern — the one the conformance test was written to eliminate — arriving inside the fix for its mirror image.

Fix:

1. drop "cache.checkpoint" (:44) and "session.dispose" (:49) from the embedding branch;
2. add the paired assertion (~20 LOC beside assertInferAdvertisesRegistered): for each fixture id, every inferred capability must have a registered run-fn whose serves set is a subset of that id's inferred set.

Found during the 2026-08-24 review. Snapshot: workglow-dev/prd → analysis/grades/2026-08-24/libs-providers.md.

[sroussey]

2026-08-31 review — still open, re-verified by execution (cycle 2).

git diff afef1c1091ca49c3f6ae7bdc871b2d48ba359319..9a80450 -- providers/node-llama-cpp/src packages/test/src/contract/ai-provider/ returns empty. Both files named in this issue are byte-identical to when it was filed.

providers/node-llama-cpp/src/ai/common/LlamaCpp_Capabilities.ts:41-51 at HEAD still emits "cache.checkpoint" (:44) and "session.dispose" (:49) from the embedding branch.

This cycle the finding was measured rather than read — importing the real module, no fixture:

$ bun probe.ts
Mistral-7B-Instruct-v0.3.Q4_K_M.gguf -> HAS cache.checkpoint
nomic-embed-text-v1.5.Q4_K_M.gguf    -> HAS cache.checkpoint

The consumer path is unchanged: packages/ai/src/task/CacheCheckpointTask.ts:117 declares requires = ["cache.checkpoint"], and the run-fn it selects — providers/node-llama-cpp/src/ai/common/LlamaCpp_CacheCheckpoint.ts:76-92 — calls getOrCreateTextContext(model), then new LlamaChatSession({contextSequence, …}), then chatSession.setChatHistory(history) and chatSession.preloadPrompt(""). Selection succeeds; the call throws inside the provider.

The conformance assertion is also unchanged. packages/test/src/contract/ai-provider/assertions/inferAdvertisesRegistered.ts:19-21:

const registeredCaps = new Set(registered.flat());
const inferredCaps   = new Set(inferred.flat());   // union across every fixture id
const missing = [...registeredCaps].filter((cap) => !inferredCaps.has(cap)).sort();

Both .flat() calls collapse the per-id structure before comparison, so the property asserted is ⋃ᵢ registeredᵢ ⊆ ⋃ⱼ inferredⱼ. Deleting :44 and :49 leaves the test green, because the generative fixture supplies both capabilities — which is the proof the over-advertisement was never needed to satisfy it.

One thing that did land this window is a useful template for the second half of this issue. packages/test/src/test/ai-provider/provider-model-search.test.ts:234-249 (commit 9ada0ea) added a new conformance axis — "search offers a model for every inference capability it serves" — with the lifecycle exclusions named and justified at :224-232, and it found a real gap (vision.face-detection served with no catalog model). The paired per-id honesty assertion asked for here can be written the same way, beside assertInferAdvertisesRegistered.

Still DO-NOT-CLOSE. Both asks remain: (1) delete the two capabilities from the embedding branch, (2) add the per-fixture-id honesty assertion.

---

Generated by Claude Code