Repository navigation
Agent re-investigates things already in the conversation history, wasting tokens and time (especially on local LLMs) #12579
Description
Problem
When a user asks about something that was already done, read, or discussed earlier in the same session, the agent often starts a fresh investigation instead of answering from the conversation history. It re-reads files it has already read, re-runs searches it has already run, and can spend many agentic round-trips re-deriving information that is already in its own context window.
This is very costly on local / self-hosted LLMs. In a complex coding session, a simple recall question ("what did we just change?", "which file holds the logic we discussed?", "is the fix already in place?") can trigger several minutes of tool calls and a large number of tokens, only to produce an answer that was already present in the conversation.
Why it happens
The core system prompt is strongly biased toward investigation and verification:
Verify (Tests)/Verify (Standards)mandatesNEVER assume ... Verify its established usageContinue read-only investigationCodebase Searchguidance that routes questions to grep/glob/Explore
None of these is wrong on its own. But there is no directive that tells the model to first check the conversation history before launching an investigation. So every question — including a pure recall question — starts with the same "go look" bias, even when the answer is already in context.
The stateless agent loop already keeps the full conversation history, and that history is the working memory: the questions, the answers, the files read, and the decisions form enough context to answer many follow-up questions directly, with no tool calls.
Impact
- Token waste. Industry write-ups put coding agents at 80–99% of tokens spent re-reading files they already processed. A large share of that is re-reading within a session for questions the history could already answer.
- Latency. On a local model, each unnecessary tool round-trip is seconds; a chain of them is minutes for a question that should be instant.
- No new information. The investigation re-derives what is already in context, so the extra cost buys nothing.
Prior art
This is a known "when to retrieve vs. answer from memory" problem:
- Adaptive-RAG routes a query before retrieval: no retrieval (answer from context if already answerable) / single-step / multi-step.
- Self-RAG emits a retrieve-decision token and can skip retrieval entirely.
- Answer-from-context-first heuristics try the provided context first and retrieve only when confidence is low, the context is insufficient, or fresh facts are required.
The consistent lesson is that the right shape is a context-first bias with a staleness guardrail, not a hard rule — answering from stale context can be wrong (one reported benchmark had a model return outdated answers in 26 of 40 cases with zero tool calls).
Proposed fix
Add a context-first answering policy to the core system prompt:
- Treat the conversation history as working memory.
- Before launching any investigation (read/grep/glob/search or a research sub-agent), check whether the answer is already present in the history.
- If it is, and nothing since then has changed the facts it rests on, answer directly with no tool call.
- Investigate only when the history is genuinely insufficient, the user asks for the current / post-change state, the answer must be grounded in code or data not yet shown, or the remembered facts are not trusted.
- Keep the existing verify-before-done rule intact: the bias is about answering, not about claiming work is done.
- Carry an explicit stale-context guard: if the answer depends on state that may have changed since it was last observed (for example after an edit), re-check rather than trust the earlier snapshot.
This is a prompt-level policy (the "is the history sufficient?" judgment is inherently the model's), low-risk, and does not change the control flow. It should make the common case — a follow-up about recent work — immediate and near-zero-cost, while leaving real investigation untouched when it is actually needed.
Activity
doudouOUC commentedon Sep 23, 2026
Independent source verification of the premise against
packages/core/src/core/prompts.tsatmainHEADd0cd622a68. The reporter's central factual claim — the core system prompt is strongly biased toward investigation/verification and contains no directive telling the model to first consult the conversation history before launching one — is confirmed.Where the investigation bias lives (all in the assembled base prompt,
buildDefaultBasePrompt,prompts.ts:493-576)The base prompt is built by static string concatenation; there is no retrieval-routing or retrieve-decision logic at the prompt layer. Every section pushes the model toward "go look":
Core Mandates(prompts.ts:538-548)::539—Conventions: Never assume file contents. Read relevant code, imports, tests, and configuration before making changes.:547—Plan before uncertain work: ... Continue read-only investigation, make a plan in the current mode ...Software Engineering Tasks(getSoftwareEngineeringTasksSection,prompts.ts:402-450): the mandated iterative loop isPlan → Implement → Adapt → Verify (Tests) → Verify (Standards) → Report.:414Verify (Tests): NEVER assume standard test commands ... verify it actually works./:415Verify (Standards). "Read-only or explanatory turns do not require verification" (:415) is the only carve-out, and it gates verification, not investigation.Codebase Search(:458CodeMode /:481direct): routes lookups togrep/glob/Explore, with the only throttle being "use Explore only when >3 queries needed" — a cost heuristic, not a recall check.Final Reminder(:660):Keep going until the user's query is completely resolved— reinforces action over pause-to-recall.The "check history first" directive is genuinely absent
Grepping the agent-facing base prompt for
conversation history/working memory/context-first/answer from context/recall/before you (search|investigate|read)returns zero hits. The onlyhistory/conversation historymentions inprompts.tsare in prompts for other subprocesses, not the main agent loop:getCompressionPrompt(:~892-960) — the context-summarizer's system prompt; it consumes history to emit a<state_snapshot>, it does not gate investigation.getProjectSummaryPrompt(:961) — "analyze the conversation history above and generate a ... project summary" — same, a summarizer, not the interactive loop.So a pure recall follow-up ("which file holds the logic we just discussed?") enters a loop whose every directive says "read/grep/verify," with nothing telling the model the answer may already be in its own context. The behavior the reporter describes is the predictable consequence.
One interaction worth scoping the fix against
## Git as Source of Truth(prompts.ts:646-648) is the one place the prompt already carries an anti-staleness guard — but it points the opposite direction::647—Git history ... git log / git blame are authoritative. Do NOT rely on memory or assumption when you need to know what changed. Always run the command.:648—If asked about recent or current state of the codebase, prefer git log or reading the code over any cached assumption. A memory or snapshot is frozen in time.This is legitimately scoped (git state is authoritative vs a frozen in-context snapshot, and the working tree may have changed externally), but it contributes to the general "go look" reflex and is the case where a naive "answer from history first" policy would be wrong. The reporter's proposed point 6 (stale-context guard: re-check when state may have changed since last observed, e.g. after an edit) is exactly the carve-out that keeps a context-first bias from regressing here — worth calling out as the load-bearing constraint, since
Git as Source of Truthis the existing directive a new policy must compose with rather than override.Verdict
Premise confirmed against source: the gap is real and prompt-level (no control-flow change required, as the reporter states — the "is the history sufficient?" judgment is inherently the model's). The proposed
context-first answering policyis a sound shape for it, with the scoping note above: the staleness guardrail must explicitly defer toGit as Source of Truth(:646-648) for git/current-state questions, and the existingRead-only or explanatory turns do not require verificationcarve-out (:415) is the natural anchor point for the "answer directly, no tool call" branch.Andrea-Bruno commentedon Sep 24, 2026
Thank you for the careful source verification and for naming the load-bearing constraint. I agree that the context-first bias must compose with the existing Git as Source of Truth rule rather than override it, and that the read-only carve-out is the natural anchor for the answer-directly branch.
Pull request #12580 adds the Answer From Context First mandate. Following your point I just pushed a refinement (commit 076365b) so the bullet states explicitly that for questions about git history or the current state of the repository the model should still run git log or git blame or read the file, instead of answering from a remembered snapshot. That keeps the new bias from regressing the one place the prompt already guards against staleness. On the prompt-budget concern raised in triage, the change is a single mandate kept deliberately short, and it reuses the existing carve-out rather than adding a new always-on section, so the added cost is small.