development
Your AI Agent Is Mr Magoo: Why Knowledge Search Changes Everything
The Convention Your AI Never Saw
Picture this. You ask your AI coding assistant to add a new API endpoint. It writes clean, well-structured code. Tests pass. Types check. The code is, by any reasonable standard, good.
There’s just one problem: your project registers all endpoints through a central router with a specific middleware chain, a naming convention for handler files, and a validation pattern documented in a spec that lives three directories away. The AI didn’t know any of that. It wrote a perfectly valid endpoint that violates every convention your team established.
This isn’t a hallucination. It’s not a bug. It’s a knowledge gap.
Your AI is Mr Magoo. Supremely competent, well-intentioned, and confidently striding forward — but short-sighted. It can’t see what it hasn’t been shown. It’s not making mistakes because it’s stupid. It’s making mistakes because it’s navigating your codebase with reading glasses that only reach to the end of its context window, cheerfully rearranging furniture it can’t quite make out.
And here’s the uncomfortable truth I keep arriving at: an agent that knows your conventions produces better code than a smarter agent that doesn’t. The prescription isn’t a better model. It’s better glasses.
Why Context Windows Don’t Scale
The obvious fix is to put everything in context. Write a massive CLAUDE.md. Stuff it with every convention, every pattern, every decision you’ve ever made. Problem solved.
Except it isn’t. Our CLAUDE.md for Injectionator — a security tool for AI applications — runs to 730 lines, and it still can’t hold the contents of 417 project documents. Five spec documents alone cover inspectors, API surfaces, system architecture, frontend contracts, and multi-tenant isolation. Add the feature docs, decision records, architecture guides, and operational runbooks, and you’re looking at a small library.
The phone book analogy is useful here. You don’t read the entire directory to find one number. You look it up. Cramming everything into context is like photocopying the phone book and handing it to someone every time they need to make a call. It’s expensive, it’s mostly irrelevant, and the person still has to find the right entry in a wall of text.
Context windows are getting bigger. That doesn’t help. A bigger phone book is still a phone book. What you need is search.
The Missing Piece
In a previous article, I described four layers for keeping AI-generated code coherent: hooks for real-time correction, specs as source of truth, agents for specialist review, and skills for on-demand checks. But I glossed over a question that kept nagging: how do the agents know things?
A drift detector that compares specs against code needs to find the right spec. An architecture reviewer needs to know what patterns are established before judging violations. An inspector generator needs to discover existing coverage before creating duplicates. All of them need knowledge that isn’t in their context window and isn’t in CLAUDE.md.
The answer turned out to be a knowledge layer — searchable project documentation that agents query on demand, the way a researcher queries a library. Not reading every book. Just finding the right one at the right moment.
What Search Already Exists (and Where It Stops)
Our tool of choice for building Injectionator is Claude Code — Anthropic’s CLI agent that lives in your terminal and works directly with your codebase. It’s not an autocomplete widget. It’s an agent that reads files, runs commands, and writes code. And it already has search.
Claude Code can use Bash tools like grep and ripgrep to search your repository. It does this constantly — finding function definitions, tracing imports, locating files by name. When you ask it to add a feature, it’ll grep for related code, read the matches, and use what it finds to inform its work. This is genuinely useful. It’s how the agent discovers your codebase’s structure in real time.
But grep is keyword search over source code. It finds exact text matches — strings, function names, file patterns. It returns chunks: lines of code surrounding a match, stripped of broader context. If you grep for “inspector” in Injectionator, you get 200+ hits across dozens of files. Technically comprehensive. Practically overwhelming. And it tells you what exists without telling you why it exists, how it should be used, or what decisions shaped it.
This is the progression that matters:
Context windows give the agent what you put in front of it — CLAUDE.md, the current file, whatever fits. Static. Limited. The phone book approach.
Grep and file search let the agent explore the codebase on demand. Dynamic. Powerful for code. But keyword-only, code-only, and context-free — you get matching lines, not architectural understanding.
A knowledge layer adds something neither can provide: semantic search over authored documentation. Not “find every line containing this string” but “find the document that explains this concept.” Not code fragments but decisions, conventions, rationale — the why behind the what.
From Keyword Chunks to Semantic Documents
The difference between grep and knowledge search isn’t just what’s indexed — it’s how search works.
Grep is literal. Search for “tenant isolation” and you find files containing those exact words. If your spec calls it “data partitioning” or “passport-scoped queries,” grep misses it entirely. You get fragments — a few lines around each match — and you’re left to reconstruct the full picture from scattered pieces.
Semantic search understands meaning. Search for “how does tenant isolation work?” and it finds the multi-tenant spec, the passport documentation, and the data partitioning guide — even if none of them contain the phrase “tenant isolation.” It returns whole documents ranked by relevance, not line-numbered fragments. The agent gets the authored explanation, not a puzzle to reassemble.
For Injectionator, this means a local search engine called QMD that indexes all project documentation and exposes it via MCP (Model Context Protocol) — the standard that lets AI tools call external services. The agent doesn’t need to know where the docs live on disk or how the index works. It just searches, the same way it calls grep — but the results are qualitatively different.
Both types of search work together. Keyword search (BM25) handles exact terms — spec identifiers, inspector names, file references. Semantic search (vector embeddings) handles conceptual questions — “what’s the convention for error handling?” or “why does the scan pipeline use copy-on-write?” Combining both in a single query produces results that neither achieves alone.
The concrete numbers: 417 documents indexed across all Injectionator repositories. 5 spec documents. 15 registered inspectors. 100+ API endpoints. 13 packages. All searchable in milliseconds, all available to any agent that asks.
Grep isn’t replaced. It’s still the right tool for finding code. But for finding knowledge — the decisions, patterns, and conventions that should shape the code — you need something that searches meaning, not just strings.
”Isn’t This Just RAG?”
Yes — and no. The retrieval pattern is the same one that enterprise AI teams spend months building: index documents, search at query time, inject relevant context into the AI’s prompt. If you’ve heard the term Retrieval-Augmented Generation, this is that pattern.
But the implementation is fundamentally different from the RAG pipelines you see in conference talks and vendor demos.
Local-first. SQLite database, local vector embeddings. Your documentation never leaves your machine. No API keys for a cloud vector database, no managed service to configure, no data residency concerns.
Developer-controlled. You choose what’s indexed — markdown documentation, not source code. You tune the queries. You decide when agents should search the knowledge base versus reading files directly. The developer stays in the loop, not the platform.
Embedded in the toolchain. Exposed via MCP, the same protocol the AI agent uses for everything else. The agent calls it like any other tool — not a separate API layer, not a preprocessing step, not a pipeline that runs before the conversation starts.
Zero-latency feedback loop. When you update a document, the local index picks it up on its next scan. No deployment pipeline. No embedding job to queue. No waiting for a nightly batch process.
The insight I keep coming back to: RAG’s power isn’t in the infrastructure. It’s in the pattern. You don’t need Pinecone and a vector pipeline to give your AI agent project knowledge. You need a local search index and a way for the agent to call it. The simplest implementation that works is the one you’ll actually maintain.
Where It Changes the Game
Theory is cheap. Here’s what actually happened when we wired knowledge search into the development workflow.
The Count That Was Wrong
We were building an inventory — a cross-reference of every feature, every file, every inspector in the system. The research agents used Grep to scan the codebase and reported 18 inspectors. Technically correct: 18 inspector files exist on disk.
But wrong. Only 15 are registered inspectors. The other 3 are utility modules — shared code that inspectors use but that aren’t inspectors themselves. The spec document (SPEC-INSPECTOR-REGISTRY) is clear about this distinction: 15 registered, 3 unregistered utilities, 1 attack-capable.
When we re-ran the inventory with QMD search, the agent queried the spec instead of counting files. It got the authoritative answer. It also caught a missing KV binding, a wrong source file path, and a spec drift on the control-plane routing — none of which Grep would have found, because none of them were in the code.
The difference isn’t intelligence. The same model produced both results. The difference is knowledge.
The Feature That Almost Duplicated
A developer asked for a SQL injection detection inspector. Without the knowledge layer, the agent would have built one from scratch — reasonable, well-structured, fully functional, and entirely redundant.
With QMD, the agent searched first. It found that a code-injection declarative inspector already existed in the seeded library, complete with SQL injection patterns. It came back with a question instead of code: “A code-injection inspector already covers SQL patterns. Do you want to extend it or create a separate one?”
That question saved hours of work and prevented a maintenance burden that would have compounded forever.
The Drift That Wasn’t
The drift detector found a mismatch — an inspector registered in code but missing from the spec. Before the knowledge layer, this would have been flagged as drift requiring a spec update.
With QMD, the agent searched for the inspector name and found a journal entry explaining the intentional divergence: the inspector was experimental, deliberately excluded from the spec until it proved stable. Instead of a false positive requiring investigation, the report noted: “Undocumented — intentional, see journal entry.” The developer moved on.
The Self-Reinforcing Loop
Here’s the part that surprised me. The knowledge base doesn’t just serve agents — it gets better because of them.
The workflow creates a closed loop:
- A skill like
/add-inspectorcreates a new inspector and updates the spec as its final step. - The search engine re-indexes the updated spec.
- The next agent that searches for inspector coverage finds the new entry.
- The drift detector validates that code and spec agree.
No separate “documentation sync” step. No reminder to update the wiki. No documentation that decays on write. The same workflow that creates the code also creates the knowledge, and the same index that serves the knowledge also validates the code.
Contrast this with traditional documentation. You write an architecture doc. Two weeks later, the code has evolved. The doc is now historical fiction. Someone reads it, trusts it, builds on it, and discovers the hard way that reality diverged. We’ve all lived this. The documentation problem isn’t that people don’t write docs — it’s that docs and code live in different systems with no mechanism to keep them aligned.
When docs are indexed and searchable, and when the workflow that changes code also changes docs, the alignment isn’t maintained by discipline. It’s maintained by design.
What It Doesn’t Do
I’ve learned to be honest about limits, so here they are.
It doesn’t index source code. The knowledge layer searches documentation — markdown files that describe conventions, decisions, and architecture. For code search, you still use Grep and Glob. These are different tools for different questions: “What does this function do?” is a code question. “Why does this function exist?” is a knowledge question.
It doesn’t enforce constraints. Finding the right convention and enforcing it are different acts. The knowledge layer tells the agent how things should work. Hooks prevent the agent from doing them wrong. Knowledge informs. Enforcement constrains. You need both.
It doesn’t replace reading files. Search helps you find the right document. You still need to read it. The knowledge layer is a librarian, not a substitute for the library.
It requires investment in writing good docs. This is the one people don’t want to hear. The knowledge layer is only as good as the knowledge you put into it. If your docs are sparse, outdated, or vague, search returns sparse, outdated, or vague results. The system amplifies documentation quality — in both directions.
How to Start
You don’t need our specific toolchain. The pattern is general. Here’s the minimum viable version:
1. Index your docs with a search tool. Any tool that indexes markdown and exposes search will work. The key requirement is that your AI agent can call it — an MCP server, a command-line tool the agent can invoke, even a local API. The mechanism matters less than the accessibility.
2. Add a “Knowledge Base” section to your system prompt. Tell the AI what’s indexed, how to search it, and which collection to scope to. Without explicit instructions, the AI won’t know the knowledge base exists. This is the step most people skip, and it’s the one that matters most.
3. Be specific about when to search. “Search when needed” is too vague. “Before adding a new endpoint, search for the API surface spec” is actionable. “Before flagging drift, search for decision records that explain the divergence” gives the agent a concrete trigger. The specificity of the instruction determines the quality of the behaviour.
That’s enough to feel the difference. You’ll know it’s working when the AI starts asking better questions — when it comes back with “I found an existing implementation that covers this” instead of building from scratch, or “the spec says this should use middleware pattern X” instead of guessing.
How We Actually Set This Up
The general pattern above works with any tools. Here’s the specific implementation we used for Injectionator, step by step, if you want to replicate it.
Step 1: Install QMD
QMD is a local search engine that indexes markdown files and exposes them via MCP. Install it as a Claude Code plugin. It runs as a background process on your machine — no cloud dependency, your docs stay local.
Step 2: Add Collections
Point QMD at your documentation directories. Each collection watches a path recursively for .md files:
injectionator → ~/dev/injectionator (417 docs across all repos)
viewyonder → ~/dev/viewyonder (1,052 blog/research docs)Collections can overlap (a subdirectory collection is a scoped subset of a parent). Choose granularity based on how you want to scope searches — collections: ["injectionator"] keeps results relevant to the project.
Step 3: Register the MCP Server with Claude Code
Add QMD as a plugin MCP server in your Claude Code configuration. Once registered, Claude Code discovers the QMD tools (query, get, multi_get, status) automatically and makes them available in conversations and to agents.
Step 4: Tell CLAUDE.md to Use It
This is the most important step. Without explicit instructions, the AI won’t know QMD exists. Add a “Knowledge Base” section to your CLAUDE.md that tells the AI:
- Which collection to search (always pass the
collectionsparameter) - Which query types to use (lex for exact terms, vec for concepts)
- When to search (before making assumptions, when a doc is referenced but unread, when unsure about patterns)
This covers the main conversation. But agents run as subprocesses with separate context…
Step 5: Add QMD Guidance to Agents
Each .claude/agents/*.md file that does research-heavy work gets a ## Knowledge Base (QMD) section. The key is being specific about scenarios, not generic. Don’t say “search when needed.” Say:
- “Before flagging drift, search for decision records that explain the divergence”
- “Before creating an inspector, search for existing coverage of the detection area”
- “When reviewing unfamiliar code, query for its feature documentation”
We added QMD sections to 4 agents: drift-detector, architecture-reviewer, inspector-generator, code-reviewer.
Step 6: Test It
The proof is in the results. We tested QMD integration during the inventory build — the same one described in “The Count That Was Wrong” above. The pattern repeats: without the knowledge layer, agents count files and report technically-correct-but-wrong answers. With it, they query authoritative specs and get the real answer.
Step 7: Let It Compound
The system gets better as you use it:
- More docs = better search. Every SPEC update, feature doc, or decision record you write is automatically indexed and searchable by all agents.
- Agent feedback sharpens prompts. When an agent’s QMD query returns irrelevant results, you refine its scenario instructions.
- The feedback loop closes. Skills update SPECs → QMD re-indexes → agents search updated docs → drift detector validates. No separate “documentation sync” step.
The Knowledge Bottleneck
There’s a term gaining traction: context engineering — the discipline of shaping what an AI knows at the moment it acts. Not prompt engineering (how you ask the question) but context engineering (what the AI has access to when it answers). System prompts, tool configurations, retrieval systems, memory — everything that determines the information available to the model at inference time.
Everything in this article is context engineering. The CLAUDE.md is context engineering. Grep is context engineering. The knowledge layer is context engineering. The difference is the quality and type of context each provides. A system prompt gives the agent static instructions. Grep gives it dynamic code fragments. A knowledge layer gives it authored understanding — decisions, conventions, rationale — retrieved by meaning, not just by keyword.
We’ve spent a lot of energy making AI smarter — bigger models, longer context windows, better reasoning. And those improvements matter. But in practice, the bottleneck in AI-assisted development isn’t intelligence. It’s context. The model that reasons about your codebase but can’t find your architectural decisions is like a brilliant consultant who wasn’t briefed. Sophisticated work that misses the point — not because they’re incapable, but because nobody told them what the point was.
The knowledge layer is the briefing. It turns your AI from a general-purpose code generator into something that understands your project — your conventions, your decisions, your reasons. Not because it memorised them (context windows are too small for that), but because it can look them up. That’s context engineering at its most practical: not crafting the perfect prompt, but building the infrastructure that delivers the right knowledge at the right moment.
The investment is writing docs worth searching and telling your AI to search them. The payoff is code that fits — that follows conventions it was never explicitly taught, that respects decisions it was never shown, that asks informed questions instead of making uninformed assumptions.
Your AI isn’t Mr Magoo because it’s stupid. It’s Mr Magoo because you haven’t given it glasses.
Give it glasses.
Tools Mentioned
Injectionator — A security tool for AI applications that detects and blocks prompt injection attacks. Built with Claude Code as a solo developer project, it uses a dual-LLM scanning pipeline with trust scoring, multiple inspection stations, and four operational modes. The coherence system and knowledge layer described in this article were developed to manage its growing complexity.
QMD — A local search engine for markdown documents. QMD indexes directories of markdown files, builds both BM25 keyword and semantic vector search indexes, and exposes them via MCP (Model Context Protocol). It runs entirely on your machine — SQLite database, local embeddings, no cloud dependency. Designed to give AI coding agents searchable access to project documentation.
Steve Chambers is the founder of Viewyonder, a GenAI consultancy helping teams build coherence systems for AI-assisted development. He built the knowledge layer described here while developing Injectionator, and discovered that the hardest part wasn’t the technology — it was writing documentation worth searching. If your team is building with AI and feeling the knowledge gap, get in touch.