SIL-Wheel Docs Agent#

A Wheel deployment carries a lot of surface area: a dozen search modes, several ways to combine them, and a set of workflows that only make sense once you know which mode does what. New users ask the same questions repeatedly, and those questions are almost always answered somewhere in these docs.

A small chat assistant, served alongside the Wheel UI and grounded in the documentation, closes that gap. This page describes how we built ours, so you can build an equivalent for your own deployment. The agent itself is not distributed, but nothing here is difficult to reproduce, and the design matters more than the code.

The decision that determines whether this works at all: the model is never asked what it knows about Wheel. It is given a read-only tool over the documentation source and told to answer only from what it reads.

This matters because an ungrounded model will confidently describe search modes that do not exist, invent parameter names, and get the ranking rules subtly wrong. Those answers are worse than no assistant at all, because they are plausible. Grounding it in the docs means the failure mode becomes “I could not find that”, which is honest and actionable. It also means the assistant improves whenever the docs improve, and never needs retraining when a feature ships.

Architecture#

The assistant is a standalone HTTP server, separate from the Wheel server. The UI talks to it over a single POST /chat endpoint; everything else is internal:

Wheel UI widget  ──POST /chat──►  assistant server
                                     │
                                     ├── agent loop  ──►  LLM backend
                                     │        ▲              │
                                     │        └── tool calls ┘
                                     │
                                     └── docs tool (read-only)
                                              │
                                              └── docs source tree

Keeping it out of the Wheel server matters more than it first appears. The assistant restarts when the docs change, holds an API key the Wheel server has no business holding, and fails independently. A deployment without it loses a chat widget and nothing else.

The Docs Tool#

The tool is deliberately small. Three read-only operations are enough:

list returns every doc with its name, title, and a one-line summary. This is what lets the model decide where to look without reading everything.
read returns one doc in full, by name.
search returns the docs matching a query string, with a snippet of the matching text from each.

Two details are worth copying. Strip the RST markup before handing text to the model, since directives and roles are noise that costs tokens and confuses smaller models. And build the doc index once at startup rather than walking the tree per request; the index is small and the docs do not change while the process is running.

Nothing writes. The tool cannot be made to modify a file, which removes an entire category of risk from putting an LLM behind a public endpoint.

The Agent Loop#

The loop is the standard one: send the conversation plus tool definitions to the model, and if it asks for a tool, run it, append the result, and send again. Repeat until the model answers instead of calling a tool.

Two bounds are worth setting from the start. Cap the number of iterations, so a model that loops on tool calls fails fast rather than burning an API budget. And sanitise the incoming conversation history, keeping only user and assistant turns with string content, since anything else in that payload came from a browser and should not be trusted.

Have the model name the docs it used, and rewrite those names into links to the rendered pages before returning the answer. Citations are what make the assistant trustworthy rather than merely convenient: a user who does not believe an answer can check it in one click, and a user whose question was only partly answered lands on the page covering the rest. They are also a cheap correctness check, since an answer citing nothing, or citing a page unrelated to the question, is one worth looking at.

Talk to the model through an OpenAI-compatible chat-completions interface and select the provider from whichever API key is present in the environment. That one indirection lets the same assistant run against a hosted endpoint, an internal inference service, or a local model, without touching the loop.

Logging Questions#

Log every turn: the question, the answer, the citations, which tools were called, the latency, and the error if it failed. SQLite is enough.

The value here is not observability, it is documentation feedback. The questions people ask an assistant are the questions your docs answer badly. Clusters of similar questions point straight at a missing section, and questions where the model found nothing point at a gap. It is the most direct signal you will get about which parts of your documentation are not working.

Class Design#

Four types carry the whole design. The signatures below are a starting point rather than a required interface, but the split between them is the part worth keeping: the tool knows nothing about models, the backend knows nothing about docs, and the agent is the only thing that knows about both.

class DocsTool:
    """Read-only access to the documentation source tree."""

    def __init__(self, docs_root: Path) -> None:
        """Walk the tree once and build the name -> path index."""

    def list_docs(self) -> list[dict]:
        """Every doc as {name, path, title, summary}."""

    def read_doc(self, name: str) -> dict:
        """One doc as {name, path, title, text}, markup stripped."""

    def search_docs(self, query: str, max_results: int = 5) -> list[dict]:
        """Matching docs as {name, path, title, snippet}."""

name is the doc’s path without its extension (search/caption_search), which is stable, readable in a citation, and maps to a URL with one substitution.

class ChatBackend(Protocol):
    """Anything that speaks OpenAI-style chat completions."""

    def chat(self, messages: list[dict], tools: list[dict]) -> ChatResponse:
        """Return the assistant message plus any tool calls it requested."""

Keeping this a protocol rather than a concrete class is what makes the provider swap free. A single implementation over an OpenAI-compatible HTTP API covers every hosted and local option worth using.

@dataclass
class AgentTurn:
    answer: str                      # final text, citations rewritten to links
    citations: list[str]             # doc names the answer drew on
    tool_invocations: list[dict]     # what was called, and whether it worked


class DocsAgent:
    """The tool loop."""

    def __init__(
        self,
        tool: DocsTool,
        backend: ChatBackend,
        max_iterations: int = 6,
    ) -> None: ...

    def chat(self, message: str, history: list[dict] | None = None) -> AgentTurn:
        """Send, run any requested tool, repeat until the model answers."""

Returning tool_invocations alongside the answer costs nothing and pays for itself the first time an answer is wrong: it shows whether the model looked in the wrong place or read the right page and still got it wrong. Those are different bugs, and one of them is yours.

What to Build First#

A useful assistant is a weekend of work, in roughly this order:

The docs tool, with list, read, and search over your docs source tree.
The agent loop, with a system prompt that says to answer only from the docs and to cite what it used.
A POST /chat endpoint and a plain HTML page that talks to it, which is enough to evaluate answer quality before you build any widget.
The QA log, ideally before you show the assistant to anyone, so the early questions are captured.
The UI widget, last, once the answers are good enough to be worth surfacing.

Points 1 to 3 are the assistant. Everything after that is polish.

SIL-Wheel Docs Agent