Retrieval that refuses to guess which law applies

Legal text is close to the worst case for ordinary retrieval-augmented generation. Similarity search across a corpus of statutes returns passages that read exactly right and are about the wrong law, because the vocabulary barely shifts between one area and the next.

A cheque dishonour notice, a breach of contract claim and a wrongful termination letter all talk about parties, obligations, deadlines, notice periods and remedies. Embedded as plain language they sit close together. Cosine distance cannot separate them, because what separates them is not how they are written — it is which statute governs them.

The failure that follows is the dangerous kind: quiet. The model does not decline or hedge. It produces a fluent, confident compliance analysis grounded in a statute that does not apply, and nothing in the output signals that anything went wrong.

Route first, retrieve second

Rather than embedding the corpus and searching across it, the system embeds only the category descriptions and routes the incoming document against those. Once a category wins, that category's rule text is fetched by its metadata ID — a direct lookup, not a search.

That inversion is the whole design. Because retrieval is a keyed fetch rather than a ranked query, it structurally cannot return a blend of two areas of law. It returns one category's rules, or it returns nothing at all.

A threshold with somewhere to fall

Routing returns top-K category matches with confidence scores, gated at a threshold. Below it, a no-confident-match branch fires and the document goes to a person instead of to the model — the system declines rather than picking the least bad of four wrong categories.

That threshold is a starting guess, and it is commented as one in the code. Turning a raw vector distance into a calibrated confidence is rough, and tuning it properly needs a labelled test set that does not exist yet.

Cost tiering, pointed the right way

Classification runs on local embeddings: no model call, no per-document inference cost. The single LLM call in the pipeline is the comparison step, and it deliberately uses the strongest model available, because that is the step doing the actual legal reasoning over the rule text and the submitted document.

Plenty of RAG systems get this backwards — a model is asked to choose what to retrieve, then a cheaper one writes the answer. Here the free step decides scope and the expensive step is reserved for the judgement.

The cheapest step decides which law applies. The expensive one only has to reason about the law it was handed.

The human gate is structural, not a setting

Every path through the graph terminates at a human review node — the confident ones included. Confidence in the routing step is not treated as licence to publish a verdict unseen.

Each rule document also carries a review_status field recording that it has not been checked by a licensed lawyer, and that flag is carried through to the output rather than stripped on the way to the reader. A caveat that only exists in the repository is not a caveat.

This is an in-house build in active development, not a finished product, and it is worth being exact about which half is which. The routing and comparison slice runs end to end; the wider assistant around it is designed and not yet written.

• Built: embedding-based category routing, with a confidence threshold and a no-match fallback
• Built: metadata retrieval, the comparison node, and the LangGraph pipeline wiring them together
• Built: a FastAPI service exposing classify, retrieve and combined endpoints, plus a Docker image that ingests the corpus at build time
• Corpus: four rule categories — cheque dishonour, breach of contract, tenancy eviction and wrongful termination — none reviewed by a licensed lawyer
• Not yet built: intake, drafting, summarisation and citation-verification agents, designed as further graph nodes
• Not yet built: a real human-in-the-loop — the review gate auto-approves today, pending a LangGraph interrupt
• Not yet measured: there is no evaluation set, so output is inspected rather than scored, and we will not quote an accuracy figure until it is