Legal Document Assistant for Meridian Legal

1. Executive summary

Meridian should build a routed retrieval workflow, not an agent. Lawyers ask questions and get cited answers over 40 million documents; the task is retrieve-then-synthesize with no autonomous actions, which sits at rung 2 of the escalation ladder [knowledge/decision-trees.md]. Adding an agent loop would add cost, latency, and unpredictability with no benefit, so the anti-escalation rule rejects it.

The entire design turns on one requirement: zero confidentiality breaches. Ethical walls change daily as conflicts arise, so entitlements must be enforced at the retriever under the caller identity and refreshed daily, never inferred by the model or embedded in a prompt [knowledge/rag-patterns.md]. The system is strictly read-only (tier 0), every answer carries source citations, and an unsourced claim is refused rather than shown. At 2,500 queries per day the design runs well inside the 25,000 USD per month ceiling, dominated by vector index infrastructure rather than model spend.

Recommendation Build a workflow, not an agent. Enforce ethical walls at retrieval time against a daily-synced ACL, verify every citation, and keep the system read-only.

2. Requirements and NFRs

Answer questions across matter files, contracts, briefs, and firm precedent, with citations to source documents.
Summarize documents on request.
Read-only against the document management system. Draft nothing that leaves the firm. Give no legal advice to clients.

Non-goals: no document creation, no client-facing output, no writes of any kind.

NFR	Target	Source
Throughput	2,500 queries/day peak, bursty around filings	customer
Latency	First token < 3s, complex query < 15s p95	customer
Availability	99.5% (business hours), RTO 8h	customer
Corpus	40M documents, +30k/week, variable OCR quality	customer
Cost	≤ 25,000 USD/month all in	customer
Compliance	ABA Model Rule 1.6, GDPR (EU matters)	customer
Confidentiality	Zero cross-matter leaks (hard requirement)	customer

The requirement that dominates A single confidentiality breach is career-ending for the lawyer and firm. Confidentiality is therefore designed as a hard invariant enforced in the architecture, not a quality metric to optimize.

3. Decision record

Gate	Decision	Why, and what lost
Need generative AI?	Yes, for synthesis	Lawyers need composed, cited answers across documents, not just a hit list. Rejected: search alone (no synthesis).
Escalation ladder	Rung 2, routed workflow	The path is fixed: retrieve, rerank, synthesize, verify. Nothing branches on intermediate model decisions [knowledge/building-effective-agents.md]. Anti-escalation rule applied: an agent was explicitly rejected. A tool-loop agent would add latency and unpredictability and a wider blast radius, with no capability this task needs.
Knowledge strategy	Advanced RAG, permission-aware	40M documents, growing weekly, per-matter entitlements. This is the textbook case for retrieval with entitlement filtering [knowledge/rag-patterns.md]. Rejected: fine-tuning (corpus churns weekly, and a model cannot hold per-matter walls) and long-context (40M docs do not fit anything).
Autonomy	Tier 0, read-only	No writes exist in the design, so there is no autonomous action to govern [knowledge/security-governance.md].
Memory	None beyond the query	No cross-session personalization is required, and less stored state means a smaller confidentiality surface [knowledge/context-memory.md].

The headline decision The right answer here is explicitly not an agent. Recognizing that is the design.

4. System architecture

A fixed pipeline: identity resolution applies the current ethical walls, a cheap router classifies the query, the retriever runs an entitlement-filtered hybrid search, a reranker selects passages, a synthesis model writes a cited answer, and a citation guard verifies every claim resolves to a retrieved source before anything returns.

Figure 1. The retrieval workflow. The dashed boundary is the ethical-wall enforcement zone; the Wall Sync job refreshes entitlements daily, and there is no agent and no tool loop.

Component	Serves	Scaling	Failure mode and fallback	Tech example
Identity + walls	Entitlement resolution	Stateless, horizontal	Fail closed: no walls, no retrieval	Entra ID + DMS ACL sync
Query router	Cheap triage	Stateless, horizontal	Default to full retrieval	Haiku-class model
Retriever	Entitlement-filtered search	Stateless, read replicas	Degrade recall, never bypass ACL	Hybrid dense + BM25 [knowledge/rag-patterns.md]
Vector index	40M doc embeddings	Stateful, sharded	Rebuildable from the DMS	OpenSearch or pgvector-class, disk-ANN
Reranker	Passage precision	Stateless, horizontal	Fall back to retriever order	Cross-encoder
Answer synthesis	Grounded, cited answer	Stateless, horizontal	Refuse rather than guess	Sonnet-class model
Citation guard	Every claim sourced	Stateless, horizontal	Suppress unsourced claims	Claim-to-source check [knowledge/evaluation.md]
Wall sync	Daily entitlement refresh	Batch job	Stale walls block, not leak	Scheduled ACL pipeline

Trust boundary Entitlements are resolved before retrieval and enforced inside the retriever. The model never sees a document the caller is not entitled to, so no prompt-level mistake can cause a cross-matter leak.

5. Data and retrieval

This is the heart of the design [knowledge/rag-patterns.md].

Ingestion

40M documents with variable OCR quality, growing 30k per week. The pipeline OCRs, chunks (with small-to-big passage windows so citations point to precise spans), embeds, and writes each chunk with its matter id and entitlement metadata. Incremental ingestion keeps the index within a week of the DMS; a full rebuild is acceptable over a weekend per the RTO.

Permission-aware retrieval, and why it is the crux

Every chunk carries its matter entitlement. At query time the retriever receives the caller identity and the current ethical walls, and filters candidates by entitlement before ranking [knowledge/security-governance.md]. Walls change daily as conflicts arise, so a daily Wall Sync job refreshes the ACL the retriever reads. The failure direction is deliberate: if the sync is stale, access is denied (a lawyer waits), never granted (a wall is breached).

Retrieval quality

Hybrid dense plus keyword search handles both concept and exact-term legal queries; a cross-encoder reranker lifts precision so the synthesis model sees only the strongest passages, which also controls token cost on a large-context model.

6. Tools and integrations

There are no action tools. The only integration is a read-only connection to the document management system and its ACLs [knowledge/interoperability-observability.md].

Integration	Contract	Tier	Enforcement
DMS read	doc id + identity → content	0 (read)	Read-only credential, ACL-checked
ACL sync	daily → entitlement snapshot	0 (read)	Batch, fail-closed on error

No writes by construction The system has no tool that mutates anything. Read-only is enforced by absence of any write path, not by policy.

7. State and memory

No persistent memory. Each query is answered from retrieval alone [knowledge/context-memory.md]. Declining to store conversation history or user memories is a deliberate confidentiality choice: there is no store of matter content to leak, subpoena, or mis-scope. Within a single session the current question and its retrieved context live only for the duration of the request.

8. Security, identity, and guardrails

Attack surfaces [knowledge/security-governance.md]: user queries (prompt injection) and retrieved document text (indirect injection from adversarial or quoted content). Because the system is read-only, the excessive-agency and lethal-trifecta risks are largely designed out: there is no exfiltration channel and no write.

Control	Purpose (OWASP / rule)
Entitlement-filtered retrieval	LLM02 sensitive-info disclosure; ABA 1.6 confidentiality
Identity propagation (caller permissions)	No broad service account can over-retrieve
Citation guard	LLM09 misinformation; unsourced claims refused
Injection handling on query and passages	LLM01 prompt injection
Full audit log of query, retrieved doc ids, answer	Conflicts audits, ABA compliance evidence

Governance: maps to NIST AI RMF MAP (contextualize the confidentiality risk), MEASURE (leak testing in evals), and MANAGE (audit and access review) [knowledge/security-governance.md]. GDPR applies to EU matters: retrieval respects data residency and the audit log supports access requests.

9. Evaluation plan

Confidentiality and citation accuracy are the two things that must be measured before launch [knowledge/evaluation.md].

Leak testing (gating): a red-team set of cross-matter queries that must return zero entitled-but-walled documents. Any leak is a launch blocker.
Golden set: real lawyer questions with known correct sources, stratified by practice area, refreshed as precedent grows.
Retrieval metrics: recall@k and reranked precision measured separately, since a citation is only as good as the retrieval behind it.
Citation faithfulness: every claim must resolve to a retrieved passage (RAGAS-style faithfulness, the TruLens groundedness leg) [knowledge/evaluation.md].
Refusal quality: the system should refuse when sources are weak rather than fabricate, measured explicitly.

Gate	Unlocks
Zero leaks on the red-team set, citation faithfulness above threshold	Shadow with a pilot practice group
Pilot precision and refusal quality hold, zero reported leaks	Firm-wide rollout

10. Observability

Every query emits a trace with retrieval, rerank, and synthesis as spans, capturing the retrieved document ids, tokens, latency, and cost, using OpenTelemetry GenAI conventions [knowledge/interoperability-observability.md]. A tamper-evident audit log records who asked what and which documents were surfaced, which is required for conflicts audits and ABA compliance. Dashboards track p95 latency, cost per query, refusal rate, and citation-faithfulness drift; alerts fire on any entitlement-filter error (treated as a security incident, not a bug) and on latency or cost anomalies.

11. Scale and cost analysis

Model pricing used, [live-sourced 2026-07-14]: Haiku 4.5 at 1.00 / 5.00 USD and Sonnet 4.6 at 3.00 / 15.00 USD per million input / output tokens, with prompt caching near 10% of input price on cached tokens.

Per-query cost

Element	Tokens	Cost
Router (Haiku)	~1k in, ~50 out	~0.001 USD
Embedding + cross-encoder rerank	query-scale + candidate set	~0.002 USD
Synthesis (Sonnet), 1 call	~1k cached + ~9k passages in, ~800 out	~0.040 USD
Total per query		~0.045 USD

Monthly: ~50,000 queries at ~0.045 USD is only ~2,250 USD in model spend. The budget is dominated by fixed infrastructure: hosting a 40M-document vector index (roughly 250 GB of vectors before overhead, so a sharded disk-ANN cluster), the OCR and embedding pipeline for 30k new docs per week, rerank compute, and 25-year audit retention. Estimate the index and pipeline at 10,000 to 14,000 USD per month, bringing the all-in figure to roughly 16,000 to 18,000 USD, inside the 25,000 USD ceiling with headroom.

Latency budget (p95 ≤ 15s)

Identity and walls ~0.3s, router ~0.5s, retrieval over 40M docs ~1.5s, rerank ~0.8s, synthesis with large legal context ~9s (first token under 3s via streaming), citation guard ~0.6s. There is no agent loop, so there is no tail-latency multiplication across steps [knowledge/latency-cost-reliability.md].

At 10x (25,000 queries/day)

Model spend approaches ~22,000 USD/month, which alone nears the ceiling. The index size does not grow with query volume (it grows with the corpus), so the levers are on the query path: cache repeated precedent lookups (legal research repeats heavily within a matter), route simple lookups to a smaller synthesis model, and add retrieval and rerank replicas. The fixed index cost stays roughly flat.

12. Failure modes and degradation

Failure	Degradation
Wall Sync stale or failed	Deny access on affected matters (fail closed), page an admin. Never widen access.
Model provider outage	Retrieval still returns cited passages; synthesis queued or served by a fallback model [knowledge/latency-cost-reliability.md]
Retrieval degraded (index node down)	Reduced recall, flagged to the user; never bypass the ACL filter
Weak or no sources	Refuse with an explanation rather than synthesize an unsupported answer
OCR failure on a document	Document excluded from the index with a quality flag, not silently half-indexed

13. Rollout plan

Crawl: one pilot practice group, read-only, with lawyers instructed to report any surprising document surfacing. Gate: zero leaks on the red-team set and in the pilot.
Walk: expand to several practice areas, monitor citation faithfulness and refusal quality. Gate: metrics hold, zero reported confidentiality issues [knowledge/evaluation.md].
Run: firm-wide. Ongoing conflicts-audit review of the access log, and sampled queries feed the golden and red-team sets [knowledge/genai-sysdesign-loop.md].

14. Request walkthrough

A single query, from authorization through a verified, cited answer.

Figure 2. The query sequence. Entitlements are resolved before retrieval, the index returns only documents within the caller's walls, and the citation guard verifies sources before any answer returns.

15. References

Knowledge base

knowledge/decision-trees.md, building-effective-agents.md (the workflow vs agent decision and the anti-escalation rule)
knowledge/rag-patterns.md (permission-aware retrieval, hybrid search, reranking, ingestion)
knowledge/security-governance.md (identity propagation, confidentiality, OWASP and NIST mappings)
knowledge/evaluation.md (leak testing, citation faithfulness, refusal quality), interoperability-observability.md (tracing, audit)
knowledge/latency-cost-reliability.md, enterprise-architecture.md (capacity, cost, fail-closed DR)

Live-sourced

Anthropic API pricing, retrieved 2026-07-14, for the cost model in section 11.

Primary-source citations for each knowledge document are in agentify/knowledge/SOURCES.md.