Skip to content

TrustRAG

Retrieval that respects permissions: cited answers that never cross a clearance boundary.

My role
AI systems engineer
When
2024
Status
Open source
Stack
Python, RAG, Vector DB
TrustRAG permission-aware retrieval system

01

Where it started

Standard RAG pulls every semantically similar chunk it finds, leaking confidential documents across roles and inventing unsupported claims. That's a non-starter in compliance-heavy domains.

02

What I owned

AI systems engineer

03

How I built it

  • Pushed role and tenant filters down into the vector query, so unauthorised chunks never reach the context window.
  • Mapped every synthesised sentence back to chunk IDs and source versions for a full audit trail.
  • Built an evaluation harness covering 200+ permission-boundary cases, and refused to answer below a 0.78 similarity threshold.
[User Query + User JWT / Role Context]
                   │
                   ▼
       [Permission Gatekeeper] ── Validate clearance level (Public, Staff, Exec)
                   │
                   ▼
     [Hybrid Vector & Metadata Search]
   (Dense embeddings + metadata filter: tenant_id == X AND clearance <= user_clearance)
                   │
                   ▼
        [Verified Chunks + Provenance Citations]
                   │
                   ▼
    [Grounded Synthesis Engine (Hallucination Guardrails)]
                   │
                   ▼
   [Strictly Cited Answer + Source Audit Trail]
System architecture
  • Pre-retrieval filtering: metadata filters pushed into the vector query so unauthorised chunks never enter the context window.
  • Provenance tracking: every synthesised sentence maps to chunk IDs, byte offsets and source document versions.
  • Terminal fallback: below a 0.78 cosine similarity threshold the model refuses rather than guessing.

Token-level access control

JWT-embedded role verification checks tenant isolation and permissions before embedding generation.

Evaluation benchmark suite

Harness measuring retrieval precision, context recall and leakage rate across 200+ permission-boundary cases.

Hardening: Separate partition keys and row-level security across vector collections to prevent cross-tenant leakage.

04

What changed

100% boundary isolation across the cross-clearance leakage benchmarks, with cited answers or an explicit refusal.

  • ✓100% boundary isolation in cross-clearance leakage benchmarks.
  • ✓No hallucinated answers on out-of-domain queries thanks to the confidence gate.
PythonRAGVector DBRBACEvaluation