Secure enterprise knowledge retrieval where authorization is enforced before information reaches model context.

ContextGate is an enterprise knowledge platform built around a simple security principle:

Retrieved does not mean authorized.

It combines document ingestion, access-control metadata, hybrid retrieval, freshness/version handling, deterministic authorization, and grounded citations so that only permitted, current, traceable enterprise knowledge can reach an AI system.

Project Overview

Enterprise AI assistants often need access to internal knowledge: policies, product documentation, engineering runbooks, customer information, operational procedures, and other private documents.

Retrieving the most relevant information is only part of the problem.

The harder question is:

Should this user, in this context, be allowed to send that information to the model at all?

ContextGate treats authorization as part of the retrieval architecture rather than as a filter applied after generation.

Documents and chunks carry tenant, access-control, version, and freshness metadata. Retrieval can identify potentially relevant content, but a deterministic authorization boundary decides which chunks are allowed into the final model context.

The platform then produces grounded answers using only the permitted evidence and keeps citations traceable to their source.

This design intentionally separates retrieval relevance from authorization and model behavior. GitHub_Portfolio_Sprint_Aug24_S…

The Problem

A traditional RAG pipeline often looks like:

Question
   ↓
Retriever
   ↓
Top-K Documents
   ↓
LLM
   ↓
Answer

That architecture assumes that anything the retriever finds is safe to send to the model.

In an enterprise environment, that assumption is dangerous.

Two employees may ask exactly the same question but have different permissions. A document may still exist in an index even after access has been revoked. An older version may rank higher than the current one. Retrieved content itself may contain instructions attempting to manipulate the downstream model.

ContextGate introduces an explicit security boundary:

RELEVANT
!=
AUTHORIZED

RETRIEVED
!=
ALLOWED IN MODEL CONTEXT

Authorization happens before final model context is assembled. GitHub_Portfolio_Sprint_Aug24_S…

Core Design Principle

Only authorized, current, traceable enterprise knowledge should reach model context.

This leads to three separate decisions:

Can we retrieve it?
        ↓
Is the user allowed to see it?
        ↓
Is this the version that should enter context?

Only after those questions are answered does content become model-visible evidence.

How ContextGate Works

1. Knowledge ingestion

Enterprise documents enter the ingestion pipeline with metadata describing their identity and access boundaries.

The core model includes information such as:

Tenant
Document
Chunk
ACL / permissions
Version
Freshness
Source metadata

Documents are chunked and indexed while preserving the metadata needed for later authorization and traceability.

The design includes explicit delete/reindex behavior rather than treating the vector/search index as an immutable knowledge store. GitHub_Portfolio_Sprint_Aug24_S…

2. Hybrid retrieval

ContextGate uses hybrid retrieval to find potentially relevant knowledge.

The implementation combines:

BM25
+
LSA-based semantic retrieval
+
Reciprocal Rank Fusion (RRF)

The goal is not to treat the retrieval score as a security decision.

Retrieval answers:

Which pieces of knowledge appear relevant?

It does not answer:

Which pieces is this caller authorized to expose to the model?

That decision belongs to the next layer.

3. Deterministic authorization gate

Retrieved candidates pass through a deterministic authorization boundary.

Retrieved candidates
        ↓
ACL / tenant / access checks
        ↓
Freshness + version checks
        ↓
AUTHORIZED CONTEXT

Unauthorized chunks are excluded before final model context is built.

This means a document can be highly relevant and still never become visible to the model.

That distinction is one of the central architectural properties of ContextGate. GitHub_Portfolio_Sprint_Aug24_S…

Why Authorization Before the Model Matters

Consider an internal company assistant.

A user asks:

“What is the compensation policy for the executive team?”

The retrieval engine may discover an executive-compensation document with an excellent relevance score.

But if the caller is not permitted to access that document:

Retriever
    ↓
Document found
    ↓
Authorization Gate
    ↓
DENIED

The document never enters the prompt.

The security guarantee therefore does not depend on asking the LLM:

“Please don’t reveal confidential information.”

It is enforced by deterministic application logic outside the model.

Grounded Answers and Citations

After authorization, only approved chunks become answer context.

Authorized Evidence
        ↓
Grounded Answer
        ↓
Citations

The answer path preserves evidence identity so citations can be checked against the actual material that was allowed into context.

This creates an important distinction:

MODEL GENERATED TEXT
!=
AUTHORIZED EVIDENCE

The model may generate the answer, but the application owns the evidence boundary.

The project includes allowed and unauthorized fixtures specifically to demonstrate that unauthorized material does not become answer context while authorized answers remain traceable to evidence.

Freshness and Versioning

Enterprise knowledge changes.

A policy may be updated. Documentation may be replaced. Access may be revoked. A deleted document may still exist in a stale search index.

ContextGate therefore treats freshness and version metadata as part of knowledge correctness rather than only retrieval metadata.

The lifecycle includes:

Document v1
    ↓
Indexed
    ↓
Document updated
    ↓
v2 becomes current
    ↓
Old representation removed / reindexed
    ↓
Current version becomes retrievable

The project explicitly models delete/reindex and freshness/version behavior rather than assuming indexed data remains valid forever. GitHub_Portfolio_Sprint_Aug24_S…

Prompt-Injection Boundary

Retrieved documents should be treated as data, not as trusted application instructions.

ContextGate therefore tests a boundary where retrieved content may itself contain malicious or instruction-like text.

The important protection is not:

“The model will recognize malicious instructions.”

Instead, deterministic authorization and context construction remain outside model control.

For example:

Potentially malicious document
          ↓
Retriever may discover it
          ↓
Authorization / context gate
          ↓
Denied if caller is not authorized
          ↓
Never reaches model context

This is intentionally described as a prompt-injection boundary, not complete prompt-injection prevention. The project does not claim that ACL filtering alone solves every form of prompt injection. GitHub_Portfolio_Sprint_Aug24_S…

Architecture

For the website, this is the clearest architecture view:

Enterprise Documents
        ↓
     Ingestion
        ↓
     Chunking
        ↓
Metadata + ACL + Version + Freshness
        ↓
       Index
        ↓
  Hybrid Retrieval
 BM25 + LSA + RRF
        ↓
Retrieved Candidates
        ↓
┌───────────────────────────┐
│ Deterministic ContextGate │
│                           │
│ Tenant                    │
│ ACL                       │
│ Version                   │
│ Freshness                 │
└─────────────┬─────────────┘
              ↓
      Authorized Evidence
              ↓
       Grounded Answer
              ↓
          Citations

The key architectural boundary sits between retrieval and model context.

Security Model

ContextGate deliberately assigns different responsibilities to different layers:

Layer	Responsibility
Retriever	Find potentially relevant knowledge
Authorization Gate	Determine which retrieved chunks are permitted
Freshness / Version Logic	Prevent stale or invalid knowledge from silently becoming current context
Context Builder	Assemble only authorized evidence
Model	Generate an answer from provided evidence
Citation Layer	Keep the answer traceable to permitted sources
Application	Own security policy and access decisions

The model does not own authorization.

That decision remains deterministic and externally enforceable.

Example: Same Question, Different Context

Imagine two employees ask:

“What is the migration plan for Project Atlas?”

The search engine may retrieve:

Public engineering overview

Internal migration plan

Restricted leadership document

For Engineer A:

Public overview              ALLOW
Internal migration plan      ALLOW
Leadership document          DENY

For Employee B:

Public overview              ALLOW
Internal migration plan      DENY
Leadership document          DENY

The retrieval ranking may be identical.

The final model context is not.

That is the architectural problem ContextGate is designed to demonstrate.

Evaluation

ContextGate includes offline fixtures and tests around the properties that matter most for a secure enterprise knowledge system.

The implementation verifies authorization exclusion, allowed evidence, grounded citations, freshness/version behavior, deletion/reindex handling, retrieval behavior, and the retrieved-content security boundary.

The completed Enterprise Knowledge v0.1 milestone contains 58 automated tests.

Importantly, the project does not reduce these properties to one artificial “RAG quality score.”

Access correctness, retrieval relevance, freshness, and grounding are different failure dimensions.

Engineering Decisions

Authorization is not retrieval filtering

A relevance algorithm should not be responsible for security policy.

The retrieval layer can return candidates. The deterministic authorization layer decides whether those candidates may progress.

Model behavior is not a security boundary

The system does not rely on a prompt such as “only reveal documents this user may access.”

Unauthorized information is filtered before the model receives it.

Freshness is part of trust

A correctly authorized but obsolete document can still produce an incorrect answer.

Version and freshness are therefore part of the knowledge lifecycle.

Citations must point to permitted evidence

Grounding is useful only if the supporting evidence itself was allowed into the invocation context.

Retrieval and authorization remain independently testable

Keeping these concerns separate makes failures easier to identify:

Bad retrieval?
Security failure?
Stale knowledge?
Citation failure?
Model behavior?

Each can be investigated independently.

Technology & Engineering

Area	Implementation
Core	Python
Knowledge Model	Tenant, document and chunk metadata
Authorization	Deterministic ACL-aware context gating
Retrieval	Hybrid BM25 + LSA
Fusion	Reciprocal Rank Fusion
Knowledge Lifecycle	Version, freshness, delete and reindex paths
Answering	Grounded answer path using permitted evidence
Citations	Traceable evidence references
Security Testing	Authorized/unauthorized fixtures and retrieved-content attack boundary
Provider Dependency	Offline-friendly; external model not required for core tests
Testing	58 automated tests

What ContextGate Does Not Claim

I would keep this section because it makes the project much more credible.

ContextGate does not claim to be a production identity provider, a full enterprise search product, a universal prompt-injection defense, a complete data-loss-prevention platform, or a substitute for application-level authorization infrastructure.

The project demonstrates the architectural pattern and controls required to ensure that retrieval relevance does not bypass enterprise authorization.

Relationship to a Normal RAG System

A conventional RAG architecture emphasizes:

Retrieve
→ Generate

ContextGate changes that to:

Retrieve
→ Authorize
→ Validate freshness
→ Construct permitted context
→ Generate
→ Cite

That small-looking extra boundary changes the trust model considerably.

Project Artifacts

Project Artifact	What it contains	Link
Source Repository	Complete ContextGate implementation and project history.	View Repository
Project Overview & Architecture	Core design, deterministic authorization boundary, hybrid retrieval flow, grounded answers, limitations, and quick start.	Read README
Architecture & Design Documentation	Supporting design and security documentation for the enterprise knowledge platform.	Browse Documentation
Knowledge & Security Fixtures	Synthetic scenarios used to exercise authorization, versioning, freshness, retrieval, and grounded-answer behavior.	View Fixtures
Core Implementation	ContextGate source code covering ingestion, retrieval, authorization gating, model-context construction, and related behavior.	Browse Source Code
Automated Tests	Offline tests covering the deterministic authorization boundary, retrieval behavior, citations, freshness, and security invariants.	View Tests
Security Policy	Security guidance and reporting information for the repository.	View Security Policy
Build & Demo Workflow	Make targets for running tests, demo flows, and benchmark workflows.	View Makefile

The original portfolio design explicitly called for the flow ingestion → chunking → metadata/ACL → indexing → ACL-aware hybrid retrieval → grounded answer + citations, with authorization before final model context.

The hardest problem in enterprise RAG is not retrieving the right document. It is making sure the wrong document never becomes model context.

ContextGate makes that boundary explicit.