Secure RAG ingestion: quarantine documents before indexing

A RAG system can retrieve a malicious document as accurately as a trusted policy. Embeddings do not distinguish evidence from instructions, and a high similarity score does not prove that a file is safe, authorized or current. The secure design is to quarantine each new document, establish provenance and access scope, extract it in an isolated worker, scan the extracted representation, publish an immutable index version, and still treat every retrieved passage as untrusted data at answer time.

This is a practical engineering guide, not a new OpenAI product announcement. OpenAI's official documentation says prompt injection begins when untrusted text or data attempts to override the system, recommends keeping untrusted variables out of developer messages and using structured outputs between workflow nodes. OpenAI File Search and Retrieval document how uploaded files become searchable vector-store content. NIST identifies resource control, data poisoning, malware and RAG-specific risk reassessment as risks worth testing. The quarantine workflow below is an engineering synthesis of those sources, verified on October 26, 2026.

The answer first

Do not upload a customer file directly into the live vector store. Give the upload an immutable identity, store the original privately, parse it in an isolated environment, classify and scan the output, bind trusted authorization metadata outside the document, and publish a complete index version only after every gate succeeds. At query time, derive tenant and ACL filters from authenticated application state, hydrate authorized text from the source of truth, delimit retrieved content as evidence, and require approval for any side effect.

Threat model the ingestion boundary

The uploader, source system and document author may be different actors. A legitimate employee can upload a supplier PDF containing hidden text. A crawler can import a compromised page. A trusted knowledge source can be replaced after its initial approval. A parser can expand a small archive into gigabytes of data. A retrieved passage can tell the model to ignore policy, reveal secrets or invoke a tool.

Model at least these failure classes:

• indirect prompt injection: instructions arrive inside a document, web page, email or image-derived text;
• data poisoning: crafted content changes which chunks are retrieved or what the assistant concludes;
• parser exploitation: malformed files, decompression bombs or vulnerable converters attack ingestion workers;
• scope poisoning: attacker-controlled metadata moves content into another tenant, audience or retention class;
• provenance loss: chunks can no longer be tied to exact source bytes, version and location;
• stale deletion: a revoked file remains searchable in an old index, cache or replica.

NIST's adversarial machine-learning taxonomy calls control of external resources consumed at inference time resource control and connects it to indirect prompt injection. That is a threat classification, not a promise that any single scanner detects it.

Build a quarantine state machine

Keep the original file outside the searchable corpus until a state machine reaches published:

received -> stored -> type_verified -> extracted -> scanned
         -> classified -> approved -> indexed -> published

any gate -> rejected | quarantined | needs_review
published -> superseded -> tombstoned -> verified_absent

Each transition writes the actor, time, policy version, input digest, output digest and reason. A retry resumes the same document version instead of creating a second visible copy. Store failures explicitly; an ingestion job that times out after writing half its chunks must not make the partial version searchable.

A useful identity separates the logical document from its immutable versions:

{
  "tenant_id": "derived-by-server",
  "document_id": "policy:returns",
  "version_id": "sha256:original-bytes",
  "source_uri": "controlled-reference",
  "ingestion_policy": "rag-ingest-v7",
  "parser_version": "pdf-pipeline-v4",
  "acl_revision": "acl-218",
  "status": "quarantined"
}

The digest detects byte-level change; it does not prove authorship or safety. A signed upstream event, repository commit, content-management audit trail or human approval can strengthen provenance.

Verify type before parsing

Do not trust the extension or client-declared Content-Type. Compare magic bytes, detected type and allowed formats. Enforce compressed and expanded size limits, page and object counts, recursion depth, timeout, memory and CPU budgets. Reject encrypted or password-protected files unless the product has an explicit secure handling path.

Run parsers and OCR in short-lived workers with no production credentials, a read-only input mount, a separate output location and restricted network access. Patch the parser image independently of the web application. Treat macros, scripts, external references and embedded files as active content: strip, reject or route them to specialist analysis according to policy. Antivirus is one control for known malware, not proof that prose is safe for an LLM.

Retain both the original artifact and the normalized extraction manifest. The manifest should record pages, headings, tables, images, links, embedded objects, OCR confidence and extraction warnings. A parser that silently drops a table can create an integrity failure even when no attacker is present.

Separate document data from control metadata

Never let the document choose tenant_id, ACLs, source authority, retention, legal status or approval state. Trusted application code derives those fields from the authenticated upload route and policy service. Store them as typed metadata beside the artifact and project them into every searchable point.

OpenAI File Search supports metadata filtering, and the Retrieval API searches vector stores with file-attribute filters. Those capabilities are useful enforcement inputs, but the application still owns the policy. A public request must not submit an arbitrary tenant filter. Build the filter server-side and apply it to every retrieval branch.

For high-risk corpora, hydrate the final chunk through the authorized repository after vector search. The vector store returns candidate identities; the source-of-truth service rechecks the user's current entitlement and returns canonical text. This catches stale ACL projections and prevents a vector point from becoming the authority.

Scan content without pretending detection is perfect

Use several narrow detectors rather than one safe flag:

A prompt-injection classifier can reduce exposure, but false negatives remain. Therefore the runtime must stay safe even when malicious text passes ingestion. Do not rewrite suspicious text into a developer message. Delimit it as retrieved evidence, constrain node-to-node data with structured schemas, and keep authorization and tool policy outside the model.

Publish indexes atomically

Chunk, embed and index into a staging version. Record every chunk's source version, location, content digest, parser and embedding versions, ACL revision and publication state. Count expected and written chunks, run sample retrieval and permission tests, then switch an alias or application pointer atomically.

OpenAI's Retrieval documentation states that adding a file to a vector store automatically chunks, embeds and indexes it. It also notes that removing files is eventually consistent, so search can briefly return removed content. For sensitive revocation, do not treat a delete call as immediate proof of absence. Deny the document ID at the application layer, stop hydration, clear caches, poll or reconcile until the old content is absent, and record evidence of completion.

Treat retrieved passages as evidence, never instructions

At generation time, use a fixed contract:

1. System and developer policy define the task and permitted tools.
2. User text states the request but cannot grant new authority.
3. Retrieved passages are labeled evidence with stable citation IDs.
4. The model returns a structured answer or a typed abstention.
5. A validator checks citation IDs and output schema.
6. Any write action is independently authorized and approved.

Do not include secrets, broad tool credentials or unrelated private records in context. Cap tokens and chunks per document so one poisoned source cannot dominate the prompt. When sources conflict, prefer policy-defined authority and effective dates; otherwise expose the conflict instead of asking the model to decide truth from similarity.

Evaluate attacks and lifecycle failures

Build a security evaluation set alongside relevance tests. Include a benign policy that contains words like “ignore previous,” a malicious instruction hidden in a footer, white text recovered by OCR, a poisoned duplicate with a newer-looking date, a cross-tenant document, a revoked document, a parser timeout, an archive bomb, and a file that changes after approval.

Measure more than classifier accuracy:

• unauthorized retrieval rate must be zero in the test suite;
• revoked-content retrieval and hydration rates;
• injection-following rate and forbidden tool-call rate;
• citation validity and source-version accuracy;
• quarantine false positives and manual-review time;
• time from revocation to verified absence;
• incomplete publication and orphaned-point counts.

NIST's Generative AI Profile recommends testing value-chain risks including poisoning, malware, privacy and localization, reassessing risk after RAG implementation, and documenting sources and provenance. Those are governance recommendations; translate them into the executable controls and evidence your application can actually prove.

Best practices and anti-patterns

Best practices

• Preserve immutable original bytes and source provenance.
• Parse in isolated, resource-bounded workers.
• Keep ACL and authority metadata outside document control.
• Publish complete index versions atomically.
• Reauthorize during hydration, not only during ingestion.
• Keep retrieved text out of privileged messages.
• Verify deletions across index, cache and replicas.
• Red-team the end-to-end pipeline after parser, model or policy changes.

Anti-patterns

• Uploading directly into the live vector store.
• Trusting extension, MIME header or file name.
• Treating antivirus or an injection classifier as a guarantee.
• Allowing document text to set tenant or audience metadata.
• Giving a RAG answer an unrestricted write tool.
• Marking deletion complete when the API accepted the request.
• Logging raw private documents and prompts in general observability systems.

Implementation checklist

1. Define accepted sources, formats, data classes and review rules.
2. Create immutable document and version identities.
3. Store originals privately and parse in isolated workers.
4. Record a typed extraction manifest and security findings.
5. Derive tenant, ACL and authority from trusted application state.
6. Index into staging with complete provenance metadata.
7. Run retrieval, permission, injection and deletion tests.
8. Publish atomically and keep a rollback pointer.
9. Reauthorize retrieved candidates before hydration.
10. Validate citations, abstention and tool proposals at runtime.

Related reading: Production RAG architecture, Qdrant tenant isolation, RAG retrieval evaluation, RAG citation provenance, and the AI, RAG & Vector Search hub.

Primary references

• OpenAI — Safety in building agents
• OpenAI — File Search
• OpenAI — Retrieval
• NIST AI 100-2e2025 — Adversarial Machine Learning
• NIST AI 600-1 — Generative AI Profile
• NIST IR 8579 — NCCoE RAG chatbot security learnings
• Wikimedia Commons — Sheet Fed Scanner
• Creative Commons Attribution 4.0

Prepared by: Noor Yasser