RAG Security Best Practices: 9 Controls for Production

Controlling what a RAG application says does not require access to the model or the network around it. It requires write access to a document the retriever will pick up. In the PoisonedRAG work presented at USENIX Security 2025, five crafted texts per target question, dropped into a knowledge base of millions, steered the answer 90% of the time, and the defenses the authors tested did not reliably stop it.

That works because retrieval and generation share no trust boundary. A transformer reads its context as a single token stream, so the instructions you wrote and the text a document happened to contain arrive with equal authority. Nothing is malfunctioning when this happens. The pipeline is retrieving relevant material and handing it to the model, which is the job.

The operational problem is that none of it is visible from outside. A planted instruction, a retrieval that returned a file the user was never entitled to see, and a chunking strategy that simply picked badly all produce the same output: a confident wrong answer and a confused user. Latency and error rate look normal in all three cases.

Encryption at rest and a well-scoped access list are both worth having, and neither one tells you which chunk entered the context window on the request that went wrong. A control nobody can verify is a control you are trusting rather than running. That is the test the RAG security best practices below are built around: change something in the pipeline, then keep the evidence that proves the change held.

What Is RAG Security?

RAG security is the practice of protecting a retrieval-augmented generation pipeline across every stage where untrusted data can enter or sensitive data can leave: document ingestion, vector storage, retrieval at query time, and the generated response. It treats retrieved content as untrusted input rather than as trusted context.

Securing a standalone model call is mostly a question of what the user sends and what the model returns. You control the system prompt, you validate the input, and the surface stops roughly there. A retrieval pipeline widens that surface in a direction application security tooling was not built to watch, because the dangerous input now arrives from your own document store, signed off by your own connectors, on a path nobody classifies as user input.

The stages break out cleanly enough to organize the work:

• Ingestion covers everything that writes into the index, including connectors pulling from shared drives and ticketing systems.
• Storage covers the vector database itself, where embeddings sit as a second copy of source content.
• Retrieval covers what gets selected at query time and on whose authority.
• Generation covers what the model does with the material it received and what its output can reach.

A control at one stage rarely compensates for a gap at another, which is why encryption at rest does nothing about a poisoned chunk and a well-written system prompt does nothing about an over-permissioned namespace.

If you want the mechanics of how the pipeline is assembled before working through the risks, the walkthrough of what a RAG pipeline is covers chunking, embedding, and retrieval architecture in order.

Why Is RAG Security Important?

RAG moved from a pattern people prototyped to the default way enterprise applications reach live internal data, and the security properties came along with it whether or not anyone reviewed them. Three of those properties do most of the damage:

• Retrieved content carries the same authority as your system prompt - A transformer reads its context as one token stream, so instructions you wrote and text a document contained are not distinguishable to the model. Prompt injection sits at LLM01 in the OWASP Top 10 for LLM Applications for this reason, and the indirect variant, where the instruction rides in on a retrieved file rather than a user message, is the version RAG introduces. The 2026 edition widened its Hidden Context Exposure entry specifically to cover retrieved policy text and tool schemas assembled into the context window, not just the system prompt.
• A poisoned chunk and a bad answer look identical from outside - Retrieval quality problems and retrieval attacks present the same way to the user and to your error rate. Nothing in latency, token count, or HTTP status separates them. Teams usually find out from a complaint, and then have to reconstruct what happened from an application log that recorded the question and the answer but not the seven chunks in between.
• Agentic RAG turns a bad answer into an action - Once the retrieval output feeds an agent with tools, a planted instruction stops being a wrong sentence and becomes a database write, an outbound request, or a refund. The OWASP 2026 list moved Excessive Agency up to third place on the strength of incident data, and its framing is that you should build the system around the model so that a successful manipulation has nowhere to go.

CVE-2025-32711 is the version of this that already happened at scale. Aim Labs disclosed it in June 2025 at CVSS 9.3, after finding that a crafted email containing hidden instructions was retrieved by Microsoft 365 Copilot as ordinary context, and that those instructions caused it to exfiltrate data from the user's environment. Microsoft patched it server-side and reported no exploitation in the wild. The researchers noted the technique should generalize to other retrieval-backed assistants and agents, which is the part worth carrying into your own threat model.

9 RAG Security Controls for Production

The controls are grouped by where they sit in the pipeline. Order matters less than coverage, though the retrieval group is where most teams find the largest real exposure and the cheapest fix.

Ingestion Controls

1. Record Document Provenance at Ingestion

Every chunk in the index should carry a record of where it came from, written at ingestion time and stored alongside the vector. That means:

• A content hash of the source document - Comparing current hashes against what you recorded surfaces documents that changed after they entered the index.
• The source system - Which connector, drive, or repository produced it.
• The ingesting identity - The account or service that performed the write.
• The ingestion timestamp - What the index looked like at the time of a given request.

This costs one column and a few lines in the ingestion job. The value shows up during incidents rather than during normal operation, because when an answer turns out to be wrong in a way that looks deliberate, provenance lets you go from the chunk in the context window to the document it came from, then to the connector that wrote it and the account that owned the write. Without it, you have a bad answer and a vector store with no history.

2. Sanitize and Normalize Content Before Embedding

Hidden text is the delivery mechanism in most published indirect injection work: white-on-white spans, HTML comments, zero-width characters, and Unicode homoglyphs that render as nothing to a human reader and as instructions to a model. The ingestion path should flatten all of it before the embedding model ever sees the text.

1. Extract deliberately. PDFs, spreadsheets, and email bodies carry structure that survives naive text extraction, so the extraction step deserves the same scrutiny as the filter after it.
2. Strip markup you do not need. Comments and styling attributes are where invisible payloads live.
3. Apply Unicode normalization. Homoglyphs and zero-width characters get resolved to something you can actually inspect.
4. Flatten whitespace. Removes the layout tricks that separate what a human sees from what the model reads.
5. Then embed.

Doing this at ingestion rather than at query time matters more than it looks. Once a payload is embedded, it is inside a vector that participates in similarity search, and the retriever will happily return it because it was optimized to be returned. Filtering at query time means you are inspecting content the retriever already decided was relevant, on the request path, under latency pressure.

3. Restrict and Log Write Access to the Index

Read credentials and write credentials for the vector store should be separate, and the application serving queries should hold only the first. This is ordinary least privilege, and it transfers directly from twenty years of application security practice, but it is skipped often because the pipeline was built by one team using one API key.

Connectors are the common gap. A pipeline that syncs from a shared drive, a wiki, or a support inbox has effectively delegated write access to whoever can put a file in those places, which in most organizations is a much larger group than anyone intended. That is not an argument against connectors, since the data they bring is the point of the system. It is an argument for knowing which sources can write, logging what each one wrote, and treating a source with open external submission differently from a curated internal one.

Retrieval Controls

4. Scope Retrieval by Identity, Not by Application

The filter has to be applied to the query, not to the results. Retrieving the top matches and then removing the ones the user should not see still put those documents in your application's memory, and any bug in the filtering step turns into a disclosure. Pass the requesting identity into the retrieval call and let the vector store return only what that identity is entitled to.

There is a second version of this that catches teams later. The permissions that matter are the model's service account permissions, not the end user's session permissions, because the retrieval happens as the service. If the service account can read everything and the only scoping lives in your application code, then a prompt that manipulates the application into a different code path has full reach.

Permissions also do not survive the embedding process on their own. A file that was restricted in SharePoint becomes a vector with no memory of that restriction unless you carried the access metadata across, which is a chunk-level attribute you have to design in rather than inherit.

5. Isolate Tenants at the Namespace Level

Multi-tenant applications that share one index and separate customers with a metadata filter have put the entire tenancy boundary inside a query parameter. One missing filter in one code path exposes another customer's documents.

Namespace or collection-level isolation moves that boundary into the storage layer, where a missing filter returns nothing instead of returning someone else's data. The tradeoff is real: more namespaces means more index overhead and a more involved provisioning path for new tenants. For an application handling regulated data across customers, that overhead is cheaper than the disclosure.

6. Filter Retrieved Content Before It Enters the Context Window

Between retrieval and generation there is a place to inspect what came back, and most pipelines leave it empty. A pre-context check can look for instruction-like patterns in retrieved text, flag chunks whose provenance is unverified, and drop content from sources that should not be reachable for this query type.

Be honest with yourself about what this catches. Pattern-based detection finds known shapes and misses novel phrasings, and a classifier tuned aggressively enough to catch subtle injections will also drop legitimate documents that happen to contain imperative language, which is most policy and procedure content. It is a layer that raises cost for the attacker rather than a gate that closes. The defense-in-depth approach to prompt injection detection works through the input filter, output filter, and dual-LLM patterns in detail, along with the false-positive rates each one produces.

Generation and Output Controls

7. Separate Retrieved Content From Instructions in the Prompt

Delimit retrieved material clearly, label it as data rather than direction, and state in the system prompt that content inside those boundaries is reference material and never instruction. Attaching provenance labels to each chunk helps further, since a model given source attribution has something to weigh when two retrieved passages disagree.

The limitation is structural, and the OWASP LLM Top 10 says so directly: because instructions and data arrive as one token stream, there is no equivalent of a parameterized query here. Delimiting reduces success rates against unsophisticated payloads and does not eliminate the class. Which is why the next control exists, and why teams that lean on prompt hardening alone tend to discover its limits in production rather than in review.

8. Constrain What the Model Can Do With Retrieved Content

Assume the injection succeeds and ask what it can reach. If the answer is a chat response, you have a misinformation problem. If the answer includes a tool that writes to a database, issues a refund, or fetches an arbitrary URL, you have an exfiltration and integrity problem, and the fix is scope rather than detection.

• Narrow the tool set - Give the agent the smallest set of tools the task actually needs, scoped to the credentials that task requires rather than the ones the service happens to hold.
• Gate anything irreversible - A human approval step on writes, payments, and outbound messages costs latency on a minority of requests and caps the damage on all of them.
• Allowlist outbound fetches - Attacker-controlled URLs are a favorite exfiltration path precisely because a fetch looks like normal behavior.
• Plant a canary URL - Something nothing legitimate would ever request, which gives you a signal the moment something reaches for it.

The agentic RAG architecture guide covers where tool calls sit relative to retrieval, which is the diagram you want in front of you when scoping this.

Evidence

9. Log the Retrieval Event, Not Just the Response

Application logs almost always record the question and the answer. Everything that determined the answer sits in between and goes unrecorded, which is why RAG incidents turn into reconstruction exercises. What you want on the retrieval span:

• Chunk IDs - Which specific chunks were returned for this request.
• Source documents - Resolved through the provenance metadata from control 1.
• Similarity scores - Separates a chunk that barely cleared the threshold from one that dominated the retrieval.
• The requesting identity - Who asked, and what that identity was entitled to.
• The tool calls that followed - What the model did with what it received.

With those in place, the ambiguous case from the top of this piece becomes a question you can answer in one view, because you can see whether the wrong answer came from a chunk that should not have been retrievable, a chunk carrying text nobody wrote deliberately, or a retriever that simply picked poorly.

Attaching customer and identity attributes to the trace is what makes it useful at the fleet level rather than one request at a time. With those you can ask which tenant's queries touched which namespace, and whether any request returned content from outside its own. The production RAG observability guide covers the telemetry layers and the span attributes worth attaching to retrieval and generation.

RAG Security and Privacy Considerations

Security and privacy diverge in RAG pipelines in a way worth separating out. The controls above keep the wrong people and the wrong content out. The considerations below are about data you put in on purpose, which then exists in more copies and more systems than the original.

PII That Survives Chunking and Embedding

Documents that were fine in their original repository become a different exposure once chunked, because a chunk strips the surrounding context that made a piece of information innocuous. A paragraph naming a customer alongside a diagnosis reads as a case note inside a full document and as an unlabeled fact inside a 400-token chunk sitting in a semantic index.

Detection at ingestion is the practical answer, and it is more effective than filtering at output because you only have to do it once per document rather than once per request. Redaction, tokenization, and a policy of excluding whole classes of source from the index are all defensible depending on how much retrieval quality you can afford to lose.

Text Reconstruction From Stored Embeddings

Embeddings are frequently treated as opaque, and they are not. OWASP states plainly under Vector and Embedding Weaknesses that attackers can invert embeddings and recover significant amounts of the source information, which makes a vector store a copy of your sensitive text rather than a derived artifact of it.

The practical consequence is that a vector database inherits the classification of everything embedded into it. Access controls, retention policy, and encryption requirements that apply to the source repository apply to the index too, and a vector store standing outside your data governance because nobody thought of it as a database is a gap worth closing.

Logs and Traces as a Second Copy of Your Sensitive Data

Instrumenting retrieval well enough to investigate incidents means recording chunk content, which means your observability platform now holds the same regulated text as your index. Control 9 and this consideration pull against each other, and pretending otherwise leads to teams either logging nothing or logging everything.

The workable middle is masking at the point of capture and omitting payloads on request paths where the content is never worth the risk. Masking sensitive fields before they leave the process keeps the trace structure, the chunk IDs, the identity, and the tool calls, which is most of what you need for an investigation, while dropping the raw text. Retention limits do the rest, since a trace you keep for 30 days is a smaller liability than one you keep indefinitely.

Retention and Deletion in a Vector Store

Deletion requests are harder here than in a relational database, because one source document has fanned out into many chunks and their embeddings, possibly across more than one index and any downstream copies. If a user exercises a GDPR deletion right, you need to resolve their identity to every derived vector, which is the same provenance metadata from control 1 doing a second job.

Retention windows deserve an explicit decision rather than a default. Indexes tend to accumulate content nobody would re-approve today, and stale documents are both a retrieval quality problem and a compliance one.

Regulated Workloads: HIPAA, GDPR, and What Auditors Ask For

The questions auditors ask about retrieval systems are consistent: what was retrieved, when, by which identity, and what was generated from it. Answering those from application logs after the fact is a reconstruction, and reconstruction is what gets findings written. Which specific records you owe depends on the regime you are operating under, and the breakdown of the major AI governance frameworks covers what each one requires in the way of controls, records, and oversight.

Building the audit trail into the pipeline architecture means the answer is a query rather than a project. For teams operating under HIPAA, that record plus a signed BAA covering every processor in the path, including whichever platform holds your traces, is the baseline the assessment starts from.

Securing RAG Pipelines With Respan

Most of the work above is preventive, and preventive controls have one weakness in common: nothing tells you whether they hold until something gets through. Respan closes that by running the attack yourself first.

Respan's AI agent red teaming connects to your deployed agent, profiles it, and runs adaptive adversarial campaigns against it, changing tactics when a request gets refused rather than replaying a fixed prompt list. Every confirmed finding comes back with the decisive prompt, the response or tool action that confirmed it, and a severity grade, and each campaign closes on a letter grade and a resistance rate your release process can block on.

The same platform then keeps that visibility once real traffic arrives. Every LLM call, retrieval, and tool run becomes a span in one trace with its own input, output, latency, and cost, so a bad answer resolves to the chunks behind it instead of a reconstruction exercise. Instead of reading logs after the fact, use Respan to run observability in production, know when production shifts, and act before it spreads.

• Attack your own pipeline before users do - Campaigns probe for prompt injection, document poisoning, data leakage, cross-customer disclosure, unsafe tool use, and attacker-controlled outbound fetches, against the deployed agent rather than a bare model endpoint.
• Evidence with every finding - Attack prompts, responses, and recorded tool actions are preserved as an audit trail, so a finding is something an engineer can reproduce and fix rather than a score to interpret.
• Test on your terms - Start on a hosted sandbox that never touches your systems, then connect your own agent through a local adapter with credentials that stay on your machine.
• See exactly what your agents did - Traces nest parent to child across retrieval and generation, and threads group multi-turn sessions so a bad output on turn five traces back to context set on turn two.
• Attribute every request - Attach customer IDs and custom attributes to spans, then filter and analyze by them across the platform to answer which tenant, which namespace, and which source.
• Catch unsafe behavior on live traffic - Behavior classifiers semantically sort production traffic into patterns including jailbreak, unsafe, and escalation, with support for your own, and monitors fire to Slack, email, or a webhook the moment a metric or score breaches.
• Score quality continuously - Online evaluators run on live production spans, sampled to control cost, so a retrieval regression surfaces as a moving score rather than a support ticket.
• Control what gets stored - PII masking, omit-logs, configurable retention, and async bulk export to CSV or JSONL keep the trace useful without turning it into a second copy of everything sensitive.
• Route everything through one endpoint - Route, observe, and evaluate every LLM call across 1,000+ models, with automatic failover, spend limits per key and per customer, and caching built in.
• Enterprise compliance posture - SOC 2, HIPAA with a BAA, GDPR, and ISO 27001, with a 99.99% uptime SLA on Enterprise.

How to Implement RAG Security

Shipping nine controls at once is not realistic for most teams, and the order below is sequenced by exposure closed per hour spent rather than by pipeline position.

1. Start with retrieval scoping. Over-permissioned retrieval accounts for more real data exposure in production than any adversarial technique, and it is the control with the clearest definition of done. Pass identity into the query, verify the service account's own permissions, and confirm that a request from one tenant cannot return another tenant's documents. Everything else assumes this is already true.

2. Add ingestion controls next. Hash and source-tag documents as they enter, split read and write credentials, and put sanitization in front of the embedding step. This is a day of work in the ingestion job and it makes every later investigation tractable, since a chunk without provenance is a dead end no amount of downstream tooling recovers from.

3. Instrument before you add detection. Filters and classifiers generate decisions you cannot evaluate without a record of what they saw and what they let through. Getting retrieval spans, chunk IDs, and identity onto the trace first means the detection layer you add afterwards can be tuned against real traffic instead of guessed at. It also means the false positives are visible, which is the number that determines whether the filter survives contact with your users.

4. Red team against a release bar. Adversarial testing that produces a report nobody acts on is theater. Testing that produces a grade and a resistance rate your pipeline can block on becomes a gate, and the difference is whether the output is a number with a threshold attached. Gating a release on a security score works the same way as gating it on an eval score, which is one of the DevOps best practices for AI systems that transfers directly from ordinary CI. Run it against the deployed system rather than a bare model endpoint, because the controls you just shipped live in the application, not the weights. The overview of what red teaming is covers how to structure campaigns, and the roundup of AI red teaming tools compares what different platforms probe.

5. Monitor what you cannot test for. Campaigns cover attacks you thought of. Production covers the rest, so run evaluators on live spans, set monitors on retrieval quality and error patterns, and treat a moving score as an incident signal rather than a dashboard decoration. Retrieval quality and retrieval security degrade through some of the same symptoms, which means the measurement you build for one serves both. Policy enforcement and attestation are a separate category with its own tooling, compared across platforms in the roundup of AI governance tools, while the RAG evaluation guide covers the metrics worth scoring and how to build a golden set from real traffic.

Working through it in this order means each step produces something the next one uses, and you are never adding a control whose effect you cannot see.

Frequently Asked Questions

What is RAG security?

RAG security is the practice of protecting a retrieval-augmented generation pipeline at every stage where untrusted data can enter or sensitive data can leave, covering document ingestion, the vector store, retrieval at query time, and the generated output. The core assumption that separates it from ordinary application security is that retrieved content is untrusted input, because the model cannot distinguish a document's text from your own instructions once both are in the context window.

What is RAG poisoning?

RAG poisoning is an attack where someone writes content into the knowledge base that is crafted both to be retrieved for a chosen query and to steer the model's answer once retrieved. The PoisonedRAG research measured a 90% success rate from five injected texts per target question against a database of millions, and found that existing defenses did not reliably stop it. It differs from prompt injection in the delivery path rather than the effect, since nothing is sent through the chat interface at all.

Is RAG more secure than fine-tuning on private data?

For most teams, yes, though the risks trade rather than disappear. Fine-tuning bakes private data into weights you cannot selectively delete from, which makes retention, deletion rights, and access scoping close to impossible after the fact. RAG keeps the data in a store you can permission, audit, and delete from, at the cost of a live retrieval path an attacker can target. The tradeoff generally favors RAG, provided the retrieval path is scoped by identity and instrumented well enough to see.

How do you prevent prompt injection through retrieved documents?

No single control prevents it, so the practical goal is reducing success rate and containing the blast radius. Sanitize and normalize content before embedding, filter retrieved chunks before they enter the context window, delimit retrieved material as data rather than instruction, and narrow the tools the model can call so a successful injection has nowhere to go. Respan supports the containment half by tracing every retrieval and tool call, running behavior classifiers for jailbreak and unsafe patterns on live traffic, and red teaming the deployed agent for injection paths before release. Dedicated detection layers and their false-positive characteristics are covered in more depth in the prompt injection detection guide.

How do you secure a vector database?

Treat it as a store carrying the same classification as the source documents. Separate read and write credentials, apply retrieval filters at the query rather than to the results, isolate tenants at the namespace or collection level, and bring the store inside the same retention and deletion policy as the repositories feeding it. Log writes so that a document appearing in the index can be traced back to the connector and account that put it there.

What are the RAG security best practices for a multi-tenant app?

Namespace isolation is the one that matters most, because a shared index separated only by a metadata filter puts your entire tenancy boundary inside a query parameter that one code path can forget. Beyond that, scope retrieval by the requesting identity, attach customer IDs to every span so cross-tenant retrieval is visible rather than inferred, and include cross-customer disclosure in your adversarial testing rather than assuming the filter holds. Respan attaches customer attributes to traces and probes for cross-customer disclosure during red team campaigns, which covers the detection and verification sides of the same question.