Your prompts are shipping customer PII to a third party

You built an AI feature that summarises support tickets, or drafts replies, or answers questions over your own data. To do its job it needs the ticket, the customer record, the order. So your code takes that record - name, email, phone, maybe a card or a health note - drops it into a prompt, and sends it to a model provider you do not control. That prompt is now a data transfer. It gets logged on their side, may be retained for weeks, and on some plans can be used to improve their models. You did not decide to share your customers with a third party. Your prompt did it for you.

What actually leaves your network in a prompt

A prompt feels like a private instruction to a machine. It is not. It is an HTTPS request to another company's servers, and the whole body - system prompt, retrieved context, and the user's data you stuffed into it - lands in their logs. Under UK GDPR and US state privacy law, sending a person's name, email, phone, address, or payment or health data to a processor is a data transfer you have to account for. If your data protection agreement or your customer contracts do not cover it, you have a compliance gap that no one signed off on.

The uncomfortable part is how quietly it happens. You rarely write "send this customer's PII to OpenAI". You write "summarise this ticket", and the ticket happens to contain a full name, an email in the signature, and an order number that maps to a person. Retrieval-augmented features make it worse: you pull the top matching documents and paste them in whole, so whatever PII lives in those documents rides along. The data leaves before anyone reviews what was in it.

The three ways to redact, and when to use each

Redaction is not one technique. There are three, and the right choice depends on whether the model needs the real value to do its job.

• Block. Some data should never reach a model at all - full card numbers, government IDs, raw health records. Detect it and refuse the call, or strip the field entirely. This is the safest option when the model does not need that value to produce a good answer.
• Mask. Replace the sensitive value with a fixed token - "customer name", "[EMAIL]", "card ending 4242" - and send that. The model reasons about the placeholder instead of the person. Use this when the model needs to know a name exists but not what it is: drafting a reply that says "Hi [FIRST_NAME]" works fine on a mask.
• Tokenize (reversible). Swap each real value for a unique placeholder, keep the mapping in your own store, send the masked prompt, then substitute the real values back into the model's output before the user reads it. This is what you want when the reply must contain the real name or order number: the model never sees the real data, but your user still gets a personalised answer.

In practice a good redaction layer uses all three by data type: block the card number, tokenize the name and email so the reply reads naturally, mask the free-text address. The point is that the decision is deliberate and per-field, not "hope the model behaves".

Detection: why regex is a floor, not a ceiling

To redact PII you first have to find it. Regular expressions catch the data with a fixed shape: email addresses, card numbers you can Luhn-check, UK postcodes, phone numbers. Keep those - they are cheap and reliable for structured patterns. But a huge amount of PII has no shape. A person's name is just words. A home address is free text. An account reference could be anything. Regex will miss all of it.

Two things close the gap. First, named-entity recognition - a model trained to tag people, places, and organisations in text - catches the unstructured identifiers regex cannot. Second, and often overlooked: you already know where the PII is. It is in your database columns. If you are building the prompt from a customer record, redact the fields you control before they ever become free text, rather than trying to find them again after you have flattened everything into a string. Structured redaction at the source is more reliable than pattern-matching after the fact. Getting this wrong is one of the ways an AI feature quietly does the wrong thing - the same class of problem we cover in guardrails and safety nets for AI.

Where the redaction layer belongs

Scattering redaction across features is how it fails - one feature forgets, and that is the leak. Put it in one place: a gateway or proxy that every model call routes through. Outbound, it detects and redacts before the request leaves; inbound, it re-substitutes tokenized values into the response. Because every call passes the same audited component, you can log what was detected and prove to a customer or an auditor exactly what left your network and what did not.

This is also where rate limiting, provider fallback, and cost controls live, so the gateway earns its place several times over. When we build production agents, the redaction proxy and the reliability layer are the same component - it is the natural home for the controls that keep an AI feature both safe and dependable. It sits alongside the wider program of controls in an AI compliance checklist for GDPR, DPDP, and the EU AI Act, and for regulated data it pairs with the infrastructure in our guide to Bedrock setup when your agent touches PHI. This is the kind of work our AI development team and AI agent builds put in from day one, not bolted on after a review.

Frequently asked questions

Does turning off training on my provider account make redaction unnecessary?

No. An enterprise plan that excludes your data from training is worth having, but the data is still transmitted to and logged by the provider, still sits in their systems for a retention window, and is still a transfer you have to account for under privacy law. Redaction means the sensitive values never leave your network in the first place, which is a stronger position than a contractual promise about what happens after they arrive.

Will redacting names and details make the model's answers worse?

Rarely, if you tokenize instead of dropping. The model reasons about a placeholder just as well as a real name, and you swap the real value back into the reply afterward, so the user sees a fully personalised answer. The only time quality drops is when a real value genuinely carries meaning the model needs - and those cases are specific enough to handle deliberately rather than by sending everything.

Can I just tell staff not to paste customer data into AI tools?

Policy helps for people using a chatbot directly, but it does nothing for your own applications, which assemble prompts in code without a human in the loop. The leak in most businesses is not an employee pasting a record - it is a feature that pulls records automatically. That has to be fixed in the code path, not in a policy document.

Is a redaction layer expensive to build?

Far less than a breach or a regulator's finding. A working layer is a gateway that runs pattern and entity detection on outbound text, a small store for reversible tokens, and a re-substitution step on the way back. It is a contained piece of work, and because it sits in one place it protects every current and future AI feature at once. The cost is a few days of engineering against the risk of shipping customer data you were not cleared to share.

The short version: your AI features are only as private as the prompts they send, and right now those prompts are probably carrying more personal data than anyone signed off on. Detect it, redact it by type before it leaves, and do it at one gateway you can audit. Then your team gets the AI feature they wanted, and your customers keep the privacy they were promised.