The customer's security team asks where their data goes when it is sent to the model. How do you answer?

Answer with a data-flow diagram and the provider's written terms, not reassurance. Show every place data travels and rests (the application, the model provider or cloud platform, logs, caches, vector stores, evaluation sets), and for each one state: who can access it, where it is located, how long it is kept, and whether it is used for training. Then offer the options that close any gaps: a zero-retention arrangement, a cloud deployment in their own account and region, masking, and shorter log retention.

1. Draw the flow

User → the customer's application → (masking) → model endpoint → response → application → logs and traces. Add retrieval stores, caches, file uploads and any third-party tools. Security teams trust a diagram they can question.

2. The model provider's terms, from the documents

Using Anthropic as an example, its documentation says:

• Training: inputs and outputs from commercial products, the API included, are not used to train models by default.
• Retention: the API does not keep prompts and responses by default, except for certain models that require 30-day retention. Zero data retention can be arranged per organisation. Some features that store data by nature (files, batches, code execution containers) have their own retention and may not be covered.
• Cloud platforms: on Amazon Bedrock or Google Cloud, the cloud provider is the data processor. AWS says Bedrock's model providers have no access to customer prompts and completions, and offers encryption and private network connections (VPC, PrivateLink).

Always link the current documents and contract terms. Never quote them from memory: they change.

3. Your application's part

Often the bigger exposure is your own system: prompts in logs, traces in an observability tool, documents in a vector store. State the location, access controls, encryption and retention of each, and how deletion requests are handled.

4. Offer controls

• Masking of personal data before the call.
• Region and residency choices; deployment in the customer's own cloud account.
• Zero-retention arrangements where needed.
• Short log retention, restricted access, and no production data in evaluation sets without anonymisation.
• Single sign-on and role-based access for the application itself.

5. Paperwork

Data processing agreement, sub-processor list, security certifications (for example SOC 2 or ISO 27001 reports from each vendor), and a completed security questionnaire. Have a standard pack ready. It shortens every deal.

Tone

Be precise and honest about what you do not know ("I'll confirm the retention for batch jobs in writing"). A security team's trust is lost by one over-confident answer.