Data, privacy, and security for Azure Direct Models in Azure AI Foundry

This article provides details regarding how data provided by you to Azure Direct Models in Azure AI Foundry are processed, used, and stored. Azure Direct Model means an AI model designated and deployed as an “Azure Direct Model” in Azure AI Foundry, and includes Azure OpenAI models. Azure Direct Models store and process data to provide the service and to monitor for uses that violate the applicable product terms. Please also see Microsoft Products and Services Data Protection Addendum, which governs data processing by Azure Direct Models. Azure AI Foundry is an Azure service; learn more about applicable Azure compliance offerings.

What data does Azure AI Foundry process to provide Azure Direct Models?

Azure AI Foundry processes the following types of data to provide Azure Direct Models:

• Prompts and generated content. When prompts are submitted by the user, content is generated by the service, via the completions, chat completions, images, and embeddings operations.
• Uploaded data. You can upload your own data for use with certain service features (e.g., fine-tuning, assistants API, batch processing) using the Files API or vector store.
• Data for stateful entities. When you use certain optional features of Azure Direct Models and Agents, such as the Responses API, the Threads feature of the Assistants API, and Stored completions, the service creates a data store to persist message history and other content, in accordance with how you configure the feature.
• Augmented data included with or via prompts. When you use data associated with stateful entities, the service retrieves relevant data from a configured data store and augments the prompt to produce generations that are grounded with your data. Prompts may also be augmented with data retrieved from a source included in the prompt itself, such as a URL.
• Training & validation data. You can provide your own training data consisting of prompt-completion pairs for the purposes of fine-tuning a model.

How does Azure AI Foundry process data to provide Azure Direct Models?

The diagram below illustrates how your data is processed. This diagram covers several types of processing:

1. How Azure AI Foundry processes your prompts via inferencing with Azure Direct Models to generate content (including when additional data from a designated data source is added to a prompt using Azure OpenAI on your data, Assistants, or batch processing).
2. How the Assistants feature stores data in connection with Messages, Threads, and Runs.
3. How the Responses API feature stores data to persist message history.
4. How the Batch feature processes your uploaded data.
5. How Azure AI Foundry creates a fine-tuned (custom) model with your uploaded data.
6. How Azure AI Foundry and Microsoft personnel analyze prompts and completions (text and image) for harmful content and for patterns suggesting the use of the service in a manner that violates the Code of Conduct or other applicable product terms.

As depicted in the diagram above, managed customers may apply to modify abuse monitoring.

Generating completions, images or embeddings through inferencing

Azure Direct Models (base or fine-tuned) deployed in your Azure AI Foundry resource process your input prompts and generate responses with text, images, or embeddings. Customer interactions with the model are logically isolated and secured employing technical measures including but not limited to transport encryption of TLS1.2 or higher, compute security perimeter, tokenization of text, and exclusive access to allocated GPU memory. Prompts and completions are evaluated in real time for harmful content types and content generation is filtered based on configured thresholds. Learn more at Content filtering overview.

Prompts and responses are processed within the customer-specified geography (unless you are using a Global or DataZone deployment type), but may be processed between regions within the geography for operational purposes (including performance and capacity management). See below for information about location of processing when using a Global or DataZone deployment type.

The models are stateless: no prompts or completions are stored in the model. Additionally, prompts and completions are not used to train, retrain, or improve the base models.

Understanding location of processing for "Global" and "Data zone" deployment types

In addition to standard deployments, Azure AI Foundry offers Azure Direct Model deployment options labeled as 'Global' and 'DataZone.' For any deployment type labeled 'Global,' prompts and responses may be processed in any geography where the relevant Azure Direct Model is deployed (learn more about region availability of models). For any deployment type labeled as 'DataZone,' prompts and responses may be processed in any geography within the specified data zone, as defined by Microsoft. If you create a DataZone deployment in an Azure AI Foundry resource located in the United States, prompts and responses may be processed anywhere within the United States. If you create a DataZone deployment in an Azure AI Foundry resource located in a European Union Member Nation, prompts and responses may be processed in that or any other European Union Member Nation. For both Global and DataZone deployment types, any data stored at rest, such as uploaded data, and including the abuse monitoring data store created for Global and DataZone deployments, is stored in the customer-designated geography. Only the location of processing is affected when a customer uses a Global deployment type or DataZone deployment type in Azure Direct Models; Azure data processing and compliance commitments remain applicable.

Augmenting prompts to "ground" generated results "on your data"

The Azure OpenAI "on your data" feature lets you connect data sources to ground the generated results with your data. The data remains stored in the data source and location you designate; Azure OpenAI does not create a duplicate data store. When a user prompt is received, the service retrieves relevant data from the connected data source and augments the prompt. The model processes this augmented prompt and the generated content is returned as described above. Learn more about how to use the On Your Data feature securely.

Data storage for Azure Direct Models features

Some Azure Direct Models features store data in the service. This data is either uploaded by the customer, using the Files API or vector store, or is automatically stored in connection with certain stateful entities such as the Responses API, the Threads feature of the Assistants API, and Stored completions. Data stored for such features:

• Is stored at rest in the Azure AI Foundry resource in the customer's Azure tenant, within the same geography as the resource;
• Is always encrypted at rest with Microsoft’s AES-256-encryption by default, with the option of using a customer managed key (certain preview features may not support customer-managed keys). Microsoft-managed keys are always used to ensure baseline encryption for all stored data.
• Can be deleted by the customer at any time.

Stored data may be used with the following service features/capabilities:

• Creating a customized (fine-tuned) model. Learn more about how fine-tuning works. Fine-tuned models are exclusively available to the customer whose data was used to create the fine-tuned model, are encrypted at rest (when not deployed for inferencing), and can be deleted by the customer at any time. Training data uploaded for fine-tuning is not used to train any generative AI foundation models without your permission or instruction.
• Batch processing. Learn more about how batch processing works. Batch processing is a Global deployment type; data stored at rest remains in the designated Azure geography until processing capacity becomes available; processing may occur in any geography where the relevant Azure Direct Model is deployed (learn more about region availability of models).
• Responses API. Learn more about how the Responses API works. This API stores message history and other content related to message history. This is required for multi-turn conversations and workflows.
• Assistants API (preview). Learn more about how the Assistants API works. Some features of Assistants, such as Threads, store message history and other content.
• Stored completions (preview). Stored completions stores input-output pairs from the customer’s deployed Azure OpenAI models such as GPT-4o through the chat completions API and displays the pairs in the Azure AI Foundry portal. This allows customers to build datasets with their production data, which can then be used for evaluating or fine-tuning models (as permitted in applicable Product Terms).

Preventing abuse

To reduce the risk of abusive or harmful use, Azure Direct Models includes abuse monitoring features. To learn more about abuse monitoring, see abuse monitoring.

Safety evaluations of fine-tuned models evaluate a fine-tuned model for potentially harmful responses using Azure’s risk and safety metrics. Only the resulting assessment (deployable or not deployable) is logged by the service.

The Azure Direct Models abuse monitoring system is designed to detect and mitigate instances of recurring content and/or behaviors that suggest use of the service in a manner that may violate the code of conduct or other applicable product terms. As described here, the system employs algorithms and heuristics to detect indicators of potential abuse. When these indicators are detected, a sample of customer’s prompts and completions may be selected for review. Review is conducted by automated means including by AI models such as LLMs by default, with additional reviews by human reviewers as necessary. Detailed information about automated review and human review is available at Abuse monitoring.

For automated review, customer’s prompts and completions are not stored by the system or used to train the AI models or other systems. The abuse monitoring data store where prompts and completions are stored for human review is logically separated by customer resource (each request includes the resource ID of the customer’s Azure AI Foundry resource). A separate data store is located in each geography in which the Azure Direct Model is available, and a customer’s prompts and generated content are stored in the Azure geography where the customer’s Azure AI Foundry resource is deployed, within the Azure Direct Models service boundary. Human reviewers assessing potential abuse can access prompts and completions data only when that data has already been flagged by the abuse monitoring system, or when the prompts and completions are part of a potentially abusive pattern of use. The human reviewers are authorized Microsoft employees who access the data via point wise queries using request IDs, Secure Access Workstations (SAWs), and Just-In-Time (JIT) request approval granted by team managers. For Azure Direct Models deployed in the European Economic Area, the authorized Microsoft employees are located in the European Economic Area.

If the customer has been approved for modified abuse monitoring (learn more at Abuse Monitoring)), the data storage and human review process described above is not performed. However, automated review may still be conducted, leveraging algorithms including AI models that review prompts and completions at the time provided or generated, as applicable. If such automated review detects content potentially indicating severe or recurring abuse in the customer’s subscription, the customer may be subject to limitations on access, as provided in the Product Terms for Responsible Use of Microsoft AI Services. The customer may also be asked to agree to have abuse monitoring with human review turned on to reduce the risk of future limitations on access (e.g., throttling and/or suspension of the account or subscription where abuse has been detected).

Preventing harmful content generation

Azure Direct Models include a system designed to detect and prevent the output of harmful content. To learn more about content filtering, see Content Filtering.

Content filtering occurs synchronously as the service processes prompts to generate content as described above and here. No prompts or generated content are stored in the content classifier models, and prompts and outputs are not used to train any generative AI foundation models without your permission or instruction.

How can a customer verify if data storage for abuse monitoring is off?

There are two ways for customers, once approved to turn off abuse monitoring, to verify that data storage for abuse monitoring has been turned off in their approved Azure subscription:

• Using the Azure portal, or
• Azure CLI (or any management API).

Prerequisites

1. Sign into Azure
2. Select the Azure Subscription which hosts the Azure AI Foundry resource.
3. Navigate to the Overview page of the Azure AI Foundry resource.

{ 
    "name":"ContentLogging",
    "value":"false"
}

az cognitiveservices account show -n resource\_name -g resource \_group

To learn more about Microsoft's privacy and security commitments see the Microsoft Trust Center.

Change log

See also

• Code of conduct for Azure OpenAI Service integrations
• Overview of Responsible AI practices for Azure OpenAI models
• Transparency note and use cases for Azure OpenAI Service
• Data Residency in Azure
• Compare Azure OpenAI in Azure Government
• Limited access to Azure OpenAI Service
• Report abuse of Azure OpenAI Service through the Report Abuse Portal
• Report problematic to cscraireport@microsoft.com