Disaster Recovery for Azure SRE Agent

This article explains how to prepare Azure SRE Agent for a regional outage by using a pre-provisioned agent in a secondary Azure region.

Azure SRE Agent is a regional service. Each agent is deployed to one region and has its own resource ID, endpoint, configuration, and runtime state. Azure SRE Agent doesn't automatically fail over an individual agent between regions.

For workloads that require faster recovery than rebuilding an agent during an outage, use a cold-standby architecture:

• Deploy a primary agent and a backup agent in different supported regions.
• Apply the same source-controlled definitions to both agents, with production automations disabled on the backup.
• Attach the same user-assigned managed identity to both agents.
• Keep the primary agent running and the backup agent stopped during normal operation.
• If the primary region becomes unavailable, start the backup agent and redirect integrations and users to its endpoint.

A standby agent is an independent resource, not a replica of the primary agent. Conversation history, accumulated memory, and in-flight work don't transfer between agents.

---

Scope and definitions​

This article covers disaster recovery for the SRE Agent resource and the configuration, identity, and operational dependencies attached to it.

Term	Definition
Recovery time objective (RTO)	The maximum acceptable time between the start of a disruption and restoration of service through the backup agent.
Recovery point objective (RPO)	The maximum acceptable amount of configuration or runtime state that can be lost during recovery.
Cold standby	A fully deployed backup agent that remains stopped until a disaster is declared.
Blast radius	The scope affected by a failure. A regional failure affects agents deployed in that region.

Microsoft doesn't publish an RTO or RPO commitment for disaster recovery of an individual agent. Establish targets based on your requirements and validate them through regular recovery drills.

This article doesn't cover:

• Disaster recovery for the Azure resources managed by the agent.
• Azure platform-level recovery of an affected region.
• Disaster recovery for external incident management, source control, or observability systems.

---

Cold-standby architecture​

The deployment consists of two independent SRE Agent resources:

Component	Primary region	Backup region
SRE Agent resource	Running	Stopped
Agent configuration	Deployed from source control	Deployed from the same source
User-assigned managed identity	Shared	Shared
Agent endpoint	Primary production endpoint	Backup endpoint used after failover
Network integration	Configured for the primary region	Configured separately for the backup region
Runtime state and history	Stored with the primary agent	Independent and initially empty

The shared user-assigned managed identity provides both agents with the same Azure role assignments. Because the identity is a separate Azure resource, its permissions remain available when either agent is stopped or unavailable.

The agents still have distinct system-assigned identities. If any dependency uses a system-assigned identity instead of the shared user-assigned identity, configure and validate its permissions separately.

---

Shared responsibility model​

Disaster recovery for SRE Agent is a shared responsibility.

Responsibility	Owner	Detail
Platform availability and recovery within a region	Microsoft	Microsoft operates the SRE Agent platform and restores affected regional services.
Deploying agents in two regions	You	Provision the primary and backup agents before an incident.
Maintaining equivalent definitions	You	Store the definitions in source control, deploy them consistently to both agents, and keep backup production automations disabled.
Managing permissions	You	Attach the shared user-assigned managed identity and maintain its role assignments.
Detecting an outage and declaring failover	You	Use Azure Service Health and your monitoring and incident processes.
Activating the backup agent	You	Confirm the primary is unavailable, then start the backup resource.
Redirecting users and integrations	You	Update callers to use the backup agent's endpoint.
Validating connectors and automations	You	Confirm each dependency works from the backup region.
Managing runtime-state loss	You	Conversation history, memory, and in-flight work don't move to the backup agent.

---

Disaster recovery readiness checklist​

Complete these steps before an incident.

• Select two supported regions. Choose primary and backup regions that meet your availability and data residency requirements.
• Define both agents as infrastructure as code. Store the agent resources and supported child-resource configuration in source control. See Deploy with Infrastructure as Code.
• Use one deployment source. Deploy equivalent definitions to both agents and detect configuration drift. Preserve an intentional environment-specific override that keeps production automations disabled on the backup.
• Attach a shared user-assigned managed identity. Grant the identity the least-privilege roles required by the agents.
• Review system-assigned identity dependencies. Record and separately configure anything that doesn't use the shared identity.
• Configure regional networking. Provision the backup region's virtual network, subnet, DNS, routes, and firewall rules. Configure private endpoints for dependencies reached through the agent's outbound VNet path. VNet integration is outbound only, and platform and connector traffic can use Microsoft-managed infrastructure. See Network Controls and Network Requirements.
• Inventory connectors and credentials. Identify connectors that require interactive sign-in or region-specific configuration.
• Inventory and fence every activation source. Keep scheduled tasks, incident response plans, HTTP triggers, other triggers and scanners, and tenant-specific automation disabled on the backup. Prevent production callers and upstream integrations from reaching the backup, and verify it has no in-flight runs before stopping it.
• Preserve knowledge sources. Keep uploaded documents in an external system of record so they can be deployed or uploaded to either agent.
• Record both endpoints. Document every caller that must switch from the primary endpoint to the backup endpoint.
• Keep the backup stopped. Verify its powerState is Stopped during normal operation.
• Run an isolated recovery drill. Keep production automations disabled, use dedicated read-only test definitions and nonproduction destinations, and route only isolated test callers to the backup. Don't manually invoke production definitions. After the drill, remove test routing, confirm all test and production automations on the backup are disabled with no runs in progress, stop the backup, and verify powerState is Stopped.

---

Recovery characteristics​

Component or state	Available after failover?	Notes
Agent resource	Yes	The backup is deployed before the incident and started during failover.
Source-controlled configuration	Yes	Available up to the last successful deployment to the backup agent.
Shared UAMI role assignments	Yes	Both agents use the same identity and permissions.
Regional network integration	Yes, if preconfigured	The backup requires its own regional networking resources.
Connector definitions	Depends	Definitions may be deployed, but credentials and connectivity must be validated separately.
Scheduled tasks and HTTP triggers	Depends	Configuration may exist, but activation and caller routing must be tested.
Uploaded knowledge	Only if copied separately	Keep the source outside the primary agent.
Conversation threads and history	No	History is tied to the primary agent and isn't copied to the backup.
Accumulated memory and session insights	No	The backup builds its own state after activation.
In-flight investigations and runs	No	Restart interrupted work on the backup agent.
Primary agent endpoint	No	The backup has a different endpoint. Callers must be redirected.
System-assigned identity permissions	No, unless configured	The backup has a different system-assigned identity principal.

---

Recovery objectives​

Set business RTO and RPO targets before running a recovery drill. Use each drill to validate the targets and improve the procedure.

RTO starts when the disruption begins, not when failover is declared. It therefore includes detection, operator decision, activation, endpoint switching, and validation. During a declared failover, measure from the disruption time until:

1. The backup agent reports Running.
2. Required health and connectivity checks pass.
3. Users and integrations are redirected to the backup endpoint and paused production callers are resumed.
4. Required production automations are enabled only on the backup.
5. A representative investigation completes successfully.
6. Monitoring confirms that duplicate execution isn't occurring.

During a drill, measure the equivalent isolated test steps without enabling production automations or resuming production callers. Record separately any production cutover steps that the drill simulated rather than executed.

RPO varies by state type:

• Configuration RPO is the time since the last successful deployment to the backup agent.
• Knowledge RPO depends on how frequently external knowledge sources are synchronized.
• Conversation, memory, and in-flight execution RPO is total loss because those states don't transfer between agents.

Don't publish targets based only on estimated startup or deployment times.

---

Fail over to the backup agent​

1. Declare the regional outage​

Use Azure Service Health and your monitoring signals to confirm that the primary agent or region is unavailable.

Record the incident start time, affected agent and region, person authorizing failover, and expected impact on active investigations and automations.

2. Prevent simultaneous production activity​

Establish a fence before activating the backup. Don't treat an unresponsive primary endpoint as proof that outbound scheduled work, scanners, or incident processing has stopped.

Before starting the backup:

• Confirm that the primary agent is Stopped, or independently disable or isolate every production activation source on the primary or its upstream system. Verify that no primary runs remain active, or independently fence the downstream effects of any run that can't be stopped.
• Pause or queue push-based production callers, including HTTP trigger callers and webhooks.
• Confirm that all production automations on the backup are disabled and that it has no in-flight runs.
• Verify that incident integrations, trigger routing, scanners, manual invocation paths, and other automation callers can't activate both agents.

Don't start the backup until each production execution source has only one possible active owner. If the primary region can't be reached, use upstream integration controls and monitoring evidence to establish the fence.

3. Start the backup agent​

Start the SRE Agent resource in the backup region. See the control-plane operations for the start operation and resource properties.

Wait until:

• provisioningState is Succeeded.
• powerState is Running.
• The backup endpoint responds successfully.

4. Validate dependencies​

Before redirecting production traffic, verify:

• The shared user-assigned managed identity is attached.
• Required Azure role assignments remain effective.
• Network, DNS, firewall, and private connectivity work from the backup region.
• Required connectors can authenticate and query their dependencies.
• Knowledge sources required for investigations are available.
• A representative read-only investigation completes successfully.
• Production automations remain disabled on the backup after startup, and no unexpected or in-flight runs are present.

5. Switch callers to the backup endpoint​

While push-based production callers remain paused or queued, update every dependency that references the primary endpoint, including:

• User bookmarks and operational links.
• HTTP trigger callers.
• Incident management integrations.
• Webhooks and automation systems.
• Internal routing, aliases, or configuration stores.

The endpoint switch is manual unless your organization provides a separate routing layer. Verify with an isolated test request that routing targets only the backup before resuming production callers.

6. Activate production work on the backup​

After the primary fence and backup routing are validated:

• Enable the required scheduled tasks, incident response plans, triggers, scanners, and other production automations only on the backup.
• Enable backup HTTP triggers before resuming their paused or queued callers.
• Resume production callers and verify requests reach only the backup.
• Confirm that the primary remains fenced and doesn't start new work.

7. Monitor the backup agent​

After activation, monitor agent health, request success rate, connector failures, automation execution, authorization failures, and duplicate investigations or actions. Compare the measured recovery time with your target RTO.

---

Fail back to the primary region​

Failback is a controlled operation. Runtime state created by the backup agent can't be merged into the original agent.

1. Confirm the primary region and original agent are healthy.
2. Reconcile source-controlled configuration so both agents have the intended version.
3. Record work performed by the backup that must be retained outside its conversation history.
4. Pause production callers and disable new automation activation on the backup.
5. Wait for active backup runs to complete, or terminate them and record any interrupted work that must be restarted.
6. Stop the backup agent.
7. Confirm that every production activation source on the primary is disabled or isolated before startup.
8. Start the primary agent and wait for powerState to become Running.
9. Validate identity, networking, connectors, knowledge, and a representative read-only investigation while primary production automations remain disabled.
10. Redirect paused callers to the primary endpoint and verify with an isolated test request that only the primary is targeted.
11. Enable the required production automations on the primary, then resume callers.
12. Confirm the backup remains stopped, its production automations remain disabled, and no runs are in progress.
13. Record the failback results and update the runbook with lessons from the event.

If continued operation in the backup region is preferred, designate the backup agent as the new primary and keep the recovered original agent stopped.

---

Business continuity​

Disaster recovery is one part of business continuity. Plan how the team operates while the agent is unavailable or failover is in progress.

• Keep operational runbooks executable by a person.
• Preserve incident management and escalation paths that don't depend on the agent.
• Route work to the existing on-call process while recovery is underway.
• Restart interrupted investigations after the backup becomes active.

---

Related​

Resource	Description
Supported regions →	Regions where Azure SRE Agent is available
Deploy with Infrastructure as Code →	Define and deploy agents from source control
Data Privacy and Residency →	How Azure SRE Agent stores and processes data
Network Requirements →	Firewall and regional connectivity requirements
Audit Agent Actions →	Retain an independent record of agent activity