Skip to main content
azuredown
criticalAzure DNSGlobal

The May 2019 Azure DNS Outage: A Delegation Change Gone Wrong

AzureDown Research · Incident analysisPublished May 6, 2019Updated June 18, 20267 min read

Timeline

  1. Impact begins

    As part of a planned migration, a change to name-server delegation propagates and a subset of DNS queries for Microsoft and Azure domains begin failing to resolve. Customers see connection errors to services that are otherwise operational.

  2. Investigating

    Microsoft acknowledges that customers may be unable to reach multiple Azure services and Microsoft 365, and identifies DNS resolution as the common factor rather than the individual services.

  3. Identified

    The delegation change introduced during the migration is identified as the cause. Engineers begin reverting the change and correcting the affected DNS records.

  4. Mitigating

    The correction propagates and resolution success rates climb. Cached negative responses and TTLs slow full recovery for some customers.

  5. Resolved

    DNS resolution for Azure and Microsoft services returns to normal globally. Residual failures clear as caches expire.

Root cause

The incident happened during routine maintenance - a planned migration of Azure DNS infrastructure - which is exactly when the highest-blast-radius mistakes tend to occur, because change is being made deliberately to load-bearing systems. A name-server delegation change propagated as part of the migration and left a subset of authoritative queries for Microsoft and Azure domains unable to resolve. Nothing downstream had failed: Storage was serving, SQL Database was accepting connections, Microsoft 365 was up. Clients simply could no longer translate the names into addresses, which is functionally identical to those services being gone.

DNS failures are uniquely deceptive for this reason. Every dashboard for the individual service stays green while customers report a total outage, and the correlation - "everything Microsoft is unreachable at once" - is the tell that the fault is in the shared resolution layer rather than in any one product.

Business impact

Because the failure was at the DNS layer, the blast radius was broad and shallow: many services affected, each of them healthy underneath. Enterprises lost access to Azure Storage and SQL Database-backed applications, and Microsoft 365 users saw sign-in and connectivity failures. Recovery had a long tail that Microsoft could not fully control - once a bad or missing DNS answer is cached, the client honors it until the TTL expires, so some customers recovered minutes after the fix and others considerably later.

Prevention and lessons

  1. Treat DNS changes as the highest-risk category. Delegation and record changes have no partial blast radius - they either resolve or they don't, globally. Stage them behind long-lead-time review and change windows, and rehearse the rollback before the change, not during the incident. Our business continuity hub builds this into the change-management tier.
  2. Keep TTLs deliberately short on critical records so that when a bad change does propagate, the correction propagates just as fast. Long TTLs turn a three-hour fix into a six-hour outage for the unlucky.
  3. Don't co-locate your own DNS dependency with the thing being changed. Applications that cached resolved endpoints or used secondary resolution rode parts of this out; those that re-resolved on every request during the window did not.
  4. Watch Azure DNS status directly. During DNS incidents, per-service status pages lag reality because the services themselves are healthy - the resolution layer is where the truth is.

SLA credit eligibility

Yes for many affected workloads - unreachability counts as unavailability under most service SLAs. Around three hours of impact approaches the 99.99% monthly threshold (~4.3 minutes of allowed downtime) for zone-redundant services if fully unreachable; claimable at the 10% tier with evidence, filed within Azure’s ~two-month window.

Work out what your bill was owed with the independent Azure SLA credit calculator, or automate breach detection with Next Signal (sponsor). For the full claim process, see the Azure credit-recovery playbook.

Sources: Azure status history, CloudDowntime - cloud reliability research

Questions about this outage

What caused the May 2019 Azure outage?

A name-server delegation change made during a planned Azure DNS migration disrupted resolution for Microsoft and Azure domains. The affected services - Storage, SQL Database, Microsoft 365 - were healthy; customers simply could not resolve their addresses to reach them.

How long did the May 2019 Azure DNS outage last?

Roughly three hours of global impact (about 19:29 to 22:35 UTC on May 2, 2019), with a tail as DNS caches and TTLs expired for individual customers.

Why does a DNS problem take down so many services at once?

DNS is the lookup layer in front of everything. When authoritative resolution for a provider’s domains fails, clients cannot find any service on those domains - so a single DNS fault presents as a simultaneous outage of storage, databases, identity, and productivity apps that are all individually fine.

Did the May 2019 Azure DNS outage qualify for SLA credits?

For affected workloads, roughly three hours of unreachability in the month can breach the relevant service SLAs, putting customers in the 10% credit tier where they could evidence the impact. Azure requires the claim to be filed within about two months of the billing month.

Next time, hear it from us first

Instant alerts when Azure incidents are detected - services and regions included.

Get outage alerts

More post-mortems

Storage

Azure Central US Outage (July 18–19, 2024): The Storage Allowlist Failure, Explained

An allowlist configuration error on Azure Storage scale units made storage unavailable across an availability zone in Central US. Because Virtual Machines cannot run without their disks, the failure cascaded into VMs, dependent Azure services, and Microsoft 365 for many hours - landing in the same week as the unrelated CrowdStrike incident.

Entra ID

The June 2023 Layer-7 DDoS Attacks on Microsoft: Portal, Outlook, and OneDrive Disrupted

Layer-7 DDoS attacks - attributed by Microsoft to the actor it tracks as Storm-1359 - flooded the web front ends of the Azure Portal, Outlook on the web, and OneDrive across several days in early June 2023. Authentication and portal access failed intermittently, showing how an attack on web tiers reads as an identity outage to the business.