Article Details

Cloudflare blames recent outage on BGP hijacking incident. Internet giant Cloudflare reports that its DNS resolver service, 1.1.1.1, was recently unreachable or degraded for some of its customers because of a combination of Border Gateway Protocol (BGP) hijacking and a route leak. The incident occurred last week and affected 300 networks in 70 countries. Despite these numbers, the company says that the impact was "quite low" and in some countries users did not even notice it. Incident details Cloudflare says that at 18:51 UTC on June 27, Eletronet S.A. (AS267613) began announcing the 1.1.1.1/32 IP address to its peers and upstream providers. This incorrect announcement was accepted by multiple networks, including a Tier 1 provider, which treated it as a Remote Triggered Blackhole (RTBH) route. The hijack occurred because BGP routing favors the most specific route. AS267613’s announcement of 1.1.1.1/32 was more specific than Cloudflare’s 1.1.1.0/24, leading networks to incorrectly route traffic to AS267613. Consequently, traffic intended for Cloudflare’s 1.1.1.1 DNS resolver was blackholed/rejected, and hence, the service became unavailable for some users. One minute later, at 18:52 UTC, Nova Rede de Telecomunicações Ltda (AS262504) erroneously leaked 1.1.1.0/24 upstream to AS1031, which propagated it further, affecting global routing. This leak altered the normal BGP routing paths, causing traffic destined for 1.1.1.1 to be misrouted, compounding the hijacking problem and causing additional reachability and latency problems. Cloudflare identified the problems at around 20:00 UTC and resolved the hijack roughly two hours later. The route leak was resolved at 02:28 UTC. Remediation effort Cloudflare’s first line of response was to engage with the networks involved in the incident while also disabling peering sessions with all problematic networks to mitigate the impact and prevent further propagation of incorrect routes. The company explains that the incorrect announcements didn’t affect internal network routing due to adopting the Resource Public Key Infrastructure (RPKI), which led to automatically rejecting the invalid routes. Long-term solutions Cloudflare presented in its postmortem write-up include: