Massive Outage Reported by AT&T Customers, Disrupting Phone Service Nationwide
Brian Fung, CNN | 2/22/2024, 11:57 a.m.
AT&T’s network went down for many of its customers across the United States Thursday morning, leaving customers unable to place calls, text or access the internet. By late morning, the company said most of its network had been restored.
Although Verizon and T-Mobile customers reported some network outages, too, they appeared far less widespread. T-Mobile and Verizon said their networks were unaffected by AT&T’s service outage and customers reporting outages may have been unable to reach customers who use AT&T.
Thursday morning, more than 74,000 AT&T customers reported outages on digital-service tracking site DownDetector, with service disruptions beginning around 4 am ET. That’s not a comprehensive number: It tracks only self-reported outages. Reports had been rising steadily throughout the morning but leveled off in the 9 am ET hour. By 12:30 pm ET, the DownDetector data showed some 25,000 AT&T customers still reporting outages.
AT&T acknowledged that it has a widespread outage but did not provide a reason for the system failure.
“Some of our customers are experiencing wireless service interruptions this morning. We are working urgently to restore service to them,” AT&T said in a statement. “We encourage the use of Wi-Fi calling until service is restored.”
To set up Wi-Fi calling, users can go to their Settings app on their phone. iPhone users should tap “Cellular” and Android users should click “Connection” and then users will be prompted to turn on the Wi-Fi calling feature. AT&T says on its website that there is no extra cost for this feature. Once set up, Wi-Fi calling works automatically when you’re connected to a Wi-Fi that you choose.
By late morning, AT&T said most of its network was back online.
“Our network teams took immediate action and so far three-quarters of our network has been restored,” the company said. “We are working as quickly as possible to restore service to remaining customers.”
The company did not have a timeframe for when its system would be fully restored. AT&T has been responding to customer complaints online, asking them to send direct messages to customer service.
Why AT&T went down
AT&T has encountered sporadic outages over the past few days, including a temporary 911 outage in some parts of the southeastern United States. Although outages happen from time to time, nationwide, prolonged outages are exceedingly rare.
Although AT&T provided no official reason for the outage, the issue appears to be related to how cellular services hand off calls from one network to the next, a process known as peering, according to an industry source who spoke on the condition of anonymity.
There’s no indication that Thursday’s outage was the result of a cyberattack or other malicious activity, the industry source said.
However, the US Cybersecurity and Infrastructure Security Agency is “working closely with AT&T to understand the cause of the outage and its impacts, and stand[s] ready to offer any assistance needed,” Eric Goldstein, the agency’s executive assistant director for cybersecurity, said in a statement to CNN.
Verizon believes the nationwide outage involving AT&T customers “is close to being resolved,” according to Richard Young, a Verizon spokesman.
Carriers are notoriously mum about why their networks go down. In the past, there have been construction accidents that have cut fiberoptic cables, incidents of sabotage or network updates filled with bugs that became difficult to roll back.
Outages often happen for mundane reasons, several telecom experts told CNN.
Common causes include construction-related digging that punctures fiber optic cables and software misconfigurations that can lead to interruptions, said TJ Kennedy, a public safety communications expert.
“I can’t think of every incident in the last few years, but I can think of things related to routers, things related to backhaul, things related to software,” Kennedy said. “This has happened across all major carriers, multiple times in the past few years alone.”
Thursday’s outage could have been caused by human errors in AT&T’s cloud-based networking system, said Lee McKnight, an associate professor at the Syracuse University School of Information Studies.
“The dirty secret of telecom networks these days is they are just a bunch of wires and towers connected to the cloud,” McKnight said. “Someone making a mistake, and others on their team — and their automated tools — not catching it, is quite common in cloud computing.”
Local governments report outages
Several local governments said AT&T’s outage was disrupting its services.
San Francisco’s Department of Emergency Management said in a statement on X Thursday morning that its 911 center remained operational, but many AT&T customers were unable to reach the emergency line because of the outage. It suggested people call from a landline or find someone with a rival’s service to dial 911.
“We are aware of an issue impacting AT&T wireless customers from making and receiving any phone calls (including to 911),” the department said in its post. “We are actively engaged and monitoring this.”
The Fire Department in Upper Arlington, Ohio, said the AT&T outage was affecting its fire alarms. St. Joseph County, Michigan, advised residents to use Wi-Fi to place 911 calls if they can’t reach 911 on AT&T’s network. Cobb County, Georgia, said its 911 operations remained unaffected by the outage but noted customers may want to find alternate methods of reaching emergency services. Cabel County, West Virginia, said customers that couldn’t reach 911 could text to 911 as a last resort.
New York Police Department officials told CNN that they were not able to make calls or utilize emails on AT&T phones Thursday morning unless they were connected to Wi-Fi.
The Massachusetts State Police warned people not to test their phone service by placing 911 calls.
“Many 911 centers in the state are getting flooded w/ calls from people trying to see if 911 works from their cell phone. Please do not do this,” the state police said in a post on X. “If you can successfully place a non-emergency call to another number via your cell service then your 911 service will also work.”
Atlanta Mayor Andre Dickens, meanwhile, said the city is “actively gathering information to determine how the City of Atlanta can assist in resolving this issue,” in a statement posted on X. Dickens said Atlanta’s “e-911 is able to receive inbound and make outbound calls” and encouraged AT&T customers to direct inquiries to restore service to the company.
An AT&T spokesman said the company’s FirstNet network has remained operational. FirstNet provides coverage for first responders and is advertised as a more robust network than the AT&T commercial network. It uses a mix of its own infrastructure plus AT&T’s broader network. Its customers include police and fire departments, as well as first responders during natural disasters.
Verizon and T-Mobile say they’re unaffected
There also have been about 1,000 outages reported by both Verizon and T-Mobile customers Thursday morning, the DownDetector website indicates.
“We did not experience an outage,” T-Mobile said in a statement. “Our network is operating normally.”
Verizon had a similar comment, saying it was unaffected by AT&T’s outage.
“Verizon’s network is operating normally,” Verizon told CNN in a statement. “Some customers experienced issues this morning when calling or texting with customers served by another carrier. We are continuing to monitor the situation.”
User reports on Downdetector about a T-Mobile outage, the company added, are “likely reflecting challenges our customers were having attempting to connect to users on other networks.”
Downdetector offers “real-time status information for over 12,000 services across 47 websites representing 47 countries,” the website says.
The FCC will probably investigate
The Federal Communications Commission will almost certainly investigate this week’s incident, multiple experts said. The FCC requires carriers to report information linked to network disruptions.
“The carriers are required to report their outage numbers over time, and the commission can track the number of consumers and cell sites down and things like that,” said a former FCC official.
Fines may be possible in connection with 911 outages, although they aren’t a certainty, said Blair Levin, a telecom policy analyst and another former FCC official.
“The FCC cares a lot more about the inability to connect with 911 [than other types of calls],” said Levin. “It’s a more serious problem from the FCC’s perspective.”
Telecom carriers have every reason to fix any outages quickly, said the first former FCC official, “because it creates black eyes for the brand.”
“Everybody’s incentives are aligned,” the former official said. “The FCC is going to want to know what caused it so that lessons can be learned. And if they find malfeasance or bad actions or, just poor quality of oversight of the network, they have the latitude to act.”