A combination of “human and procedural errors” following a massive windstorm caused Warren County’s 911 system to go down for 15 hours in late June, representatives from the county’s 911 phone company told Warren County commissioners Tuesday.
Warren County Commissioner Dave Young has another term for it: “a comedy of errors.”
A late afternoon “derecho” caused the county’s 911 system to go down on June 29, according to Warren County Telecommunications Director Paul Kindell. A derecho is an intense and long-lived series of straight-line winds often associated with fast-moving thunderstorms or rain.
Residents were unable to reach 911 operators for more than an hour. For an hour after that, callers found themselves talking to dispatchers in distant Geauga County, near Cleveland.
Geauga County dispatchers gave callers Warren County’s seven-digit number or they relayed the caller’s information to Warren County.
Calls were later rerouted back to Warren County, but 911 service remained in “disaster mode” until the next morning. Dispatchers could only receive calls on regular business lines, which lack many of the identifying features of 911 calls, such as callers’ names, phone numbers or locations, said Kindell.
While officials have no way of knowing how many calls were not answered, Kindell said he was not aware of any cases in which lives or property were lost.
“Unfortunately, we had some human and procedural errors that occurred on June 29 as a result of that storm. We take full accountability for those errors and apologize for the disruption,” said Mike Conaghan, director of network operations for CenturyLink, the La.-based company that provides 911 service to Warren County.
The county’s 911 system is operated through two T1 circuit-houses located in Mansfield and Lima. Both of those lines went down during the June 29 storm that saw winds of more than 88 mph and severe thunderstorms across Ohio.
When those lines failed, it automatically generated critical alarms and three help tickets at CenturyLink. A single employee received all three tickets within minutes, but failed to act on them, said Andrea Gassman, process engineer for CenturyLink.
“That individual should have picked up the phone and contacted our [Warren County] point of contact and said, ‘You’ve got a problem,’” said Gassman. “She did not follow that process. We have coached and taken disciplinary action against that employee.”
Kindell said that when the 911 system went down, he called CenturyLink but spoke to a different operator who was unfamiliar with the system and the rerouting process.
“That individual essentially abandoned the call and told him to call again and press a different option,” said Gassman. “She made a mistake; she screwed up. She was the second person to let (Kindell) down.”
Kindell eventually reached an operator who rerouted calls. However, because a CenturyLink programmer had transposed digits in the rerouting number months earlier, those calls were mistaken routed to Geauga County
That error might have been discovered sooner had the county and CenturyLink tested the rerouting system as the company recommends, said Nancy Serafino, a public safety engineer at CenturyLink.
Kindell said testing is disruptive to the system and can result in missed calls, but says his office has since developed procedures for future testing every quarter.
“We’re changing our policy because that’s not acceptable,” said Young. “That’s on us.”
CenturyLink technicians reestablished full 911 service by 8:30 a.m. June 30, but failed to notify Kindell that service had been restored.
“We didn’t call him,” said Gassman. “That was a disappointment to me.”
As a result of this incident, Gassman said the company has enacted new procedures to improve communications and ensure service outages are addressed in a more timely manner.
Young however, says he’s not convinced and issued stern words to CenturyLink for their handling of the incident.
“I don’t have a warm and fuzzy feeling… that if folks are going to dial 911 that it will be there. It’s almost a comedy of errors. Almost every interaction, your procedures failed and almost every employee has made the wrong choice,” he said.
“This is not the service we expect of any vendor,” he added. “What is most disheartening to me is that we’re kind of stuck. That’s not exactly making me have a lot of confidence that the next major event that comes down the pike that the system is going to work.”