LIPA report details PSEG LI's failures in Isaias storm response

For weeks before Tropical Storm Isaias landed on Aug. 4, PSEG Long Island officials knew of significant computer-system problems that ultimately crippled their response to the storm, according to a scathing new LIPA report obtained by Newsday.

But a combination of "mismanagement" and the lack of the autonomy from their New Jersey parent stifled their ability to fix them — even to this day, according to the report, which finds the problems so urgent that it recommends LIPA consider terminating PSEG’s contract if it can’t be reformed.

The report, by a LIPA task force investigating PSEG’s response to the storm that caused more than 646,000 outages, puts the blame for the widely criticized response on "mismanagement" by PSEG, charging the company "did not adequately prepare" for the storm, didn’t properly stress-test computer and telecom systems, and didn’t have manual backup plans in place.

The report also provides new indications that the region is not free of the problems that left more than 500,000 customers without power and in the dark about restoration times over a sweltering summer week.

According to the report, "many" of the root causes of PSEG’s failed response remain uncorrected. Recent tests of PSEG’s plan to fix the high-volume telephone system that failed during the storm show it "continue[s] to fail under only moderate testing loads." And the outage management computer system also "continues to fail when subjected to appropriate stress testing," LIPA’s report says, citing "deficiencies" in PSEG management of projects and outside vendors.

In an interview, LIPA chief Tom Falcone blamed PSEG’s "absentee management in New Jersey" and a "lack of accountability to Long Island" as core problems.

"They knew they had a problem" with the computer system on Long Island and were "totally casual about it," he said. "Worst of all, they didn’t tell LIPA" of the computer problems "when it was failing in June and July."

LIPA has presented PSEG with a list of more than 100 recommendations to fix the problems, some of which must be acted upon quickly. Computer and telecom systems "need to be fixed with urgency," Falcone said, adding, "If we need to, we’ll go to court to compel that."

PSEG spokeswoman Ashley Chauvin said the company was reviewing the LIPA report.

"PSEG Long Island recognizes that it did not perform well in many areas during and after Tropical Storm Isaias," she said. " We remain fully committed to understanding what went wrong, and continue to improve our organization and our systems to avoid similar failures in future storms."

LIPA has two options to fix the problems: Terminate the PSEG contract, which expires in 2025, or renegotiate it.

Under a new contract, LIPA would seek broader approval rights for PSEG Long Island senior management and disclosure of PSEG Long Island’s senior officials’ pay, according to recommendations in the report. It would eliminate the "matrix" organizational structure between the Long Island and New Jersey operations and change the score card that incentivizes PSEG for meeting a limited array of performance targets, the report said.

LIPA also wants PSEG to form a crisis-management team, and to hire a "turnaround" chief information officer and a vice president of emergency management. It also wants the Long Island computer system separated from the New Jersey parents' system.

Among the emails scoured by investigators on the task force was one from the PSEG Long Island operations supervisor, who found the outage management system, after a recent upgrade, was "NOT even managing on a day to day basis, and we are definitely NOT prepared for [a] weather event."

The email went up the chain of command, ultimately landing with PSEG Long Island chief operating officer Dan Eichhorn. Eichhorn was unable because of the management protocols of PSEG's New Jersey parent to undo the recent upgrade of the system, the report said.

"Unfortunately, PSEG Long Island’s most senior officer could not simply act on his own judgment and order the rollback, despite this high-risk situation with a [computer] system that was ‘not prepared for [a] weather event,’" Falcone wrote in an introduction to the report.

PSEG Long Island operates the LIPA-owned Long Island service territory under a long-term contract that has already paid the New Jersey company $467 million over seven years.

The state Department of Public Service is also investigating PSEG’s storm response, and in a letter Monday to the LIPA board it identified more than 70 potential violations of PSEG’s emergency response plan related to the storm.

Chauvin, of PSEG, said the company was "committed to continuing to work with LIPA and with the New York State Department of Public Service to provide information and improve our future performance for the people of Long Island."

The LIPA task force report details a management system at PSEG that leaves many key Long Island functions, including information technology, reporting back to New Jersey, where PSEG is based. That system can offer cost reductions and give Long Island access to higher levels of expertise.

But in the case of Isaias, it left a broken computer system largely out of order weeks before the storm threatened the region. By the time Isaias was preparing to wallop Long Island, there was a bottleneck of more than two hours of customer calls and texts reporting outages, LIPA’s report found. That bottleneck would only compound as the afternoon wore on, with hundreds of thousands of customer outage calls and texts, leading the entire system to fail by the afternoon of Aug 4.

Falcone in his remarks noted that LIPA "relied too much" on PSEG’s claims that it was "meeting contractual obligations," including for stress-testing the computer system, rather than verify the claims itself. "It is now clear that whatever stress testing PSEG Long Island performed, it did not test end-to-end functionality" under "realistic severe storm conditions." LIPA said it will now independently verify and validate those stress tests.

LIPA also found that PSEG was "not transparent" about computer system problems, including those detailed in the email exchange, which LIPA didn’t learn about until October, Falcone said. LIPA also missed signals of the "declining quality" of system problems, he acknowledged, blaming the management structure between PSEG’s Long Island and New Jersey operations.

State Sen. Todd Kaminsky (D-Long Beach) said he was shocked to find that most of the computer and telecom issues remain and said he was worried PSEG isn't prepared to respond. "The report makes clear that we’re not better off than we were the day after Isaias, which is quite concerning," Kaminsky said.

Falcone said the computer system "hits a certain wall of performance" when outage calls accumulate past 100,000 to 200,000 "and then it just stops."

LIPA trustee Matthew Cordaro said he was pleased to see LIPA’s task force "take a hard stance" in dealing with PSEG’s deficiencies. But he said LIPA should have cracked down sooner. "I’ve seen this coming the last several years," he said. "PSEG would make very bland reports to the board that really didn’t mean much and skirted all the issues that relate to their storm response."

Falcone said LIPA has options if it ultimately resorts to terminating PSEG's contract, including finding a new utility contractor, finding multiple different contractors to handle specific utility functions that are now centralized by PSEG, or even turning the Long Island utility into a fully public municipal utility.

The report found that more than a million calls and texts were lost or left unanswered in the aftermath of the storm.

It also found that PSEG didn’t come clean about prestorm problems with the outage management system, even after it crashed during Isaias "and LIPA launched its investigation."

PSEG was "not transparent about what it knew until questioned about the reports the system was failing before the storm," the report says, calling PSEG’s own analysis of its Isaias response "at best incomplete" while "resolutely ignoring management deficiencies while attempting to shift the blame to vendors."

"The root cause here was bad management," Falcone said. "We need a cure for bad management."

By Mark Harrington

mark.harrington@newsday.comMHarringtonNews

Mark Harrington, a Newsday reporter since 1999, covers energy, wineries, Indian affairs and fisheries.