The SAP project that was three years in running and marred by delays and budget overruns was scheduled to go live on November 5. Failure to go live meant a delay of another five months, likely another $50 million in additional spending, and a trip back to the Utilities Rate Commission to request approval to pay for the overruns.
At the same time, Hurricane Sandy was pounding up the East Coast. By mid-October, forecasts for damage in NGUSA’s service area were extensive. For a utility company, power restoration takes precedence over everything else after a hurricane.
NGUSA had to know they had a bumpy ride coming when they made the decision to go live. What they clearly didn’t understand was just how bumpy the ride would be.
In the weeks that followed the go-live, while the NGUSA crews were working tirelessly to restore power, the SAP project team was just beginning to understand the full extent of the damage being caused because of the screwed-up SAP implementation. The problems spanned many areas, including:
Payroll: The new SAP system miscalculated time, pay rates, and reimbursements, so that employees were paid too little, too much, or nothing at all. Over-payments of more than $6 million were made to employees and not recovered. There was $12 million paid in settlements to employees related to short pays and deductions. Delays in the generation of W2s and other tax reporting occurred.
Vendor payments: Just two months after go-live, NGUSA had 15,000 vendor invoices that they were unable to process for payment, inventory record-keeping was in shambles, and vendors were being issued payments with the understanding that reconciliation would take place later.
Financial reporting: Prior to go-live it took NGUSA four days to close its financial books. Following go-live the close took 43 days. So bad was the financial reporting, NGUSA lost its ability to access short-term borrowing financial vehicles based upon its inability to provide satisfactory financial reports.
To deal with the many accounting, payroll, and supply-chain issues NGUSA was grappling with, they launched a stabilization program. To support the program, 300 contractors were initially brought in to assist with payroll issues. A total of 450 contractors were eventually brought in to address payroll problems. Another 200 contractors were brought in to assist with supply-chain issues. And, another 200 contractors were brought in to support the financial close issues.
The first priority of the stabilization effort was to ensure that NGUSA could comply with its obligations, including:
> Paying employees accurately and on time
> Paying vendors accurately and timely
> Providing legal, regulatory, and other reports to external stakeholders that are accurate and timely
The team’s second stabilization priority was to enable NGUSA to be efficient and self-sufficient in operating the SAP system and realize the benefits the system can provide without significant reliance on external support.
The continuing effort to stabilize SAP was anticipated to be about $30 million per month in September 2013.
The problems were so profound that the cleanup took more than two years to complete with a calculated cost of $585 million, more than 150 percent of the cost of implementation.
If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.
If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.
Timeline of Events
2007The journey to the decision to go live on November 5 was not unlike that experienced by other companies that have followed a similar path. In 2007, NGUSA had finalized a major acquisition, making it one of the largest privately held power distribution companies in the U.S.
This acquisition left the company with two sets of financial and operating systems. Capturing the synergies of combining these systems and adopting new sets of business processes were key components of the justification for the project. The project was also viewed as a method to allow NGUSA to address significant audit deficiencies in its financial business processes.
2008
The project to upgrade NGUSA's legacy systems – many of which were running on Oracle – began in 2008.
2009
In mid-2009, NGUSA hired Deloitte as its systems integrator and set a project budget of $290 million that was submitted and approved by the Utilities Rate Commission.
2010
Deloitte was initially employed as the lead implementation partner, project manager and systems integrator, but in June 2010 it was replaced by Ernst and Young (EY) in the first two roles, and by Wipro as systems integrator. The main reason for this switch was to lower implementation costs.
2012
The program operated with a target go-live date of December 2011. This date was later moved to July 2012, followed by October 2012, and then a November 2012 target date. The final sanctioned estimate of the project was set at $383 million, nearly 30 percent beyond the original target budget that was approved by the board.
NGUSA continued to engage Wipro after the go live in making the necessary fixes to the installed SAP system tolling their agreement (extending the statute of limitations for filing suit). In many instances, functional and technical specifications had to be completely rewritten and entire SAP modules had to be rebuilt or abandoned.
2017
On November 30, 2017, NGUSA filed a lawsuit against Wipro in the U.S. District Court Eastern District of New York. The lawsuit notes that NGUSA was unable to file suit against EY due to the language of their contract. The suit alleges that Wipro fraudulently induced NGUSA into signing the original agreements. NGUSA claims that Wipro misrepresented its SAP implementation capabilities, talent, and knowledge of the U.S. utilities business operations and common practices.
As Wipro knew or should have known, it had neither the ability nor intent to assign appropriately experienced and skilled consultants to the Project because... it in fact had virtually no experience implementing an SAP platform for a U.S.-regulated utility. – National Grid USA
Besides this, the suit alleges that Wipro breached its contract with NGUSA by:
> Failing to prepare design documents and specifications to industry standards.
> Failing to prepare programming and configuration to industry standards.
> Failing to adequately test, detect, and inform of problems.
> Failing to advise that the system was not ready to go live.
> Breaching express and implied warranties by not providing consultants that were consistent with a top 25 percent SAP implementation firm.
> Negligently misrepresenting itself for the same reasons identified in the first cause for action.
> Violating New York’s General Business Law for deceptive practices.
NGUSA was seeking damages in the form of relief of all contractual obligations, restitution of all amounts paid to Wipro, damages associated with a poor go-live, punitive damages, and attorney's fees and costs associated with the lawsuit.
On June 1, 2018, Wipro filed a motion to dismiss on three of the five causes for claims that it fraudulently misrepresented its capabilities and of negligent misrepresentation. In its response to the NGUSA RFP, Wipro claims that it identified that it had a well-established SAP practice, installed SAP globally for utilities, and had a long-running relationship with U.S. utilities. There was no explicit statement indicating that Wipro had not completed implementations of SAP for U.S.-based utilities nor were specific references provided in this regard.
Wipro also defended much of the language in the RFP response as common puffery, implying that NGUSA had a basic responsibility to check references.
2018
In August 2018 Wipro paid NGUSA $75 million to settle the lawsuit. Wipro states that the settlement has been effected for an amount of US$75 million and is without admission of liability or wrongdoing of any kind by the parties.
NGUSA has been a valued customer of Wipro for over a decade and both organizations have had a mutually beneficial relationship over the years. We believe that this settlement will be commercially beneficial for us and will help us remain focused on growth. – Wipro
What Went Wrong
In my experience with such large projects, it is never one party that is responsible for such a disaster by itself. Where were NGUSA’s project owners? Client project teams have responsibility for signing off on requirements, designs, project strategies, and test results. Did NGUSA provide Wipro with the appropriate access to expert personnel to properly identify requirements? Did Wipro believe that they had accurately captured all requirements based on NGUSA’s sign-off?In July of 2014, the NorthStar Consulting Group presented their findings of a comprehensive management and operations audit of the U.S. NGUSA companies sponsored by the New York Public Service Commission. The 265-page report (see references) covered a broad range of the company’s operations and governance. Throughout the report, the impact of the failed go-live was noted as well as the governance processes that led the company to determine that going live on November 5 was the best decision.
Below are some interesting observations noted in the report.
Design
The system only produced limited reports for management. Most managers have received only highly summarized reports of the costs they are responsible for since the go-live date. November 2013, eight months into the fiscal year, was the first time managers received a detailed cost report that also contained their corresponding budget figures. Some of the lack of reporting was a result of the system design, and many reports that had been provided by the predecessor systems were not provided in the design of the SAP.
Training
Another reason for the lack of information to managers is that the philosophy of information access at the SAP system is that managers are expected to request tailored information and reports from the system with the support of analysts from Decision Support. The lack of staff with the high level of skills necessary to query the data and produce reports for managers has greatly limited the success of this strategy.
See "Your (Lack Off) Training Efforts Can Easily Ruin the Outcome of an Otherwise Well-Executed Project" for more insights on this topic.
Testing
Testing was conducted during each phase of development of the SAP system. One of the lessons learned is that the testing was designed to determine where the system did work rather than identifying the areas where it did not work. Another lesson is that errors were found in the final test stages. Fixes were installed but there was no time for retesting.
See "Test Automation and Automated Tests" and "The Only Test Plan You Will Ever Need" for more insights on this topic.
Complexity
Building an SAP system requires the development of a series of components commonly referred to as RICEFWs (Reports, Interfaces, Conversions, Enhancements, Forms, Workflows). NGUSA’s design had a total of 636 RICEFWs. As Exhibit IV-5 illustrates, this was a large number for even a large power utility. The NGUSA system design was twice as complex as NGUSA UK’s R1 implementation of SAP and three times as complex as NGUSA UK’s R2 implementation.
See "Understanding and Managing Your Project’s Complexity" for more insights on this topic.
Preparation
Pre-implementation, NGUSA did not benefit from the rest of the industry’s SAP lessons learned. NGUSA did not use vendors with a strong track record of U.S. utility industry experience in SAP platform implementation and to date has had almost no interface with other U.S. utilities that have implemented SAP.
Transparency
While problems with system and company readiness were identified by particular groups within NGUSA prior to implementation, that information was subsumed by a push to go live. The overly optimistic risk scoring and executive expectations for the project in its early stages continues with stabilization work.
Validation
During the initial SAP development process, there was minimal interaction with operations personnel regarding desired information or reports. NGUSA implemented a complex field time reporting system without investigating its feasibility given how work is actually performed.
See "No Validation? No Project!" for more insights on this topic.
As part of the findings, NorthPoint documented NGUSA’s management observations as to the root cause of the failed implementation. These included:
> Overly ambitious design
> Significantly underestimated scale of transformation needed
> Limited availability of internal personnel due to ambitious business agenda
> Multi-partner model did not deliver business benefits
> Lack of ownership of certain business processes
> Testing less effective than expected due to limited range of scenarios tested and limited data availability
> Inadequate quality of data from legacy systems
> Too much focus on timeline and not enough focus on quality
> Training methods proved ineffective
Closing Thoughts
There are many checkpoints a project the magnitude of NGUSA’s SAP implementation must pass to move forward, each requiring NGUSA to sign off on the quality of the delivered product. There were many opportunities for NGUSA to identify poor-quality talent on the part of Wipro and demand replacements.The final decision to go live always rests with the client and unless Wipro was looking to deceive NGUSA regarding the results of its testing, NGUSA is partly to blame.
And where was Ernst and Young? EY was providing project management oversight. They clearly have an understanding of what it takes to put in a major SAP implementation. How did they not see or anticipate the major problems that occurred, and how did they fail to warn the NGUSA management team? Or did they?
Where was SAP? In their suit, NGUSA claims that Wipro developed an overly complex system that relied on the development of new capabilities as opposed to using the software as designed. NGUSA identified SAP as providing some level of oversight. Why didn’t SAP point out these significant deviations from standard? Or did they?
Where were the auditors? Projects of this size and impact are often reviewed by both internal and external auditing. Were project reviews performed? Were the appropriate mitigations put in place?
While there are many questions left to be answered about the botched SAP implementation that almost brought down National Grid, one thing is sure. When you start a large project like the implementation of a SAP system, you have to take responsibility and make sure that you as an organization are ready and committed to it.
If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.
If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.