In 2013, professional services company Accenture was awarded a ten-year, £46.11m contract to provide the i6 computer system to Police Scotland by the Scottish Police Authority (SPA). The i6 system was intended to replace 130 electronic and paper-based systems covering 80 per cent of police processes for recording crime and missing persons.
The program started with projected savings of £200m to the authority and Police Scotland. However, the program ended in July 2016 with wasted resources, wasted money, wasted time, and has set Scotland’s police forces back several years.
Accenture, the SPA, and Police Scotland mutually agreed to terminate the contract after a review found it would be impossible to deliver the system on time and on budget. As part of the contract termination, the SPA secured a £24.65m settlement agreement, in which Accenture refunded the £11.09m the police had already paid and made an additional payment of £13.56m.
A subsequent review of the program by Audit Scotland, published in March 2017, found that the project “ultimately collapsed due to a damaging loss of trust between those involved and fundamental disagreements about what the program needed to deliver”.
Although “good practice” was followed during the program’s procurement, it soon ran into problems, Audit Scotland said.
Commenting on the review, Lib Dem justice spokesperson Liam McArthur said: “This report shows a project that was doomed to fail from day one”.
“Dogged by miscommunication and false expectations, everyone involved stuck their fingers in their ears and carried on regardless”.
“They were blinkered by the need to avoid the i6 program becoming the latest in a string of debacles resulting from the SNP’s chaotic police centralisation. They deemed it too big to fail”.
“Years of work have all been for nothing. The frustration that any police officer must be feeling when reading this report is entirely justifiable”.
“They will be stuck using dysfunctional IT systems for many more years. The benefits centralisation promised them haven’t materialised”.
If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.
If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.
Timeline of Events
2010 — 2011
The origins of i6 and its procurement took place before the Scottish Police Authority (SPA) and Police Scotland were established. In 2010, the then Association of Chief Police Officers in Scotland (ACPOS) approved a procurement process to select a supplier. The procurement exercise was led by the then Scottish Police Services Authority (SPSA), who issued a notice in the Official Journal of the European Union on 2 June 2011. An 18-month competitive dialogue process followed.
A dedicated program team drawn from the then eight regional police forces and SPSA developed the business case for i6 using HM Treasury guidance. The program had two broad objectives:
> Develop common, national policing processes aligned to operational priorities.
> Acquire a national ICT system to support those processes and priorities.
The business case for i6 suggested it would cover 80 per cent of core police services by covering six of them: Crime, Criminal Justice, Custody, Missing Persons, Vulnerable Persons, Property. Hence the name i6.
Outcomes in the i6 business case included releasing officers from back-office functions to frontline operational duties, improving information sharing, and reducing operational risk. The business case anticipated full payback would be achieved in 2021/22 [1]. It was estimated that implementing this new IT system would generate potential efficiency savings of around £200 million over ten years.
In terms of planning and procurement, the i6 program team followed the good practice recommended in the Audit Scotland report “Managing ICT contracts: an audit of three public sector programs” [3].
For example, to ensure it could fulfil its role as an intelligent client, the i6 program team addressed expertise gaps by appointing:
> Deloitte as the external experts on procurement and managing commercial contracts;
> Eversheds as legal advisers;
> Exception UK as technical advisors.
2012
Ninety suppliers expressed an interest at the initial procurement stage. Around 20 bidders entered the pre-qualification stage, after which 11 submitted outline proposals.
Given the new system’s complex requirements, the i6 program team held around 160 dialogue sessions with interested bidders. These sessions discussed the system’s required functionality and technical aspects and helped the team set out clear requirements for bidders.
Three bidders entered the final stages of detailed competitive dialogue in October 2012. These bidders were required to respond to the detailed requirements as part of the Invitation to Submit Final Tender. They also had to demonstrate in detail the functional features of their systems against a series of business scenarios. This is a key part of the tendering stage, allowing the contracting organisation to see how the supplier will meet its requirements and identify potential gaps.
In November 2012, after evaluating all bids, the i6 program team selected Accenture as the preferred bidder. Accenture scored highest against the technical and implementation criteria and second in functionality and cost.
The i6 program team and Accenture believed that most of the i6 system could be based on an existing IT system Accenture had developed for Spain’s Guardia Civil police service. The remainder was bespoke development work.
This IT system’s existence, and its experience working with police services across Europe, contributed to Accenture being awarded the contract.
2013
The final procurement stages took place in the months before the SPA and Police Scotland’s establishment. Police Scotland’s i6 program team would oversee the operational management of the i6 program while ultimately being accountable to the SPA.
The newly established SPA approved the final business case in June 2013 and awarded a fixed-price contract of £46.11 million to Accenture. The contract included:
> Software and specialist hardware;
> Integration tools and services;
> Business change activities;
> Implementation services;
> Reporting capabilities;
> Data management activities;
> On-going support.
This was based on a standard public sector contract that protected the SPA on the basis that the supplier is obliged to meet all the requirements in the contract. It also included clauses to ensure Accenture accurately cost and scoped the IT system. The contract was robust, and this would later prove important for the SPA when agreeing with Accenture to terminate the contract.
I6 was planned to go live in September 2014. It would be rolled out regionally and be fully complete by August 2015.
The i6 program had difficulties almost immediately after the contract’s award. Within weeks of starting the high-level design phase in July 2013, there was a difference in opinion about the search function within i6. The i6 program team believed that the functionality of Accenture’s solution did not meet the requirements it had agreed in the contract.
Accenture maintained that Police Scotland had not specified a detailed description of business requirements. This issue had not emerged during months of pre-award dialogue. Accenture also believed that it had set out clearly what its solution would do and maintained that Police Scotland had accepted its qualified solution as part of the procurement process.
A dispute followed about the contract requirements’ interpretation. Police Scotland argued that, after months of competitive dialogue, the i6 system’s requirements were well-defined and that in line with the contract, these took precedence.
Accenture argued its solution had precedence and that Police Scotland was trying to extend the program’s scope. Accenture stated that meeting Police Scotland’s interpretation of requirements would require more time and money.
The very early disagreement led to a loss of trust between the two organisations but, in keeping with contractual obligations, the high-level design phase continued. Senior management on both sides were keen to try and maintain a practical working relationship between their teams while they entered into formal negotiation.
The two organisations agreed that continuing the high-level design phase would allow them to quantify the gap between Accenture’s solution and Police Scotland’s requirements. By this phase’s end, more gaps had been identified. Accenture estimated these would cost an additional £1 million to fill, which they agreed to fund.
Police Scotland’s i6 program team began raising concerns at the i6 program board about various design elements. Board members also expressed frustration over the lack of detail from Accenture about the system design structure and the timeliness and quality of documentation provided by Accenture.
In September 2013, the program was reported as having a red status. At this point, only the first milestone (award of the contract) had been achieved in full.
By November 2013, Accenture had not completed the high-level design phase but had started the detailed design phase.
In December 2013, the i6 program board discussed concerns about the risks associated with overlapping these phases.
Police Scotland’s i6 program team and the i6 program board repeatedly expressed their frustration to Accenture about the disagreement, particularly when they had followed good procurement practices and spent considerable time discussing system requirements. As the contractual dispute continued, relationships between the organisations were strained, and trust was limited.
Around this time, in addition to the open sessions, the i6 program board began operating closed sessions. During these closed sessions, members discussed the ongoing issues with the contract. Accenture was not invited to attend these sessions, and it considered this structure unhelpful for the relationship.
2014
The Scottish Parliament’s Justice Sub-Committee on Policing held several evidence sessions with the SPA and Police Scotland to explore progress with the i6 program.
In March 2014, the Sub-Committee expressed frustration at the lack of information about the problems with the i6 program that had been ongoing since August 2013. Police Scotland did not disclose the severity of the issues facing the program, nor was it overly critical of Accenture. This might have reflected a desire to maintain relationships with Accenture to keep the program on track or maintain the contract’s commercial confidentiality.
As the design of i6 progressed, it became apparent that Police Scotland and Accenture’s original assumptions looked doubtful. These were based on building on the Police Information Management System Accenture provided to Spain’s Guardia Civil and developing the remainder of the i6 system from scratch.
The original timescales and staff that Accenture had planned to allocate were based on these expectations, and it was not long before the program started to fall behind schedule. Several deliverables from milestones two and three had already been missed by this stage.
The program board queried Accenture’s delivery methodology’s suitability with fears it could lead to problems later on in the program. The board sought assurances from Accenture about the delivery plan and associated risks, staffing, and skills that it planned to deploy throughout the program.
The program board also asked Accenture to provide future program board meetings with a quantitative assessment of its confidence in delivering i6. By February 2014, the program was around seven months behind the original plan.
At the program board, the program team reported that to meet the requirements, Accenture was now developing far more from scratch than it had originally anticipated.
The program board raised concerns about errors in design documents and expressed worry over the level of policing knowledge on Accenture’s team.
On 25 March 2014, the i6 program board considered the outcomes of lengthy contract negotiations with Accenture. These were captured in a contract variation agreement, which amended specific elements of the original contract, including a revised delivery and milestone plan. Accenture agreed to amend its proposed IT system to address all the gaps that had been identified and to deliver the requirements within the fixed price. Police Scotland took on responsibility for certain plan elements, such as the transfer of data.
The SPA approved the contract variation agreement in April 2014. The i6 go-live date was revised to July 2015, with full completion by September 2016. The new contract variation agreement reset the relationship between organisations and, temporarily, improved levels of trust. The program’s momentum picked up, and the program board approved payment for milestones two and three.
For the first time since the i6 program started, its status was reported as green at the i6 program board meeting. Accenture’s assessment of its confidence in delivering i6 rose to 85 per cent in May 2014 when the detailed design phase (milestone four) was completed.
Over the next few months, the i6 system’s development continued, as did renewed disagreements about the program’s scope. Accenture maintained that Police Scotland was extending the program’s scope, which required a greater degree of bespoke development. Police Scotland maintained that there was no extension to the scope beyond changes agreed through the change control process. The program board again challenged Accenture over its timeliness of reporting problems and delays in delivery, as well as its documentation’s quality. It also raised concerns over the Accenture development team’s expertise and the high turnover of key personnel.
By August 2014, milestone five (functional design) was behind schedule. Payment for this milestone, of £2.6 million, was therefore also delayed. The program board was told that the delay in payment had been raised at the highest levels within Accenture.
Accenture cited a more complex design than originally anticipated as the reason for the delay. The relationship deteriorated again as delivery dates slipped. The go-live date was delayed again from July 2015 to September 2015, with full roll-out remaining September 2016, but achieving this would require overlapping development phases.
To keep the program on track, Police Scotland adjusted the payment schedule. Most milestones were delayed throughout the i6 program, and the program board withheld payment until Accenture had delivered. However, in the latter part of 2014, in an attempt to alleviate an extremely strained relationship, the i6 program board agreed to split a milestone (completion of the functional design stage).
This meant releasing payments when each component deliverables were achieved rather than waiting until all were achieved, as specified in the contract. While this could be considered pragmatic, given the multiple challenges and problems with the i6’s development, this is not good practice.
2015
The product testing phase began in January 2015, four months behind the original schedule. Accenture found various technical problems but assured the i6 program board it would resolve them. In February 2015, it assessed its overall confidence in delivering the program at 90 per cent.
After product testing, Accenture released the system to Police Scotland for user acceptance testing. In June 2015, the program board raised concerns about unresolved defects from Accenture’s product test phase.
Police Scotland and Accenture disagreed about how critical these were. The program board challenged Accenture’s testing strategy, which it did not consider was in line with industry standards. The relationship between the organisations was extremely fragile, with a lack of trust and frustration on both sides.
The “Waterfall” approach contributed to the fact that Police Scotland only discovered the true extent of problems with the system when it was delivered for testing. Although Accenture had provided Police Scotland with demonstrations of the developing i6 system, it was after a period of testing that the i6 program team reported to the program board in August 2015 that there were:
> critical errors in the technical coding;
> higher-than-projected levels of flaws that Accenture was not able to resolve as quickly as expected;
> serious concerns raised about the criminal justice module, which did not comply with the Integrated Scottish Criminal Justice Information System data standards;
> errors in the search and audit modules;
> problems around the limited functionality in the administration module (Accenture had already received payment for successfully delivering this element).
At the August 2015 program board meeting, Accenture agreed to analyse root causes and report back to the program board on its findings and actions to resolve these. At this meeting, Accenture assessed its confidence of meeting the go-live date of December 2015 at 91 percent. The i6 program board challenged this assessment.
In September 2015, there was an extraordinary i6 program board meeting. Accenture reported that the issues raised in user acceptance testing would need more analysis and remedial action. It said the December 2015 go-live date would not be achieved. It agreed to a joint re-planning exercise to be reported to the October 2015 i6 program board.
Accenture did not attend the i6 program board in October 2015 as its analysis exercise was not complete. In November 2015, Accenture requested a period without prejudice, meaning all organisations could speak freely and openly, without the risk of what they say being used against them later if negotiations fail.
In December 2015, Accenture reported that the work still required on i6 would take an additional 30 months. This proposal would mean go-live would be delayed until April 2018, and the cost would be many millions more than the original contract price. Police Scotland rejected this proposal. The SPA, Police Scotland, and Accenture entered detailed discussions to explore the i6 program’s future options.
2016
A series of meetings between Accenture senior management, the SPA, and Police Scotland took place during early 2016 to consider options for the way forward.
After reviewing the final options appraisal report in May 2016, the SPA decided that the revised plan was not viable. The SPA and Accenture mutually agreed to terminate the contract, and both organisations entered commercial negotiations, which concluded in July 2016 when they signed the settlement agreement.
The contract enabled the SPA to secure a settlement agreement of £24.65 million. This meant that Accenture agreed to refund the £11.09 million that the SPA had paid and to make an additional payment of £13.56 million. This reflects estimated staff costs and capital costs such as hardware maintenance and software licenses associated with i6.
2017
A review of the program by Audit Scotland, published in March 2017, found that the project “ultimately collapsed due to a damaging loss of trust between those involved and fundamental disagreements about what the program needed to deliver”.
Commenting on the report, an Accenture spokesperson said: “As the report acknowledges, the scope and the complexity of the solution for i6 increased significantly during the project. This was driven by the client. There were challenges and issues on both sides, but we worked closely with Police Scotland to review the program and recommend revised plans to successfully deliver i6. Despite our best efforts, it was not possible to agree to the necessary changes, and we mutually agreed to end the project”.
What Went Wrong
Besides closing their expertise gaps, Police Scotland also followed good practice by:
> Establishing a program board with overall responsibility for scrutinising and challenging the program’s progress and risk management arrangements.
> Appointing a senior responsible owner charged with ensuring the program met its objectives and delivered the planned benefits.
> Appointing a program manager with responsibility for the program’s day-to-day management.
The Scottish Government also provided some external assurance at various stages of the i6 program in the form of:
> Gateway reviews — these are short, focused reviews by the Scottish Government to provide an assurance check on a project’s status. It makes recommendations to help with decision-making on program management.
> Healthchecks — these are similar to gateway reviews but are more flexible in remit and scope.
> ICT technical assurance reviews — these are to ensure technical solutions meet user business needs. They review in more detail than the other forms of independent assurance.
So what went wrong?
Delivery Method
The i6 program used the classic “Waterfall” method. In this approach, the software is developed in distinct phases, each leading to the next phase in a sequence resembling a waterfall.
Once a phase is complete, the process moves on to the next phase, and there is no turning back. It meant that all of the design, coding, and construction of i6 would be completed before Accenture released it to Police Scotland for testing. Police Scotland would pay for each phase when it was completed.
The “Waterfall” method was common at i6’s origins, though “Agile” was gaining popularity. A more flexible, incremental approach where the team works on small-scale launches of a functioning product would suit the problem at hand far better. The development team tests each software launch against the user’s requirements throughout the project, and, in theory, changes can be made more easily. Recently, more organisations are adopting the “Agile” approach to software development.
The method adopted for developing the i6 system meant that the full scale of difficulties facing i6 ultimately became clear in August 2015 when the system was passed to Police Scotland for testing. There were fundamental flaws and serious errors. At this point, Accenture estimated that meeting the contract requirements would take an additional two and a half years, with go-live being delayed until April 2018, almost four years later than originally planned.
Politics
The i6 program took place in the context of a high level of public and political scrutiny. The SPA, Scottish Government, and Police Scotland did not always work together effectively before and after the merger process that established the SPA and Police Scotland.
During the i6 program, there were well publicised disagreements over responsibilities between the SPA and Police Scotland, including responsibility for ICT. In addition, the SPA and Police Scotland were facing high-profile issues such as the deployment of armed officers, stop and search policy, and emergency call handling.
The SPA and Police Scotland wanted to deliver i6. The failure of a previous police ICT project in 2012 (the Common Performance Management Platform) meant there was pressure on the SPA and Police Scotland to make i6 a success.
Furthermore, the i6 program was vital to Accenture at a global level. This might have led to misplaced optimism about the prospects of success and unwillingness to consider terminating the program.
Police Scotland considered legal action against Accenture for breach of contract as early as October 2013. Both organisations agreed to make an effort to resolve the disagreements over the contract and avoid an expensive legal challenge in court, which could have had potential political and commercial consequences.
Underestimating Complexity
The i6 program was complex and highly ambitious. Police Scotland and Accenture originally believed that most of the i6 system could be based on an existing IT system that Accenture had delivered elsewhere. This belief was incorrect. As the design and development of i6 progressed, it became apparent that Accenture would need to develop significantly more than had been originally anticipated.
Police Scotland concluded that Accenture had underestimated the system’s complexity and had, at the contract stage, overstated its own ability to deliver i6 within the timescales and fixed price agreed. The level of effort Accenture would require to complete i6 was around eight times greater than the resources Accenture had estimated when signing the original contract.
The belief that most of the system could be based on the system that Accenture had provided to the Guardia Civil was incorrect. It had become clear a virtually fully bespoke system was required.
How Scottish Police Could Have Done Things Differently
Since the i6 program’s closure, the following reviews have been completed, with learning identified and recorded for use by those engaged in future projects.
> Technical Audit of Accenture Development (by Scottish Government);
> Internal Review of i6 Procurement and Design (by Organisational Development);
> Internal Review of Training and Deployment Planning materials (by i6 program).
Each review’s objective was to identify activities and approaches which will be key to assuring future outcomes. Importantly, the reviews were approached positively, capturing areas of good practise that should be repeated and other areas that could be enhanced.
Key findings in these reviews included:
Iterative Development
> Challenges with implementing the “Waterfall” approach and a need for future consideration to be given to the alternate “Agile” iterative approach.
> Benefits of the supplier adopting a standardised “development approach”.
> The importance of realistic planning and resource allocation against distinct workstreams to ensure delivery against competing demands.
See "14 Essential Software Engineering Practices for Your Agile Project" for more insights on this topic.
Quality Control
> Increased supplier adherence to quality management practices, with an emphasis on quality control frameworks.
See "Be a Responsible Buyer of Technology" for more insights on this topic.
The Right People
> Supplier focusing on the experience and skill levels of those involved to ensure that the requisite skills are in place to assure delivery.
See "Outsourcing Technical Competence Is a Very Bad Idea" for more insights on this topic.
(Independent) Reviews
> Provision for independent technical assurance reviews at more regular intervals.
> Ensuring compliance with agreed technical entry and exit criteria when moving between the phases of the project.
See "Your Best Insurance for Multimillion-Dollar Tech Projects - Independent Project Reviews" for more insights on this topic.
Facing Reality
Despite the delays and serious problems throughout the program’s lifetime, Accenture provided regular assurance in the face of strong challenges about their confidence in delivering the i6 system. This assurance proved misplaced.
See "It Is Time to Face Your Project’s Reality" for more insights on this topic.
Effective Business Engagement
> The importance of effective business engagement and the need for appropriate investment in this area.
See "Successful Projects Need Executive Champions" for more insights on this topic.
Closing Thoughts
i6 was a complex and ambitious program. There is no single reason why it failed. Despite an 18-month competitive dialogue process, there was a fundamental disagreement between Police Scotland and Accenture about interpreting the contract and the program’s scope. This damaged relationships and trust between the two organisations from a very early stage.
Both Police Scotland and Accenture were determined to deliver the i6 program. This might have led to optimism bias and a reluctance to pause or halt the project at an earlier stage. The “Waterfall” approach meant that Police Scotland would not test the system developed by Accenture until relatively late in the development process. There was also over-reliance on the existing system that Accenture had provided to Spain’s Guardia Civil.
The i6 program was a key component of police reform. Its failure means that some of the benefits that should have arisen from implementing it have been, at best, delayed. There was a need to modernise police ICT systems six years ago when the i6 procurement began. That need has not been met. Police officers and staff continue to struggle with out-of-date, inefficient, and poorly integrated systems.
If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.
If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.
If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.
Sources
[1] i6: A Review, Audit Scotland, March 2017.
[2] Written Response Audit Scotland i6 Review, Police Scotland, May 2017.
[3] Managing ICT contracts: an audit of three public sector programs, Audit Scotland, August 2012.
