Case Study 15: How the Scottish Police Got £25 Million Back but Lost 3 Years on I6

In 2013, professional services company Accenture was awarded a ten-year, £46.11m contract to provide the i6 computer system to Police Scotland by the Scottish Police Authority (SPA). The i6 system was intended to replace 130 electronic and paper-based systems covering 80 per cent of police processes for recording crime and missing persons.

The program started with projected savings of £200m to the authority and Police Scotland. However, the program ended in July 2016 with wasted resources, wasted money, wasted time, and has set Scotland’s police forces back several years.

Accenture, the SPA, and Police Scotland mutually agreed to terminate the contract after a review found it would be impossible to deliver the system on time and on budget. As part of the contract termination, the SPA secured a £24.65m settlement agreement, in which Accenture refunded the £11.09m the police had already paid and made an additional payment of £13.56m.

A subsequent review of the program by Audit Scotland, published in March 2017, found that the project “ultimately collapsed due to a damaging loss of trust between those involved and fundamental disagreements about what the program needed to deliver”.  

Although “good practice” was followed during the program’s procurement, it soon ran into problems, Audit Scotland said.

Commenting on the review, Lib Dem justice spokesperson Liam McArthur said: “This report shows a project that was doomed to fail from day one”.

“Dogged by miscommunication and false expectations, everyone involved stuck their fingers in their ears and carried on regardless”.

“They were blinkered by the need to avoid the i6 program becoming the latest in a string of debacles resulting from the SNP’s chaotic police centralisation. They deemed it too big to fail”.

“Years of work have all been for nothing. The frustration that any police officer must be feeling when reading this report is entirely justifiable”.

“They will be stuck using dysfunctional IT systems for many more years. The benefits centralisation promised them haven’t materialised”.

Before we continue with this case study...

> For an overview of all case studies I have written please click here.

Timeline of Events

2010 — 2011

The origins of i6 and its procurement took place before the Scottish Police Authority (SPA) and Police Scotland were established. In 2010, the then Association of Chief Police Officers in Scotland (ACPOS) approved a procurement process to select a supplier. The procurement exercise was led by the then Scottish Police Services Authority (SPSA), who issued a notice in the Official Journal of the European Union on 2 June 2011. An 18-month competitive dialogue process followed.

A dedicated program team drawn from the then eight regional police forces and SPSA developed the business case for i6 using HM Treasury guidance. The program had two broad objectives:

> Develop common, national policing processes aligned to operational priorities.

> Acquire a national ICT system to support those processes and priorities.

The business case for i6 suggested it would cover 80 per cent of core police services by covering six of them: Crime, Criminal Justice, Custody, Missing Persons, Vulnerable Persons, Property. Hence the name i6. 

Outcomes in the i6 business case included releasing officers from back-office functions to frontline operational duties, improving information sharing, and reducing operational risk. The business case anticipated full payback would be achieved in 2021/22 [1]. It was estimated that implementing this new IT system would generate potential efficiency savings of around £200 million over ten years.

In terms of planning and procurement, the i6 program team followed the good practice recommended in the Audit Scotland report “Managing ICT contracts: an audit of three public sector programs” [3].

For example, to ensure it could fulfil its role as an intelligent client, the i6 program team addressed expertise gaps by appointing:

> Deloitte as the external experts on procurement and managing commercial contracts;

> Eversheds as legal advisers;

> Exception UK as technical advisors.

2012

Ninety suppliers expressed an interest at the initial procurement stage. Around 20 bidders entered the pre-qualification stage, after which 11 submitted outline proposals. 

Given the new system’s complex requirements, the i6 program team held around 160 dialogue sessions with interested bidders. These sessions discussed the system’s required functionality and technical aspects and helped the team set out clear requirements for bidders.

Three bidders entered the final stages of detailed competitive dialogue in October 2012. These bidders were required to respond to the detailed requirements as part of the Invitation to Submit Final Tender. They also had to demonstrate in detail the functional features of their systems against a series of business scenarios. This is a key part of the tendering stage, allowing the contracting organisation to see how the supplier will meet its requirements and identify potential gaps.

In November 2012, after evaluating all bids, the i6 program team selected Accenture as the preferred bidder. Accenture scored highest against the technical and implementation criteria and second in functionality and cost. 

The i6 program team and Accenture believed that most of the i6 system could be based on an existing IT system Accenture had developed for Spain’s Guardia Civil police service. The remainder was bespoke development work. 

This IT system’s existence, and its experience working with police services across Europe, contributed to Accenture being awarded the contract.

2013

The final procurement stages took place in the months before the SPA and Police Scotland’s establishment. Police Scotland’s i6 program team would oversee the operational management of the i6 program while ultimately being accountable to the SPA.

The newly established SPA approved the final business case in June 2013 and awarded a fixed-price contract of £46.11 million to Accenture. The contract included:

> Software and specialist hardware;

> Integration tools and services;

> Business change activities;

> Implementation services;

> Reporting capabilities;

> Data management activities;

> On-going support.

This was based on a standard public sector contract that protected the SPA on the basis that the supplier is obliged to meet all the requirements in the contract. It also included clauses to ensure Accenture accurately cost and scoped the IT system. The contract was robust, and this would later prove important for the SPA when agreeing with Accenture to terminate the contract. 

I6 was planned to go live in September 2014. It would be rolled out regionally and be fully complete by August 2015.

The i6 program had difficulties almost immediately after the contract’s award. Within weeks of starting the high-level design phase in July 2013, there was a difference in opinion about the search function within i6. The i6 program team believed that the functionality of Accenture’s solution did not meet the requirements it had agreed in the contract. 

Accenture maintained that Police Scotland had not specified a detailed description of business requirements. This issue had not emerged during months of pre-award dialogue. Accenture also believed that it had set out clearly what its solution would do and maintained that Police Scotland had accepted its qualified solution as part of the procurement process.

A dispute followed about the contract requirements’ interpretation. Police Scotland argued that, after months of competitive dialogue, the i6 system’s requirements were well-defined and that in line with the contract, these took precedence. 

Accenture argued its solution had precedence and that Police Scotland was trying to extend the program’s scope. Accenture stated that meeting Police Scotland’s interpretation of requirements would require more time and money.

The very early disagreement led to a loss of trust between the two organisations but, in keeping with contractual obligations, the high-level design phase continued. Senior management on both sides were keen to try and maintain a practical working relationship between their teams while they entered into formal negotiation. 

The two organisations agreed that continuing the high-level design phase would allow them to quantify the gap between Accenture’s solution and Police Scotland’s requirements. By this phase’s end, more gaps had been identified. Accenture estimated these would cost an additional £1 million to fill, which they agreed to fund.

Police Scotland’s i6 program team began raising concerns at the i6 program board about various design elements. Board members also expressed frustration over the lack of detail from Accenture about the system design structure and the timeliness and quality of documentation provided by Accenture. 

In September 2013, the program was reported as having a red status. At this point, only the first milestone (award of the contract) had been achieved in full.

By November 2013, Accenture had not completed the high-level design phase but had started the detailed design phase. 

In December 2013, the i6 program board discussed concerns about the risks associated with overlapping these phases.

Police Scotland’s i6 program team and the i6 program board repeatedly expressed their frustration to Accenture about the disagreement, particularly when they had followed good procurement practices and spent considerable time discussing system requirements. As the contractual dispute continued, relationships between the organisations were strained, and trust was limited. 

Around this time, in addition to the open sessions, the i6 program board began operating closed sessions. During these closed sessions, members discussed the ongoing issues with the contract. Accenture was not invited to attend these sessions, and it considered this structure unhelpful for the relationship.

2014

The Scottish Parliament’s Justice Sub-Committee on Policing held several evidence sessions with the SPA and Police Scotland to explore progress with the i6 program. 

In March 2014, the Sub-Committee expressed frustration at the lack of information about the problems with the i6 program that had been ongoing since August 2013. Police Scotland did not disclose the severity of the issues facing the program, nor was it overly critical of Accenture. This might have reflected a desire to maintain relationships with Accenture to keep the program on track or maintain the contract’s commercial confidentiality.

As the design of i6 progressed, it became apparent that Police Scotland and Accenture’s original assumptions looked doubtful. These were based on building on the Police Information Management System Accenture provided to Spain’s Guardia Civil and developing the remainder of the i6 system from scratch.

The original timescales and staff that Accenture had planned to allocate were based on these expectations, and it was not long before the program started to fall behind schedule. Several deliverables from milestones two and three had already been missed by this stage. 

The program board queried Accenture’s delivery methodology’s suitability with fears it could lead to problems later on in the program. The board sought assurances from Accenture about the delivery plan and associated risks, staffing, and skills that it planned to deploy throughout the program.

The program board also asked Accenture to provide future program board meetings with a quantitative assessment of its confidence in delivering i6. By February 2014, the program was around seven months behind the original plan. 

At the program board, the program team reported that to meet the requirements, Accenture was now developing far more from scratch than it had originally anticipated. 

The program board raised concerns about errors in design documents and expressed worry over the level of policing knowledge on Accenture’s team.

On 25 March 2014, the i6 program board considered the outcomes of lengthy contract negotiations with Accenture. These were captured in a contract variation agreement, which amended specific elements of the original contract, including a revised delivery and milestone plan. Accenture agreed to amend its proposed IT system to address all the gaps that had been identified and to deliver the requirements within the fixed price. Police Scotland took on responsibility for certain plan elements, such as the transfer of data.

The SPA approved the contract variation agreement in April 2014. The i6 go-live date was revised to July 2015, with full completion by September 2016. The new contract variation agreement reset the relationship between organisations and, temporarily, improved levels of trust. The program’s momentum picked up, and the program board approved payment for milestones two and three. 

For the first time since the i6 program started, its status was reported as green at the i6 program board meeting. Accenture’s assessment of its confidence in delivering i6 rose to 85 per cent in May 2014 when the detailed design phase (milestone four) was completed.

Over the next few months, the i6 system’s development continued, as did renewed disagreements about the program’s scope. Accenture maintained that Police Scotland was extending the program’s scope, which required a greater degree of bespoke development. Police Scotland maintained that there was no extension to the scope beyond changes agreed through the change control process. The program board again challenged Accenture over its timeliness of reporting problems and delays in delivery, as well as its documentation’s quality. It also raised concerns over the Accenture development team’s expertise and the high turnover of key personnel.

By August 2014, milestone five (functional design) was behind schedule. Payment for this milestone, of £2.6 million, was therefore also delayed. The program board was told that the delay in payment had been raised at the highest levels within Accenture. 

Accenture cited a more complex design than originally anticipated as the reason for the delay. The relationship deteriorated again as delivery dates slipped. The go-live date was delayed again from July 2015 to September 2015, with full roll-out remaining September 2016, but achieving this would require overlapping development phases.

To keep the program on track, Police Scotland adjusted the payment schedule. Most milestones were delayed throughout the i6 program, and the program board withheld payment until Accenture had delivered. However, in the latter part of 2014, in an attempt to alleviate an extremely strained relationship, the i6 program board agreed to split a milestone (completion of the functional design stage). 

This meant releasing payments when each component deliverables were achieved rather than waiting until all were achieved, as specified in the contract. While this could be considered pragmatic, given the multiple challenges and problems with the i6’s development, this is not good practice.

2015

The product testing phase began in January 2015, four months behind the original schedule. Accenture found various technical problems but assured the i6 program board it would resolve them. In February 2015, it assessed its overall confidence in delivering the program at 90 per cent.

After product testing, Accenture released the system to Police Scotland for user acceptance testing. In June 2015, the program board raised concerns about unresolved defects from Accenture’s product test phase.

Police Scotland and Accenture disagreed about how critical these were. The program board challenged Accenture’s testing strategy, which it did not consider was in line with industry standards. The relationship between the organisations was extremely fragile, with a lack of trust and frustration on both sides.

The “Waterfall” approach contributed to the fact that Police Scotland only discovered the true extent of problems with the system when it was delivered for testing. Although Accenture had provided Police Scotland with demonstrations of the developing i6 system, it was after a period of testing that the i6 program team reported to the program board in August 2015 that there were:

> critical errors in the technical coding;

> higher-than-projected levels of flaws that Accenture was not able to resolve as quickly as expected;

> serious concerns raised about the criminal justice module, which did not comply with the Integrated Scottish Criminal Justice Information System data standards;

> errors in the search and audit modules;

> problems around the limited functionality in the administration module (Accenture had already received payment for successfully delivering this element).

At the August 2015 program board meeting, Accenture agreed to analyse root causes and report back to the program board on its findings and actions to resolve these. At this meeting, Accenture assessed its confidence of meeting the go-live date of December 2015 at 91 percent. The i6 program board challenged this assessment.

In September 2015, there was an extraordinary i6 program board meeting. Accenture reported that the issues raised in user acceptance testing would need more analysis and remedial action. It said the December 2015 go-live date would not be achieved. It agreed to a joint re-planning exercise to be reported to the October 2015 i6 program board. 

Accenture did not attend the i6 program board in October 2015 as its analysis exercise was not complete. In November 2015, Accenture requested a period without prejudice, meaning all organisations could speak freely and openly, without the risk of what they say being used against them later if negotiations fail.

In December 2015, Accenture reported that the work still required on i6 would take an additional 30 months. This proposal would mean go-live would be delayed until April 2018, and the cost would be many millions more than the original contract price. Police Scotland rejected this proposal. The SPA, Police Scotland, and Accenture entered detailed discussions to explore the i6 program’s future options.

2016

A series of meetings between Accenture senior management, the SPA, and Police Scotland took place during early 2016 to consider options for the way forward.

After reviewing the final options appraisal report in May 2016, the SPA decided that the revised plan was not viable. The SPA and Accenture mutually agreed to terminate the contract, and both organisations entered commercial negotiations, which concluded in July 2016 when they signed the settlement agreement. 

The contract enabled the SPA to secure a settlement agreement of £24.65 million. This meant that Accenture agreed to refund the £11.09 million that the SPA had paid and to make an additional payment of £13.56 million. This reflects estimated staff costs and capital costs such as hardware maintenance and software licenses associated with i6.

2017

A review of the program by Audit Scotland, published in March 2017, found that the project “ultimately collapsed due to a damaging loss of trust between those involved and fundamental disagreements about what the program needed to deliver”.

Commenting on the report, an Accenture spokesperson said: “As the report acknowledges, the scope and the complexity of the solution for i6 increased significantly during the project.  This was driven by the client.  There were challenges and issues on both sides, but we worked closely with Police Scotland to review the program and recommend revised plans to successfully deliver i6. Despite our best efforts, it was not possible to agree to the necessary changes, and we mutually agreed to end the project”.

What Went Wrong

Besides closing their expertise gaps, Police Scotland also followed good practice by:

> Establishing a program board with overall responsibility for scrutinising and challenging the program’s progress and risk management arrangements.

> Appointing a senior responsible owner charged with ensuring the program met its objectives and delivered the planned benefits.

> Appointing a program manager with responsibility for the program’s day-to-day management.

The Scottish Government also provided some external assurance at various stages of the i6 program in the form of:

> Gateway reviews — these are short, focused reviews by the Scottish Government to provide an assurance check on a project’s status. It makes recommendations to help with decision-making on program management.

> Healthchecks — these are similar to gateway reviews but are more flexible in remit and scope.

> ICT technical assurance reviews — these are to ensure technical solutions meet user business needs. They review in more detail than the other forms of independent assurance.

So what went wrong?

Delivery Method

The i6 program used the classic “Waterfall” method. In this approach, the software is developed in distinct phases, each leading to the next phase in a sequence resembling a waterfall. 

Once a phase is complete, the process moves on to the next phase, and there is no turning back. It meant that all of the design, coding, and construction of i6 would be completed before Accenture released it to Police Scotland for testing. Police Scotland would pay for each phase when it was completed. 

The “Waterfall” method was common at i6’s origins, though “Agile” was gaining popularity. A more flexible, incremental approach where the team works on small-scale launches of a functioning product would suit the problem at hand far better. The development team tests each software launch against the user’s requirements throughout the project, and, in theory, changes can be made more easily. Recently, more organisations are adopting the “Agile” approach to software development.

The method adopted for developing the i6 system meant that the full scale of difficulties facing i6 ultimately became clear in August 2015 when the system was passed to Police Scotland for testing. There were fundamental flaws and serious errors. At this point, Accenture estimated that meeting the contract requirements would take an additional two and a half years, with go-live being delayed until April 2018, almost four years later than originally planned.

Politics

The i6 program took place in the context of a high level of public and political scrutiny. The SPA, Scottish Government, and Police Scotland did not always work together effectively before and after the merger process that established the SPA and Police Scotland.

During the i6 program, there were well publicised disagreements over responsibilities between the SPA and Police Scotland, including responsibility for ICT. In addition, the SPA and Police Scotland were facing high-profile issues such as the deployment of armed officers, stop and search policy, and emergency call handling.

The SPA and Police Scotland wanted to deliver i6. The failure of a previous police ICT project in 2012 (the Common Performance Management Platform) meant there was pressure on the SPA and Police Scotland to make i6 a success.

Furthermore, the i6 program was vital to Accenture at a global level. This might have led to misplaced optimism about the prospects of success and unwillingness to consider terminating the program.

Police Scotland considered legal action against Accenture for breach of contract as early as October 2013. Both organisations agreed to make an effort to resolve the disagreements over the contract and avoid an expensive legal challenge in court, which could have had potential political and commercial consequences.

Underestimating Complexity

The i6 program was complex and highly ambitious. Police Scotland and Accenture originally believed that most of the i6 system could be based on an existing IT system that Accenture had delivered elsewhere. This belief was incorrect. As the design and development of i6 progressed, it became apparent that Accenture would need to develop significantly more than had been originally anticipated.

Police Scotland concluded that Accenture had underestimated the system’s complexity and had, at the contract stage, overstated its own ability to deliver i6 within the timescales and fixed price agreed. The level of effort Accenture would require to complete i6 was around eight times greater than the resources Accenture had estimated when signing the original contract.

The belief that most of the system could be based on the system that Accenture had provided to the Guardia Civil was incorrect. It had become clear a virtually fully bespoke system was required.

How Scottish Police Could Have Done Things Differently

Since the i6 program’s closure, the following reviews have been completed, with learning identified and recorded for use by those engaged in future projects.

> Technical Audit of Accenture Development (by Scottish Government);

> Internal Review of i6 Procurement and Design (by Organisational Development);

> Internal Review of Training and Deployment Planning materials (by i6 program).

Each review’s objective was to identify activities and approaches which will be key to assuring future outcomes. Importantly, the reviews were approached positively, capturing areas of good practise that should be repeated and other areas that could be enhanced.

Key findings in these reviews included:

Iterative Development

> Challenges with implementing the “Waterfall” approach and a need for future consideration to be given to the alternate “Agile” iterative approach.

> Benefits of the supplier adopting a standardised “development approach”.

> The importance of realistic planning and resource allocation against distinct workstreams to ensure delivery against competing demands.

See "14 Essential Software Engineering Practices for Your Agile Project" for more insights on this topic.

Quality Control 

> Increased supplier adherence to quality management practices, with an emphasis on quality control frameworks.

See "Be a Responsible Buyer of Technology" for more insights on this topic.

The Right People 

> Supplier focusing on the experience and skill levels of those involved to ensure that the requisite skills are in place to assure delivery.

See "Outsourcing Technical Competence Is a Very Bad Idea" for more insights on this topic.

(Independent) Reviews

> Provision for independent technical assurance reviews at more regular intervals.

> Ensuring compliance with agreed technical entry and exit criteria when moving between the phases of the project.

See "Your Best Insurance for Multimillion-Dollar Tech Projects - Independent Project Reviews" for more insights on this topic.

Facing Reality

Despite the delays and serious problems throughout the program’s lifetime, Accenture provided regular assurance in the face of strong challenges about their confidence in delivering the i6 system. This assurance proved misplaced.

See "It Is Time to Face Your Project’s Reality" for more insights on this topic.

Effective Business Engagement

> The importance of effective business engagement and the need for appropriate investment in this area.

See "Successful Projects Need Executive Champions" for more insights on this topic.

Closing Thoughts

i6 was a complex and ambitious program. There is no single reason why it failed. Despite an 18-month competitive dialogue process, there was a fundamental disagreement between Police Scotland and Accenture about interpreting the contract and the program’s scope. This damaged relationships and trust between the two organisations from a very early stage. 

Both Police Scotland and Accenture were determined to deliver the i6 program. This might have led to optimism bias and a reluctance to pause or halt the project at an earlier stage. The “Waterfall” approach meant that Police Scotland would not test the system developed by Accenture until relatively late in the development process. There was also over-reliance on the existing system that Accenture had provided to Spain’s Guardia Civil.

The i6 program was a key component of police reform. Its failure means that some of the benefits that should have arisen from implementing it have been, at best, delayed. There was a need to modernise police ICT systems six years ago when the i6 procurement began. That need has not been met. Police officers and staff continue to struggle with out-of-date, inefficient, and poorly integrated systems. 

Other Project Failure Case Studies

> For an overview of all case studies I have written please click here.

References

[1] i6: A Review, Audit Scotland, March 2017.

[2] Written Response Audit Scotland i6 Review, Police Scotland, May 2017.

[3] Managing ICT contracts: an audit of three public sector programs, Audit Scotland, August 2012.

Read more…

Changing Technology Is Easy; Changing Behaviour Is Not.

This is an article I have written for one of my clients. You can find the original article here.

Creating a modern workplace is key to digital transformation. If you’re embarking on such an initiative, remember that changing technology is easy, but changing behaviour is not. Begin with the end in mind, ask what you want to achieve with your new modern workplace, and address user behaviour, user adoption and usage from the outset. 

Digital transformation is nothing new. It’s a daily reality for any company. Some are disruptors, and others are disrupted. Covid-19 has made this even clearer. Everyone understands that digital transformation - not evolution - is required to maintain a competitive edge. That’s why so many digital transformation initiatives have been started around the world. 

Modern workplace projects are a key part of such digital transformation initiatives. After all, it’s people who make companies successful, and they need to be able to do their work as efficiently and effectively as possible. So when you start thinking about a modern workplace project, it’s essential to start with the end in mind. 

Your project shouldn’t be about technology. Technology by itself is worthless. It’s what you and your users do with it that (potentially) creates value. So before we defined the scope of our own modern workplace project, we asked ourselves some hard questions. You might want to use them as a checklist when embarking on your own initiative. 

Questions to ask before a modern workplace initiative 

What do we want to achieve? 

And how is this project going to execute on this? 

What user behaviour do we want to see and support, and what behaviour do we want to stop or reduce? 

For example, do we want to facilitate more remote work? Or better quality online meetings? Do we want to start supporting hybrid meetings? Or stop sending documents as attachments and work with links and single versions of truth? 

How about BYOD (bring your own device)? 

Should employees be able to access company documents on their private laptops and mobile phones? How do we want to support working on tablets? 

What current set of applications do we want to stop using and replace? 

Is this realistic? We made sure that decommissioning them was part of the project scope to avoid simply adding a number of new applications to the landscape and increasing complexity instead of reducing it. Paying twice for the same capabilities has never been a smart thing to do for a CIO. 

How important is user experience? How important is security? 

And of course our legal and compliance requirements need to be addressed. Balancing these is one of the most difficult challenges in a modern workplace project. 

Is a SaaS solution like Microsoft 365 or Google Workspace the answer? 

Maybe. But certainly not the whole suite at once. It would be almost impossible to handle the change. So what applications should be part of this project, and who will support these applications after go-live? 

Get user buy-in

After you’ve answered these and other relevant questions, you have to think about your users. Because in the end, any technology is only as good as how well it is used. 

If users don’t know how to use applications effectively, the benefits will be small, or even negative. This means user enablement is critical to the success of any modern workplace project. Include user pilots in your project. Learn from your users. And make sure your users understand the following: 

1) Why you are implementing the new solution. 

When it comes to a new modern workplace, many employees will be instrumental in the change you’re promoting. It’s important to tend to their needs throughout the change journey. Transparent communication is key. 

2) How existing processes will change. 

Your leaders, colleagues, customers and suppliers all live in their own reality – and it’s likely to be different from yours. So invest in understanding their way of working before you force them to do something that doesn’t work for them. 

3) How to use the solution. 

Your user base can range from people who started their careers on terminals to millennials who are so accustomed to touchscreens that they don’t know what a keyboard is. Your ability to transform is only as good as the average skill level shared by the majority of your employees. And don’t forget about the new joiners. Training is never done. 

4) Who to contact if they require support with any problems or questions. 

If you’re working with an internal service desk, make sure that they’re equipped with standardised scripts. Alternatively, if you’re working with an external service provider, make sure you’ve selected a partner who can provide the highest level of support required for the new applications. 

5) Whether they can offer feedback and make suggestions to improve the solution. 

One of the best ways to build support organisation-wide is to give everyone a voice and a platform to share their views throughout the transition. Active user communities are pure gold. 

A modern workplace project is all about changing and supporting the right behaviours, supporting your business model and regulatory requirements, keeping your data secure, and implementing the capabilities you need most. It’s not about implementing new technology. Remember that people aren’t able to change their behaviour overnight, so you need to plan for a journey, not a weekend trip – which of course is true for any digital transformation initiative.

Read more…

How "Hubble Psychology” Is Skyrocketing Your ERP Program Costs

September 2012, NASA Inspector General Paul Martin released the results of an investigation that looked into why the U.S. space agency has had long-standing problems in meeting its programs’ cost, schedule and performance goals. 

For instance, in 2009, it was estimated that the James Webb Space Telescope (JWST) would cost US $2.6 billion to develop and launch by 2014. At the latest count, the tab has ballooned to over $8 billion for development (not including $940 million contributed by international partners) and another $800 million for five years of operational costs. The huge cost overrun on JWST—as well as many other projects—has not helped win the friends in Congress that NASA needs in order to maintain its funding. 

Many large ERP implementation projects suffer from such cost explosions as well. See for example Case Study 7: The $2.5 Billion Cross-Border Expansion Mistake by Target, Case Study 6: How Revlon Got Sued by Its Own Shareholders Because of a Failed Sap Implementation, and Case Study 3: How a Screwed-Up SAP Implementation Almost Brought Down National Grid. 

And although the parties involved in these projects typically don’t need friends in the US Congress, they sure do need some friends in the organization implementing the new ERP system. 

The NASA report is an interesting read and gives a number of explanations for why these cost explosions happen. A culture of optimism, lack of experienced personnel, technological complexity, and constant changes in funding to name a few. 

Many of these reasons apply to large ERP projects as well and are not new insights. But one paragraph of the report really got my attention. It states that one of the primary causes of NASA cost/schedule problems is what the inspector general calls the "Hubble Psychology" that is common among the organization's managers. 

“… Many project managers we spoke with mentioned the “Hubble Psychology” – an expectation among NASA personnel that projects that fail to meet cost and schedule goals will receive additional funding and that subsequent scientific and technological success will overshadow any budgetary and schedule problems. They pointed out that although Hubble greatly exceeded its original budget, launched years after promised, and suffered a significant technological problem that required costly repair missions, the telescope is now generally viewed as a national treasure and its initial cost and performance issues have largely been forgotten.” 

How pervasive is this psychology? The report noted that, “when asked whether their projects had been successful, every project manager we interviewed answered in the affirmative, regardless of the project’s fidelity to cost and schedule goals.” 

I have observed the same psychology with program, project and workstream managers of ERP implementations. The same for system integrators and suppliers that are involved in the project. 

Because the investments in these programs are so large, the sunk costs of these programs after they have run for a while are significant for most organizations. And that is why these projects are almost never cancelled after they have run for longer than 6-9 months. Financial and reputational losses are just too big. 

And everybody involved in these programs knows that. So why worry about going over budget and moving the go-live date a few times? 

And it seems that after an ERP system actually goes into production, no one actually remembers how much it overran its budget or schedule. Or that the budget was ridiculously high from the get go. So also after going live there is nobody that holds anybody accountable for overspending. 

To fix this you will need to find ways to reward managers and suppliers for good stewardship of your organization's resources as enthusiastically as it does for successful implementations and to hold managers and suppliers appropriately accountable for mismanagement of resources. 

In a nutshell: Spend what you need, not what others want you to spend. 

Read more…

The Biggest Challenge of Postmodern ERP - Cloud Integrations

Gartner originally coined the term “enterprise resource planning” (ERP) back in 1990 to describe a new breed of integrated software solutions. These solutions included modules for Material Requirements Planning (MRP), Manufacturer Resource Planning (MRP II), Financials, Human Resources (HR) and Customer Relationship Management (CRM).

During the 1990s and 2000s, these single monolithic ERP solutions became an essential part of most corporate IT strategies. But by the mid-2000s, such ERP solutions started to get a bad reputation. The very high price tag, the relative inflexibility of an integrated solution and the droves of ERP implementation failures that made headlines in both the tech and business world clearly showed the shortcomings of this strategy. It became hip to proclaim that “ERP is dead” or “the end is near”. 

Companies started implementing best-of-breed applications to replace individual modules within larger ERP solutions. For example, using Salesforce instead of the CRM module bundled with their ERP solution, or Workday to replace the HR module. Others ditched their ERP solution altogether and started using only best-of-breed applications and started connecting them with each other. 

Gartner coined the term “postmodern ERP” in 2014 to describe this new ERP strategy.

Postmodern ERP is a technology strategy that automates and links administrative and operational business capabilities (such as finance, HR, purchasing, manufacturing and distribution) with appropriate levels of integration that balance the benefits of vendor-delivered integration against business flexibility and agility. 

To put it another way, a traditional ERP solution is like the new car you buy every 10 years. A postmodern ERP solution is like owning the same car indefinitely, but with various components that can “easily” be changed out as needed.

Around the same time that postmodern ERP strategy became hip, we see the cloud strategy of both vendors and clients taking off. This means many of these best-of-breed applications are consumed in a SaaS model with all its advantages and disadvantages.

Whilst this all sounds like great ideas it gives us one very big challenge to solve; cloud integrations.

Why Cloud Integrations Are so Difficult

Cloud integration means integrating data used by disparate systems, between or within public or private clouds, or between cloud-based and on-premise systems. The goal is to create unified data stores that can be accessed efficiently and transparently by all relevant users and applications.

There are mature tools for data integration within public cloud providers or private cloud platforms, for example within AWS, Azure or an OpenStack data center. The main challenge begins when organizations need to integrate multiple public clouds, set up hybrid cloud environments, integrate legacy on-premise systems with cloud workloads, or lift and shift legacy workloads into the cloud.

You can increase flexibility and agility, or reduce complexity, but you can’t do both. Despite the rigid nature of traditional ERP solutions, they solved some key issues, including the need for consistency and integrity of data and processes. 

Let’s have a look at some aspects you have to deal with when implementing a postmodern ERP solution. 

Release Management

Each SaaS application that is part of your postmodern ERP solution will have a different maintenance and upgrade cycle. This means that you will have continuous change in parts of your end-to-end solution without you being able to stop this. This means you have to have a solid plan in place for regression testing. Does the end-to-end solution still work after each upgrade? Do your integrations still work? Are the data semantics still the same?

Environment Management

Tied to the previous point is environment management. In order to do development and testing you will need to have a development and test environment. This works a little differently for each SaaS application that is part of your postmodern ERP solution. And just as the production environment, they will all have different maintenance and update cycles, meaning it will be very difficult to have a stable environment for long.

Backup and Disaster Recovery

It is safe to assume that each SaaS application that is part of your postmodern ERP solution will have a solid backup and disaster recovery strategy in place. But they all will be different. Do you understand the impact of such an event on your integrations? On your data integrity? Did you test this?   

Firewalls

In hybrid cloud environments, when moving data between the cloud and on-premise systems, opening the firewall is a necessity that most network administrators anxiously resist. You will have to carefully evaluate the requirements for data integration flows that cross the firewall to find the solution that minimizes the exposure of enterprise systems and data.

Network Latency

Cloud environments are often preferred to on-premises because of their scalability: you can easily increase or decrease your usage of compute and storage resources in just a few minutes. But scaling your cloud environment will have limited effect if your network latency is too high, which puts a firm cap on the data integration workloads you can run.

Data Transfers

Moving data between clouds, and between cloud and on-premise systems, can be time-consuming and error-prone, or even unfeasible in some cases, depending on the data volumes and the required data transfer frequency. Cloud data integration won’t work without solid strategies to transfer data in a timely manner. There is a huge difference between 10 messages a second and a 100 -- trust me.

Data Integrity

How are you monitoring all points of integration and logging the data on the move? Depending on the method used, the process of monitoring and capturing data changes on source systems can increase the load on the source system causing slowdowns in operational systems.

Data Conversion

In traditional data integration projects, complex Extract Transform Load (ETL) workflows were set up to clean data and transform it into the precise format needed by target systems. Many SaaS applications work with unstructured data or provide a flexible data model for structured data. However, they still need data to be cleaned, treated and converted into the desired format. Integration strategies must consider how ETL can be performed without slowing down the integration or adding a lot of complexity.

Data Synchronisation

SaaS applications are often architected with scalability or performance in mind, not around data integration. For example, in a system that rapidly scales up or down, with data stored on dozens or hundreds of cloud instances, it may be challenging to synchronise with external systems.

Standardization

There is no standard approach or protocol for integrating data between SaaS applications, not to mention cloud and on-premise systems. Each cloud platform, service or resource tends to have different data schemas and formats. Data connectors or adaptors need to be constantly updated, as new cloud services are introduced or as applications are updated or modified.

Security and Compliance

Consider the security requirements and industry standards applicable to each of your datasets. Both your integration platform and all connected applications need to fulfil those requirements, both for one-time data transfers and ongoing data synchronisation.

In a nutshell: You can increase flexibility and agility, or reduce complexity, but you can’t do both. 

Read more…

Create a Lighthouse to Drive Your Transformation

I’m a firm believer in transformation through delivery. In my experience real change cannot be implemented without a vehicle that is used to drive it through the organisation and crash through existing barriers. 

Transformation is far more than just deploying new technologies for the sake of it. A genuine competitive advantage can only be gained through the combination of an organization’s culture, its strategic choices and way of operating. 

It’s about continuously enabling new and leaner operating models underpinned by flexible business processes, connected platforms, analytics and collaboration capabilities that enhance efficiency and effectiveness. 

It’s about searching, identifying and developing new technology supported business models and, above all, ensuring that customers and employees are at the center of whatever a company does.

A good way to start transformation through delivery is by defining a so-called lighthouse project early on in the process and letting your best people make it a success. A lighthouse project is a short-term, well defined, measurable project that serves as a model — or a “lighthouse” — for other similar projects within the broader transformation initiative.

This technical delivery project will show new ways of working and doing business combined with the use of technology and will help to identify organizational challenges that hinder you to do the same in the rest of your organization.

The 4 key attributes of such a lighthouse project are:

> It has high business value and visibility so it is less likely to be cancelled. A lighthouse project doesn’t need to directly benefit everyone in your organisation. But its value should be something that everyone in the organisation can appreciate and understand. It should have obvious results, such as “we have reduced the hardware purchasing budget by 70 percent” or “we have managed to increase our application uptime from 95 percent to 99.9 percent.”

> It has clear metrics. You’ll notice that the results mentioned above involve hard numbers. Collecting quantifiable data about your lighthouse project is important in order to make the demonstration of value as clear as possible.

> It touches multiple business units and silos to enable the identification of barriers and influence change.

> It has a hard deadline in the near future to focus the business and delivery team to provide pressure to break down barriers. You don’t want to wait years for your lighthouse project to be complete. Choose a project that can be implemented within a reasonable period of time—a year at most. Of course, you should be careful to balance time-to-completion with sophistication; don’t make the project too simple in order to make it faster to complete.

In a nutshell: A lighthouse project is the key to transformation because it shows everyone in your organisation what they can achieve by leveraging new ways of working, new ways of thinking, and using new technologies.

Read more…

Going Live Too Early Can Be Worse as Going Late

A significant part of my engagements involves post-mortems on large, failed technology projects. And out of professional interest, I study public project failures and write case studies about them. 

Based on my experience and studies, most of these disasters could have been prevented (or at least anticipated) based on the project’s available information.

While project teams are rarely surprised by the outcomes, sponsoring executives are often unaware of the impending tragedy until it unfolds – and sometimes at the expense of their careers.

One executive decision in particular can move your project from challenged to a full blown disaster. And that is going live too early with a business-critical system.

A well-known characteristic of large projects is that once “go-live” dates are planned and communicated it is hard to change them. There are multiple reasons for this. For example:

> Highly visible or public commitments may be difficult or embarrassing to come back from.

> Breaking contractual commitments may have significant financial consequences.

> Individual and company reputations may be damaged by not meeting a promised date.

> Significant costs may occur to continue the current solution beyond the cutover date.  These costs can include renewing expensive leases, licenses, or maintenance for equipment, software, or real estate.

> People currently working on the project may be needed elsewhere, creating a resource problem that would impact operations and other projects if the current efforts were extended.

Most of you have experienced such timeline pressures; they are real, and I do not intend to underplay them. But, as real and daunting as these pressures can be, they have to be balanced with the consequences that a premature go-live can have. 

You must consider both scenarios in your decision-making.

Estimating implementation timelines is an imprecise art, not a science. It is subject to error as well as risk and uncertainty. As your information systems become increasingly interconnected and sophisticated, coordinating changes takes time, patience, and hard work.

Systems often have substantial elements of organizational change that must be addressed before a go-live. If your new or changed system requires a meaningful change in how people do their work, significant time may be required to understand the implications of the change, modify processes, as well as user enablement through training and support. Changes in the processes may create downstream effects into other parts of your organization that must also be understood and dealt with.

Replacing all, or part of, an existing system can also be complicated by differences in how data is captured and stored between the old and new systems. The efforts and impact of stopping an existing system, converting and loading data, and restarting the new system are easy – and devastating – to underestimate.

Unfortunately, what happens often in large projects is that all these things are ignored as soon as the first milestones are missed. It suddenly is all about making the deadline, and not about doing the necessary work and mitigating these risks.

Below are a few prominent examples of what can happen if you go live too early:

> Case Study 13: Vodafone's £59 Million Customer Relationship Disaster

> Case Study 9: The Payroll System That Cost Queensland Health AU$1.25 Billion

> Case Study 6: How Revlon Got Sued by Its Own Shareholders Because of a Failed Sap Implementation 

> Case Study 3: How a Screwed-Up SAP Implementation Almost Brought Down National Grid

> Case Study 2: The Epic Meltdown of TSB Bank

So before you find yourself pushing relentlessly for the go-live date, I encourage you to consider the following eight questions about your team’s readiness for go-live.

1) Is your system ready?

Has it been thoroughly tested? Is there an honest assessment of the quantity and priority of its known defects? Have business users weighed in with an informed opinion about the acceptability or the feasibility of workarounds for the known issues?

2) Are your upstream and downstream systems ready?

Have interfaces been tested fully? Are instruments in place to monitor traffic and identify problems? What would be the business impact to your system and your surrounding systems if there is a significant disruption of data flow or a significant error in the data? What are the consequences of data delays? Can the teams operating your upstream and downstream systems support your cutover date?

3) Are your users ready?

Have the implications of changes been thoroughly analyzed and communicated? Are users ready for the changes to their processes? Does the organization understand the secondary effects of these changes on downstream processes? Are user staff sufficiently trained on the new system? Will there be enough staff available during cutover to deal with the learning curve of the new system and processes and work through unforeseen issues?

4) Is your technology infrastructure ready?

Are you confident that there is sufficient processing, network, and storage capacity to support the new system? Are your help desk and operations people trained and ready?

5) Is there a go-live playbook or schedule?

Did you provide details of the steps required to stop the current systems, migrate data, convert interfaces, and begin processing with the new system? Are roles and responsibilities during cutover clear? Are the timings of the cutover schedule reasonable? Has there been a successful dry run?

6) How and when will you know that the cutover was successful?

Success here would mean not just all aspects of the technology, but also the impacts on the processes and the people who interact with the system. Has monitoring been established to provide early warning of problems?

7) What is the impact of a failed implementation?

Have you considered the failure modes of the new system and its impact to your business and that of your partners? Are there feasible workaround processes in place should cutover disruption be extended?

8) Is there a feasible fallback plan?

Can the cutover be aborted if significant issues are discovered? How long can the new system be in place before the cost and complexity of backing out of the system will be preventing this? What is the “point of no return” on the implementation? Is the fallback plan as detailed as the go-live plan? Has it been tested?

Before succumbing to the inertia and inevitability of a committed date, an honest assessment must determine whether the system and organization in which they operate are ready for the transition. 

So instead of focusing on the list of reasons why a system “must” go live on the selected date, have an honest conversation with executive stakeholders about the true status of the initiative and the potential consequences of a failed implementation.

In a nutshell: Going live on time should not be the default. Going live when the system and organization are demonstrably ready should be the goal.

Read more…

Another Great Leading Indicator for Future Trouble - Issue Resolution Time

I have a very simple metric for determining the health of a project or an organization: the age of issues. 

Issues are like fish; when they get old, they stink. 

A sure sign of a lack of leadership and upcoming trouble is old issues or issues that take longer than necessary to resolve. 

Issues are obstacles that get in the way of execution. It is the project manager’s role to resolve and eliminate these issues as quickly as possible regardless of the owner or the cause. 

If the project owner cannot solve it (or doesn’t try) it is up to the executive sponsor.

If an issue stands in the way of executing project objectives or makes it difficult for project managers to perform, it’s your responsibility as an executive sponsor to resolve it. 

If the project manager is the problem, it is also your responsibility to solve this.

Fix the root causes of problems and fix them early. 

What you want to avoid is a form of collective project amnesia where issues come up and never get resolved. 

Issues have a funny way of resurfacing when they don’t get resolved.

In a nutshell: Old issues and/or issues that take too long to resolve are an indicator of poor leadership and a great leading indicator for trouble in the future.

Read more…