Sunday, December 15, 2019

Case Study 9: The Payroll System That Cost Queensland Health AU$1.25 Billion

Case Study: The payroll system that cost Queensland Health AU$1.25 billion
The payroll system implementation disaster at Queensland Health in 2010 is said to be the most spectacular technology project failure in the Southern Hemisphere and arguably the second worst failure of public administration in Australia’s history. The handling of the fires this year being first.

Queensland Health is the public sector healthcare provider for the Australian state of Queensland, located in the country’s northeast. It provides dental, medical, and aged-care facilities in Queensland, which has the most geographically dispersed population of all Australian states.

Queensland Health needs to ensure that adequate healthcare services can be provided in the most remote parts of the state, which has a population of 5.07 million across an area of 1.85 million square kilometers. Every day, the organization provides hospital services to approximately 40,000 people and is responsible for approximately 85,000 employees across 300 sites.

What began as an AU$6.19 million contract between the State of Queensland and IBM Australia to replace Queensland Health’s aging payroll system eventually led to over 35,000 payroll mistakes and will ultimately cost taxpayers a whopping AU$1.25 billion, which translates to approximately US$850 million.

When the system went live, a large number of Queensland Health employees, including doctors and nurses, were either incorrectly paid or not paid at all. It led to the resignation of the minister of health, industrial strike action and loss of staff members to other employers.

If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.

If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.

If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.

Timeline of Events

2002

At this point in time Queensland Health used two systems for their payroll: Lattice and the Environment for Scheduling Personnel (ESP) rostering engine. Lattice and ESP were rolled out progressively over six years from 1996 to 2002.

Payroll departments were part of their respective districts. Processing of pay was undertaken locally and there were close working relationships between line managers and local payroll staff.

Whilst processing of pay occurred locally, the actual running of the pay was undertaken centrally; essentially, a 'hub and spoke' model was used.

Lattice required a substantial amount of manual interventions to accommodate the complex award and incentive structures within Queensland Health.

2003

In 2003 the Queensland State government formally established a government shared services initiative (SSI) mandating that all state government departments, including Queensland Health, replace their existing legacy payroll systems with a standardized software solution that incorporates SAP HR and SAP Finance. The overarching objective of the SSI was to consolidate technology and resources through delivering a high-quality solution with standardized business processes.

The SSI was expected to deliver the following benefits: (1) increased opportunities through enabling workforce mobility; (2) increased visibility into the cost of services; (3) reduced data duplication through the consolidation of systems; (4) reduction in costs associated with licensing agreements; (5) reduction of personnel; (6) achieve economies of scale; (7) enable the government organizations to focus on their core competencies, thus increasing the standard of service; and (8) consistency of HR and finance information across all government agencies.

2005

In 2005, just three years after the progressive rollout was completed, Queensland Health received notification from the Lattice system vendor, Talent2, that their existing Lattice system was becoming obsolete and was no longer going to be
supported, with services and updates ceasing on June 30, 2008. As a result, Queensland Health needed to consider replacing the obsolete Lattice system much sooner than it would have liked.

SSI decided that the system for payroll would be SAP ECCS and Workbrain. Accordingly, it was decided that Queensland Health would replace the Lattice I ESP system with SAP ECC5 I Workbrain as part of the shared services initiative.

2007

As part of the SSI, the Queensland government established CorpTech within the Queensland Treasury to oversee the standardized implementation across all state government departments.

CorpTech was responsible for overseeing the consultant selection process (Request for Information, Request for Proposals, and Invitation to Offer) and managing the consultant organizations. CorpTech sent out a Request for Information (RFI) on July 2, 2007, and four consultant firms responded: Accenture, IBM, Logica, and SAP. Afterwards, CorpTech requested detailed proposals from the aforementioned consultant firms on July 25, 2007.

Prior to the RFI being issued, CorpTech had managed the implementation of SAP HR at the Department of Housing, and SAP Finance at the Department of Justice. These implementations proved to be quite costly as a substantial number of consultant firms and private consultants had been utilized. Due to the large expense associated with the multiple consultant firms, the consultant methodology for the SSI was changed to the prime contractor model on August 16, 2007.

Subsequently, on September 12, 2007, CorpTech released an Invitation to Offer, and IBM, Accenture, and Logica responded. Ultimately, on December 5, 2007, IBM officially signed an AU$98 million contract to be the prime contractor of the SSI.

2008

Around October 2008, IBM had not achieved any of the "contracted performance criteria." It had, however, had been paid AU$32 million of its AU$98 million contract.

The idea to build a payroll system across the entire Queensland public service was now officially scrapped, but the decision was made to push ahead with a payroll system for Queensland Health.

2009

To cater for Queensland Health's specific business needs, including the complex award structure, retrospectivity, and concurrent employment, a significant number of customizations were made to both Workbrain and SAP.

Payroll and user acceptance testing was performed in parallel over a series of stages starting July 2009. The first test of the payroll compared the pays of only 10 percent of employees from all employee groups when performed in the SAP HR and Workbrain rostering solution as opposed to the legacy Lattice system, which resulted in an AU$1.2 million discrepancy in the biweekly payroll.

2010

A second payroll test occurred in February 2010, which only resulted in an AU$30,000 discrepancy; however, casuals and overtime claims were not tested. Queensland Health accepted the inherent risks and opted to go-live without full testing of all the functionalities of the system.

IBM continued to operate under varying scope and the state government kept signing off on the change requests. The project documentation reveals that prior to the payroll system going live, the project underwent four revised go-live dates and four separate stages of change requests, often done at the last minute.

By the time the system finally went live in March of 2010, 20 months after the original start date, the bill had already ballooned to AU$101 million.

Given the significant issues identified following the initial go-live, it was decided to establish a payroll stabilization project specifically focused on stabilizing the new payroll system. The four key focus areas for this project were: standardization and improvement of district and division business processes; payroll processing; payroll system performance; and support and communications for staff, line managers, and other key stakeholders.

2012

The cost of going live with a premature system resulted in more than 35,000 payroll mistakes, and by this point it had cost the state in excess of AU$400 million just to operate the system. KPMG estimated that the cost of making the system function for the next five years would be another AU$836 million.

2013

IBM should never have been appointed as the prime contractor in Queensland’s failed health payroll project, according to the finding of the Commission of Inquiry that investigated the bungled project. The inquiry, which ran for nearly three months at a cost of AU$5 million, heard from some 50 witnesses, including former premier Anna Bligh, former health minister Paul Lucas, and senior IBM executive Bill Doak.

The report, by Commissioner Richard Chesterman, was handed down on July 31 in the Queensland Parliament by Premier Campbell Newman. The premier described the project as "arguably the worst failure of public administration in Australia’s history."

The report was particularly damning of the procurement process that led to the appointment of IBM, and decisions made by senior public servants and contractors involved with the project.

The report also found the state government could not recover any funds from IBM for the failure of the project, and that the decision to reach a negotiated settlement with IBM rather than commencing legal proceedings was made without any proper analysis.

2016

The Queensland government was ordered to pay significant costs to IBM Australia after failing in a bid to recoup losses from the health payroll debacle.

The Newman government launched legal action against IBM in 2013, arguing the company had misrepresented its capability to deliver a new payroll system on time and on budget.

But IBM challenged the lawsuit and pointed to a 2010 agreement that the company said released it from the damages claim.

A trial was held in the Brisbane Supreme Court in 2016 with Justice Glenn Martin ruling in favor of IBM.

What Went Wrong

System availability

During the payroll cut-over period to the new system, there were significant issues with the availability of the system to payroll staff which reduced the processing time available. This created an initial backlog of payroll forms and unprocessed adjustments for the period just prior to the go-live date that grew over subsequent pay periods.

It took approximately eight months to process the backlog of pay adjustments and forms to return to previous business as usual (BAU) levels.

Performance issues

The degree of retrospectivity accommodated by the Queensland Health payroll system, whereby staff could submit forms for work completed up to six years ago, was creating significant payroll system performance issues.

System issues

As of 2 May 2012, there were 570 logged system issues, 76 of which were identified as having the potential to impact on staff pay. System defect fixes and enhancements were required to occur during designated 'major release' schedules, of which there were three scheduled per annum. There were some delays in addressing specific defects and issues due to the prioritization of other 'fixes' including the pay date change, changes associated with enterprise bargaining changes, legislative compliance changes, etc. There was a need to gain endorsement for an agreed longer-term approach to implementing key system changes so that the release windows could be utilized more effectively.

Overpayments and entitlements

As of May 2012, Queensland Health had overpaid staff AU$112.3 million, of which AU$16.5 million had been repaid and AU$3.3 million was waived, leaving AU$9 million outstanding. Queensland Health had an obligation under the Financial Accountability Act 2009 to recover these amounts; however, there was a moratorium in place preventing Queensland Health implementing Queensland Health-instigated overpayment recovery.

Queensland Health was required to fund fringe benefit tax (FBT) liabilities associated with overpayments, and this represented a significant additional cost burden to Queensland Health. While the previously agreed overpayment moratorium was in place, the amount increased by approximately AU$1.7 million every two weeks.

In addition to overpayments, the issue of employee leave and balances requires further investigation and analysis. PwC has conducted a number of reviews into leave balances and they have identified that up to 20,000 leave transactions are still outstanding since the move from the previous Lattice Payroll system across to SAP.

New business processes

Part of the implementation of the new system was further standardization and centralization of payroll processing, including the introduction of central processing teams and a centralized pay run. As such, the key linkage between the districts and their local payroll providers was severed — payroll staff were required to process unfamiliar rosters for staff members across the state.

In addition, fundamental differences in how districts and line managers were providing pay information and rosters were identified, with each district continuing to provide the information in the format they had developed locally (this was a continuation of what had occurred with the Lattice system; however, now the payroll officers responsible for interpreting the pay information from the districts did not have the local knowledge or relationships that had previously assisted with the interpretation process).

How Queensland Health Could Have Done Things Differently

The key findings from the auditor general's report were as follows:

Project governance

Project governance prior to go-live, including managing relationships with key
Stakeholders, was not effective in ensuring roles and responsibilities were clearly
articulated and in ensuring there was clear accountability for efficient and effective
implementation of the system.

The governance structure for the system implementation, as it related to CorpTech, the prime contractor, and Queensland Health, was not clear, causing confusion over the roles and responsibilities of the various parties.

Management of the project became even muddier after it commenced. Numerous agencies and boards divided oversight and authority, causing significant confusion which, in the end, rendered them all "ineffective in establishing a shared understanding of stakeholder expectations in relation to the quality of project deliverables":

> The Solution Design Authority (SDA) (which, during this period, transformed into the program delivery office (PDO) of the state's CorpTech IT division);

> The Queensland Health Enterprise Solution Transition (Queensland HealthEST), the state's information technology management program and acting project manager (which inexplicably retitled the "Interim Solution" as the "Queensland Health Implementation of Continuity" (Queensland HealthIC) — no confusion there!);

> The executive steering committee (ESC), which included personnel from CorpTech as well as the shared services agency (SSA) and the Department of Education, Training and the Arts (DETA), and

> The "Release Steering Committee," which answered to both the ESC and CorpTech and counseled its chair regarding the development of the Interim Solution.

While there appeared to be lots of oversight of the program, Australia's auditor-general reported that "it was not clear which Accountable Officer had responsibility for the overall governance and successful completion of the whole project."

Scope and requirements

There was inadequate documentation of business requirements at the commencement of the project. The absence of a periodic review of the business needs contributed to subsequent difficulties with system testing and the implementation of a system which did not meet the needs of Queensland Health's operating environment.

The complexity of the project was immense and involved the management of over 24,000 differing combinations of wage payments and withholdings for over 80,000 workers and subcontractors spread over 13 contractors and multiple industrial agreements. Because of the fear that the existing system was in imminent danger of immediate failure, IBM agreed to take just seven months to develop and implement an "Interim Solution" to tide the agency over until a full replacement became operational.

Within that seven months, only two weeks were set aside at the beginning of the project to scope out the "critical business requirements" needed by the agency and the digital solutions that would respond to those demands. Not surprisingly, the lack of identifiable objectives was a significant cause of the project's abject bungling.

Go-live decision

When the system went live it was seriously deficient, causing staff not to be paid, or to be paid inaccurately. Neither party could have been under any misunderstanding that this was the inevitable result. It was obvious well before go-live that the project had been inadequately scoped. 

"When it went live it was seriously deficient, causing very many QH staff not to be paid, or to be paid inaccurately. Neither party could have been under any misunderstanding that this was the inevitable result. It was obvious well before Go Live that the Project had been inadequately scoped. Testing (especially UAT) revealed thousands of defects, a large percentage of which concerned functional aspects of the system. A competent and experienced tester (Mr Cowan) did precisely what his role called for in advising the customer (as the party deciding whether to accept the system) of these very major problems. Despite these warnings (ones which Dr Manfield said were the clearest sign he had seen of a system in distress) the State decided to Go Live” - paragraph 2.33 from the inquiry report.

"There could have been no clearer warning of the functional deficiencies in the system and the real risk of there being defects beyond the very large number which Mr Cowan had to that date detected. Had there been any doubt up until this point, Mr Cowan’s Final UAT Report put the matter beyond doubt. One does not need to have any technical knowledge to take from that report a very clear warning that, if the system proceeded to Go Live in its then state, there was a real likelihood, indeed perhaps an inevitability, that it would contain defects of a functional kind.” - paragraph 5.18 from the inquiry report.

Business processes

A number of critical business readiness activities and practices were not fully developed prior to the implementation of the new system. Several changes to payroll administration practices including the reallocation of processing duties within payroll were introduced at the same time as the release of the SAP and Workbrain systems.

Standardization

The implemented Workbrain (1,029 customizations) and SAP (1,507 customizations) systems were heavily customized and are not operating optimally in the Queensland Health environment. Customizations are costly to manage, increase risk and impact on system performance and should be minimized where practical.

Supplier

Despite its affiliation with a global digital leader, this was IBM Australia's first attempt at delivering a project of this size. That fact was not helpful considering that Queensland Health was probably the most complex of the Australian agencies needing the overhaul and was perhaps not the best candidate for IBM's first go.

Layer on top of those contributors the reality that the "Solution Design Authority," the state agency with the responsibility to define and maintain the scope, architecture, and design of the new system, was "passive, perhaps lazy" about communicating its requirements for a payroll system. Before project development began, the Solution Design Authority accepted IBM's "incomplete, ... unsatisfactory scope [of work] documents" as-is and with no questions. The project was off to a horrible start.

Closing Thoughts

As a consequence of Queensland Health’s disastrous payroll implementation project, the Queensland Government changed their Information Technology (IT) strategy and governance procedures.

Furthermore, the Queensland Government IT audit specified a series of constraints that must be placed on high-risk projects, which include the following:

1) project management personnel must be of the highest quality;

2) rigorous application of the Queensland Government project and program methodology;

3) the project must be approved by numerous chains of command, and

4) a reporting regime is to be established to increase the visibility of the costs associated with IT projects.

Being a state government department in the healthcare industry, Queensland Health is by definition already riddled with substantial layers of bureaucracy, adding to the complexity and increasing the difficulties associated with decision making, visibility, and accountability.

And these reforms, whilst necessary, add additional layers to the governance process, increasing both the number of bureaucratic decisions and the degree of red tape.

Consequently, both client and consultant organizations have been more cautious throughout recent technology projects in the public sector due to the increase in compliance which leads to project delays, and increased project costs.

Thus, aside from the financial and societal implications associated with the Queensland Health payroll implementation failure, the failure has also had national, industry-wide ramifications.

But do we really learn from these massive failures? Looking at the long list of big technology projects gone wrong after this one, I doubt it.

Especially in the area of public services.

If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.

If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.

If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.

Sources

> Queensland Health Payroll System Commission of Inquiry Report (2013)

> KPMG Review of the Queensland Health Payroll System (2012)

> Rebekah Eden and Darshana Sedera, The Largest Admitted IT Project Failure in the Southern Hemisphere: A Teaching Case (2014)

> Raul Manongdo, Queensland Health Payroll system – A case study on business process management and application enterprise integration (2014)

Read more…

Wednesday, December 04, 2019

Interview With Teresa Mandl (CEO, T.V.T swissconsult gmbh) on Project Sponsorship

Interview with Teresa Mandl (CEO, T.V.T swissconsult gmbh) on Project Sponsorship
For the last decade I have dedicated myself to helping C-level executives recovering troubled projects. If there's one thing I've learned in the process, it's that executive project support is priceless.

Engaged executive sponsors help organizations to bridge the communication gap between influencers and implementers, thereby increasing collaboration and support, boosting project success rates, and reducing collective risk.

Teresa Mandl is an example of such an engaged project sponsor. She is an expert for bringing technology-enabled innovation to market with 15+ years of experience in serving industrial and consumer-goods companies from SMEs to big corporates.

She selects and capture the right tech opportunities and turns them into successful products or services with the corresponding business models by aligning strategic perspectives, technological opportunities and customer needs in multi-stakeholder settings.

Tell us a little bit about yourself.

I am the founder of T.V.T swissconsult gmbh, a firm for innovation management, product and service design across industry boundaries. I am the managing director of the company.

My roots, education-wise, are in business and economics, while I grew up in a family business that comes from classic industrial design and engineering. Currently, my company grows together with my dad’s business, which is a very good experience. In addition, I teach innovations management in advanced studies at the University of Applied Sciences in Winterthur and Lucerne.

Here, I have gained a lot of insight from various industries and perspectives on what’s moving companies right now. I am involved in start-up juries such as the Female Innovation Forum as I care a lot about fostering entrepreneurship and at the same time sharing my experience as an entrepreneur with others.

Can you tell us something about your experience as a project sponsor?

I have been in the role of project sponsorship for more than a decade. Moreover, I have experienced many sponsorships on my client-side from an outside perspective. The size of the projects I’ve been involved in varies a lot, from two to approximately ten people.

These experiences include not just one firm, but a variety of involved companies and disciplines such as research, materials development, product design, and sourcing. The duration of the projects varies from three months to two years.

What do you think is the single most effective thing a project sponsor can do to positively influence a project?

To me, the joint definition of goals and project success factors is key. As such, it may be that the project sponsor sets the project up with goals in mind. However, I find it critical to reframe these goals from different perspectives and in the ‘language’ of the involved parties, so to speak.

Starting and conducting a project as a joint venture and not in a hierarchical way can be achieved that way. Staying involved in the course of the project, together with an open dialogue-oriented culture, is also critical.

What do you think is the single most effective thing a project sponsor can do to negatively influence a project?

Having an “I’ve seen it all, I’ve done it all, I know everything” attitude is a killer, especially in innovation projects. Also, the notion that failure is not acceptable. Innovation projects require a lot of resilience from project members, and this is something that can be nurtured by giving project members the feeling that they control the outcome.

What was your biggest success as a project sponsor, and why?

I would say my biggest successes are always projects with many critics that, in the end, turn out a success—the projects that convince the critics. One of these was starting my own company right after university in spite of the chance to enter the family business right away, or to go work for one of the Big 5 consulting firms. I had to fund the business myself, acquire clients from scratch, and grow it in a way that it is respected by, first of all, my father, and second, his staff. Now we are in an equal position where we respect each other and where one plus one is greater than two.

What was your biggest challenge as a project sponsor?

My biggest challenge so far was to deal with some offshore technology sub-suppliers in a very time-critical and prestigious project. They kept communicating that the project was going really well, but two days before the result was due for presentation at senior level, the suppliers would not deliver at all. The challenge here was not living up to expectations—both our own and those of others. First, it was hard to communicate to the client the sudden turn of the project. Second, we had to determine how to go on with the sub-supplier; they had taken advantage of our trust, but we also relied on them for other matters. In the end we were able to move critical activities to another company, so although the project took a bit longer, the outcome was positive.

The experience showed me that not everything should be done on a fully remote basis, how important regular face-to-face contact is, and that there is a need for sub-suppliers to regularly demonstrate progress in a way that is comprehensible for the sponsor. My personal learning was to add certain items to consider when judging whether to use a sub-supplier or not, and to look a bit closer and beyond merely technical criteria when selecting who to work with. It is better to do an extra loop in evaluating the team fit than be sorry afterwards. Plus, oftentimes the willingness of a team member to go the extra mile is more important than technical skills alone.

What was your biggest failure as a project sponsor, and why?

My biggest failure was a project where we thought we could convince a client to use top-edge technology when our client was not ready to do so, neither from a cultural perspective nor from a skills perspective. A couple of years later, the industry widely adopted the technology that we recommended early on.

However, in order to be innovative, all parties must be ready to risk something and try things out, which our client at that time wasn’t. We ended up developing a much too advanced product that our client could not justify within the company, and therefore the project was killed. Since then, we have tried to assess the risk aversion and cultural aspects much better before starting a project with clients.

How do you determine a project is really necessary and valuable?

To determine a project’s necessity, we match projects against a hierarchy of quantitative goals and qualitative aims that we have with the project. We determine whether a project stays valuable by clear project controlling.

It is more and more essential that such project controlling is done for various stakeholders. As innovation projects are less singular and more and more a matter of ecosystems or systems, different perspectives must be clarified; these could be a firm’s profit or societal impact perspective, an eco or sustainability perspective, or many others.

Here, I feel it is important to acknowledge that in the course of a project, the priorities of the goals may shift, so there is a continuous need for review in terms of necessity and value.

How do you recognize your project is in trouble?

Our innovation project controlling serves as a quantitative dashboard for projects and lets us determine whether projects are off track. Once we determine that a certain number of priority targets have not been met for a certain amount of time (this varies according to the estimated duration of a project), we see the call to action.

But an evaluation of the project never goes without applying a good amount of gut feeling at the same time, especially for qualitative factors. In order to allow that gut feeling to be communicated, we do regular project sounding meetings in a very open-minded atmosphere so the involved parties feel free to voice any concerns they might have. From time to time expert judgement, customer feedback, and stakeholder input is employed.

What advice would you give to a first-time project sponsor?

My advice would be, of course, to talk to people who have been sponsors before. Ask about the pitfalls and success factors as well.

Next, I am a visual person. I like to sketch stakeholders, timelines, critical incidents that could happen, and limitations. Sometimes such a visualization discloses a lot, and it also serves as a very good means for discussing a project with others. Also, to be convinced about a project, it is crucial to have intrinsic motivation on top of extrinsic motivators. It is so much easier to master a project with passion than out of pure obligation.

Finally, once a project is done, go back to what went right or wrong and why … and take these findings into the next project to do even better.

What are you looking for when selecting a project manager for your project?

I am most of all looking for a person who shares similar values and vision, with a “can-do” attitude. At the same time, a project manager must be a logical person who can overview many different things at once and put them into an order that all project members understand.

That being said, for innovation projects, a project manager needs to understand that the nature of innovation means that many things don’t work out according to plan. So for innovation projects, project managers must be trained to think in hypotheses and experiments. Once a hypothesis is tested and its viability is determined, the project manager needs to be able to define the levers for progress.

Last but not least, a project manager in innovations needs to bring in a certain level of empathy capability. Engineers function very differently from user experience (UX) designers or product managers. An innovation project manager must be aware of that and handle any feelings for the good of the project.

What are you looking for when selecting a steering committee member for your project?

Among other traits, steering committee members need to be people who are not afraid to ask questions and who can challenge approaches in order to prevent group-think in projects.

What is/are your most important lesson(s) learned as a project sponsor?

Set ambitious goals but manage expectations wisely.

This is the third in a series of interviews with executive project sponsors. The interviews will be part of my upcoming book “The Art of Project Sponsorship.”

The first interview was with Urs Monstein (COO, VP Bank).

The second interview was with Carole Ackermann (CEO, Diamond Scull).

To read more about the book just click on the image.

The Art of Project Sponsorship

Read more…

Sunday, December 01, 2019

What Is the Real Value of Your Technology Project?

What Is the Real Value of Your Technology Project?In order to assess project opportunities and make difficult trade-off and priority decisions about where to focus your limited resources, you need to be able to compare projects on a like-for-like basis.

And since there’s just no getting around the fundamental challenge that all organizations should be sustaining themselves, at some point the projects we invest in should create value.

Therefore, you should make project valuation — estimating the value of your projects — a part of your decision-making process.

So what is the value of a project? It’s simple:

Value = Benefits − Costs

In previous articles I discussed estimating project costs (see “What Are the Real Costs of Your Technology Project?”) and project benefits (see “What Are the Real Benefits of Your Technology Project?”).

If you have both the costs and benefits of your project in dollars, the computation of value is very straightforward.

But what this definition is completely ignoring is time. And time has a major impact on the value of a project.

Let’s take two projects, A and B, as an example. All figures are expressed in thousands of U.S. dollars.

Project A
Year
1
2
3
4
5
6
7
Total
Costs
800
100
100
100
100
100
100
1400
Benefits
400
500
400
300
300
200
200
2300


Project B
Year
1
2
3
4
5
6
7
Total
Costs
2000
1000
1000
1000
500
500
500
6500
Benefits
0
0
3000
3000
2000
1500
1000
10500

Now we will have a look at how time influences the value of these projects.

Measurement Period 

Let’s take one of the most used project valuation methods as an example: return on investment (ROI).

Return on investment (ROI) is a performance measure used to evaluate the efficiency of an investment or to compare the efficiency of a number of different investments. ROI tries to directly measure the amount of return on a particular investment, relative to the investment’s cost.

To calculate ROI, the benefit (or return) of an investment is divided by the cost of the investment. The result is expressed as a percentage or a ratio. In our case, the investment is our project. The return on investment formula is as follows:

ROI = (Current Value of Investment − Cost of Investment) / Cost of Investment

When we apply this formula to our project A we will get the following result:

Project A
Year
1
2
3
4
5
6
7
Accrued Costs
800
900
1000
1100
1200
1300
1400
Accrued Benefits
400
900
1300
1600
1900
2100
2300
Value
-400
0
300
500
700
800
900
ROI
-50.00%
0.00%
30.00%
45.45%
58.33%
61.54%
64.29%

And for project B we get:

Project B
Year
1
2
3
4
5
6
7
Accrued Costs
2000
3000
4000
5000
5500
6000
6500
Accrued Benefits
0
0
3000
6000
8000
9500
10500
Value
-2000
-3000
-1000
1000
2500
3500
4000
ROI
-100.00%
-100.00%
-25.00%
20.00%
45.45%
58.33%
61.54%

You see that, depending on what year you use for measuring your ROI, the results are totally different.

Time to Value

The time to value (TTV) measures the length of time necessary to finish a project and start the realization of the benefits of the project. One project valuation method incorporating this concept is the payback period (PB).

The payback period refers to the amount of time it takes to recover the cost of an investment. Simply put, the payback period is the length of time until an investment reaches a break-even point. The desirability of an investment is directly related to its payback period. Shorter paybacks mean more attractive investments.

When we look again at Project A and Project B you see a massive difference in the payback period.

Project A has a payback period of 24 months, and Project B has a payback period of 42 months.

Time Value of Money

There is one problem with the payback period: It ignores the time value of money (TVM). 

TVM is the concept that money available at the present time is worth more than the identical sum in the future due to its potential earning capacity. This core principle of finance holds that, provided money can earn interest, any amount of money is worth more the sooner it is received. TVM is also sometimes referred to as present discounted value. 

That is why some project valuation methods include the TVM aspect. For example, internal rate of return (IRR) and net present value (NPV).

Net present value (NPV) is the difference between the present value of cash inflows and the present value of cash outflows over a period of time. NPV is used in capital budgeting and investment planning to analyze the profitability of a projected investment or project.

The following formula is used to calculate NPV:

What Is the Real Value of Your Technology Project?

As you can see, the higher your rate of return “r” is, the lower the current value of your project. Typically the value for “r” is determined by management.

The internal rate of return (IRR) is the discount rate that makes the net present value (NPV) of all cash flows from a particular project equal to zero. IRR calculations rely on the same formula as NPV does.

To calculate IRR using the formula, one would set NPV equal to zero and solve for the discount rate (r), which is the IRR. Because of the nature of the formula, however, IRR cannot be calculated analytically and must instead be calculated either through trial and error or using software programmed to calculate IRR.

Generally speaking, the higher a project's internal rate of return, the more desirable it is to undertake. IRR is uniform for investments of varying types and, as such, it can be used to rank multiple prospective projects on a relatively even basis. 

Assuming the costs of investment are equal among the various projects, the project with the highest IRR would probably be considered the best and be undertaken first.

Cost of Delay

Cost of delay (CoD) is a key metric that represents the economic impact of a delay in project delivery. It is a way of communicating the impact of time on the outcomes we hope to achieve. More formally, it is the partial derivative of the total expected value with respect to time.

CoD combines urgency and value — two things that humans are not very good at distinguishing between. To make good decisions, we need to understand not just how valuable something is, but also how urgent it is.

I discussed CoD in detail in the article “What Is the Real Cost of Delay of Your Project?”. Depending on your urgency profile, your project end date can have a significant impact on the value of the project.

So what is the real value of your project? 

As we have seen in this article, the value of a project is determined by its benefits, costs, duration, and urgency. Putting it all together leads to the following diagram.
What Is the Real Value of Your Technology Project?
An ideal project valuation method is one where all metrics will indicate the same decision.

Unfortunately, the approaches mentioned above will often produce contradictory results.

Depending on management's preferences, your economic situation, and selection criteria, more emphasis should be put on one metric over another.

As explained above, there are common advantages and disadvantages associated with these widely used project valuation methods.

Nonetheless, you should use one or more of them in your selection process.

In a nutshell: Determining the dollar value of your projects is essential for selecting the right one. Value = Benefits − Costs, and is dependent on duration and urgency.

You can buy my eBook The Project Valuation Model ™ by clicking here or on the image.

Read more…

Wednesday, November 13, 2019

What Is the Real Cost of Delay of Your Project?

What Is the Real Cost of Delay of Your Project?
One challenge that every organization faces is deciding which project should be prioritized. Would it be in a company’s best interest to embark on a $1 million project that would take three months to finish or a two-year project that would cost $30 million?

And if you want to do both, which one should be done first? Or is parallel the best option? Does it makes sense to speed up or slow down an existing project?

All these questions center around the economic impact of the project's delivery time on it's business value.

Starting what date can we expect value to be delivered? And what happens when we don't deliver on that date?

Cost of delay (CoD) is a key metric that represents the economic impact of a delay in project delivery. It is a way of communicating the impact of time on the outcomes we hope to achieve. More formally, it is the partial derivative of the total expected value with respect to time.

Although the word "delay" implies this only applies to ongoing projects, exactly the same metric can be used for projects that have not started yet.

CoD combines urgency and value — two things that humans are not very good at distinguishing between. To make good decisions, we need to understand not just how valuable something is, but also how urgent it is.

Unfortunately, many organizations don’t use CoD in their decision-making process — mostly because they do not know why it's important and how to do it. This can be a very expensive missed opportunity as delays may cost millions depending on the scale your organization operates on.

In my opinion, every organization needs to understand what CoD is, how to calculate it, and how to optimize your project portfolio to reduce CoD.

Estimating cost of delay

As stated before, CoD is the economic impact of a delay in project delivery. Before we start with estimating this economic impact, here are some examples of possible CoD:

Product development – the amount of money you will lose if the launch of your new product will be two months later.

Software development – the amount of money you will lose if the release of an essential feature causes a big client to move to a competitor.

Solution implementation – the amount of money you will lose if you fail to replace the existing ERP system by the end of this year.

IT operations – the fines you have to pay if your systems do not comply with new regulations on time (GDPR, FATCA, etc.).

To estimate these potential costs of delay for your project, start with estimating the benefits that you will receive per week after you deliver anything to the market or to your organization.

These benefits typically take the form of:

1) Increased Revenue
2) Protected Revenue
3) Reduced Costs
4) Avoided Costs
5) Positive Impacts

See my article “What Are the Real Benefits of Your Technology Project?” for details on estimating your project's benefits.

For example, if you are selling a software-as-a-service (SaaS) product and you plan to deliver a new add-on feature that you anticipate will bring you $20,000 per week, every week that you postpone the release will cost your company this sum in missed revenue.

After you've estimated the benefits per week you can estimate the costs of the project per week. After all, when you are not starting a project the costs are zero, but when you do start the project you will have costs.

See my article “What Are the Real Costs of Your Technology Project?” for details on estimating your project's costs.

In order to estimate the real cost of delay per week you have to subtract these costs from the benefits. Not doing this is unfortunately a mistake many organizations make when using CoD.

Always remember that CoD is the partial derivative of the total expected value with respect to time.

Value = Benefits - Costs

When your project has a duration longer than six months you can estimate the benefits and costs per month instead of per week.

Types of cost of delay

Before you start estimating the CoD of your project it's important to understand that there are different types of CoD. Remember CoD combines value and urgency. Where value is depending on your project and your business, urgency takes usually one of four forms. These can be described with so called urgency profiles. Each of these profiles will lead to a different type of CoD.

In order to understand the urgency of projects, we need to understand the life cycle of benefits, and the effect of being late. The effect of delay can be different depending on what the type of benefits are, and whether there is any influence from the wider market.

The two main variables to consider are a) the length of the lifecycle of the benefits (how quickly the benefits ramp up and down) and b) whether the peak is affected by delay or not.

Profile 1: Short life-cycle; peak affected by delay

In some cases the life cycle of benefits is relatively short. Benefits ramp up to a peak and quickly decline again. Sometimes this is because the value-add quickly becomes standard for customers.

This is for example common for mobile phones. What is hip and differentiating today will be bargain-basement stuff in months rather than years. In other examples, the market is always creating new alternatives, moving on quickly to the new “new thing.”

The fashion industry is also a good example of this, which is why Zara competes by having very short lead-time from spotting the new look on the catwalk, to selling it on the hangers in the store.

With these types of benefits, if you are late, the peak benefits are affected, as shown below.
What Is the Real Cost of Delay of Your Project?
Profile 2: Long life-cycle; peak affected by delay

Another urgency profile is where there is a first-mover advantage, making it difficult for latecomers to recover position. This can be due to barriers to entry or the advantage that scale can bring.

A good example of this is cloud computing services. Offering something similar to Google Cloud Platform, Amazon Web Services, or Microsoft Azure can only be done with huge investments, and even then your chances are slim. This urgency profile is typical for services for which the market consolidates down to two or three major players. These products benefit from some form of preferential attachment mechanism or “network effect.”
What Is the Real Cost of Delay of Your Project?
Profile 3: Long life-cycle; peak unaffected by delay

A third urgency profile is where the lifecycle benefits are long-lived. Benefits ramp up to a peak and are sustained over a long period. In most established organizations this is the most common urgency profile you will find.

A typical example is where we are automating a process or improving efficiency, reducing time or cost. The ramp-up and peak of benefits is effectively the same whether the solution is late or not. This is also the easiest urgency profile for which to calculate the cost of delay, as it approximates nicely with a simple parallelogram.
What Is the Real Cost of Delay of Your Project?
Profile 4: Impact of an external deadline

Where requirements have a specific deadline the urgency profile is slightly different. The benefits profile itself can look like any of the above three, but the benefits only ramp up around a certain date.

As a result, the cost of delay is zero until the point where you need to start development — to avoid incurring any delay cost. To calculate the point at which the CoD kicks in, we need to consider the likely lead time, ensuring that the solution is delivered just-in-time, rather than too early or too late.

Let’s take for example a new regulation that will be effective from the 1st of January 2021. As of that date, front-office staff will need to prepare extra documentation in order to meet this regulation. The requirement is to automate the new documentation process so that the front-office employees can produce the documentation automatically.

The requirement will avoid the additional manual processing resource, which is estimated to cost about 10 full-time equivalents (FTEs) at $52K per FTE. The benefit type is categorized as “Avoided Cost.”

10 FTEs x $52K = $520,000 per year = $10,000 per week

Let’s say it is the 1st of March 2020 right now, which means we have 10 months to do something about this requirement. It’s going to take about 20 weeks to deliver this new feature. If we start developing the solution now it will be delivered in August 2020, but we don’t need it until December 31.

We deal with this by calculating when we need to start developing the solution by subtracting the duration from the external deadline. In this case we need to start development by the 1st of August at the latest (20 weeks before the external deadline of January 1st).

Before the 1st of August: cost of delay is $0 per week.

After the 1st of August: cost of delay is $10,000 per week.

In a nutshell: It’s vital for companies to understand what cost of delay (CoD) is and how to calculate it. Knowing the impact CoD will have on a business will give management greater insight on the projects in the pipeline and help them determine which ones should be prioritized.

You can buy my eBook The Project Valuation Model ™ by clicking here or on the image.

Read more…

Tuesday, November 12, 2019

Interview With Carole Ackermann (CEO, Diamond Scull) on Project Sponsorship

Interview With Carole Ackermann (CEO, Diamond Scull) on Project Sponsorship
For the last decade I have dedicated myself to helping C-level executives recovering troubled projects. If there's one thing I've learned in the process, it's that executive project support is priceless.

Engaged executive sponsors help organizations to bridge the communication gap between influencers and implementers, thereby increasing collaboration and support, boosting project success rates, and reducing collective risk.

Carole Ackermann is an example of such an engaged project sponsor. I met Carole for the first time at an event held by Business Angels Switzerland; at the time, Carole was the club's president. She is heavily involved in the startup scene of Switzerland, is an experienced corporate board member, and CEO and co-owner of Diamond Scull.

«Diamond Scull» is the leading rowing event of the «Henley Royal Regatta» on the river Thames. It’s where the best scullers meet to race. ― the spirit of going for the best especially under harsh conditions is what guides Carole.
.
I asked Carole for her insights on project sponsorship because of this highly interesting mix of experiences.

Tell us a little bit about yourself.

Passion for innovation — that's what drives me. I hold a PhD in business administration and have more than 20 years of leadership experience in SMEs and large organizations. I invest in startup companies, and I support various companies such as Allianz, BKW, BVZ Holding and Plaston as a non-executive board member.

Participating in a management buy-out and building up businesses led me to my current engagements as a member of the board of be-advanced, as a jury/investment committee member of different startup initiatives, and as a senior lecturer at the University of St. Gallen and other universities.

Can you tell us something about your experience as a project sponsor?

Through all of my career I have been sponsoring different projects. I started as a consultant with Arthur Anderson, where I launched the EMEA Branding initiative together with colleagues from the UK. The core team consisted of eight people involving all regional heads.

During my time at Saurer and Ionbond I often brought up special topics worth digging into; many times, this led to projects with several members. Depending on the project, the budget started from several thousand dollars to millions of dollars, and the duration was from several weeks up to two years.

Today I am also engaged in projects with the aim to increase the number of female entrepreneurs. I support talented women in their careers and partly secure the financing of their startups.

What do you think is the single most effective thing a project sponsor can do to positively influence a project?

There are a number of highly effective things a sponsor can do.

> Be clear about what is to be achieve and what success means
> Involve the right people
> Show passion for the project
> Trust people and give them decision power

But if I had to pick only one, it would be to make sure that the right people are in charge and strengthen their confidence in themselves.

What do you think is the single most effective thing a project sponsor can do to negatively influence a project?

Here I find it also hard to limit myself to one.

> Distributing information selectively
> No skin in the game, i.e., not taking responsibility
> Withdrawing important resources mid-project
> No milestones, organization, and success criteria

But the one thing that guarantees a negative influence on the whole project and each of its members is to sponsor a project without believing in it. People will notice and feel this.

When you don’t care and don't believe in it, why should they?

What was your biggest success as a project sponsor, and why?

Leading without power and budget. During a vacation in China i realized how big this market is and saw new business opportunities for the company I worked for. When I pitched the idea to my boss he said, “Go for it, but you will have a very limited budget." So, I commuted intensively to China, and using just a very small budget of the marketing division, I evolved the idea to a project that resulted in a substantial business with three production sites.

I was by myself far away from home, started without a budget, had very limited China experience, and finally created a strong result. This was a very satisfying and empowering experience.

What was your biggest challenge as a project sponsor?

The post-merger integration of marketing departments from six previously independent business units.

As Head of Corporate Marketing I was responsible for aligning all the marketing people of these business units. But none of them asked for this integration or for alignment. They all wanted to stay independent, and the last thing on their agenda was an integrated marketing program.

What was your biggest failure as a project sponsor, and why?

As a longtime investor and sponsor of a startup in ophthalmology I recognized too late that the first technology was not attractive enough for the market, and the second technology could principally not work.

I took decisions without fully understanding the technology and depended on blind trust in our “specialists” and “academic advisors.” Since that experience I recognized how important it is for a sponsor to have a judgement about the key matters of a project or startup.

How do you determine a project is really necessary and valuable?

As a project sponsor you need to see a purpose in the project; it must have a reasonable chance to be doable; and, if successful, it must have an impact.

This can be something that increases the value of a company or a person or does something good for the society. Today organizations are expected to create more than profits. Moreover, they are expected to focus on creating positive values for clients, employees, and society. Activities that create societal value are more likely to remain profitable in the long run.

The Integrated Profit & Loss (IP&L) approach provides all you need to report and steer your total value creation objectively and transparently. IP&L enables a better understanding of value creation per capital and stakeholder. In the IP&L, value is distributed across different stakeholders. Examples of stakeholders are investors, clients, employees, suppliers, governments, local communities, and society as a whole.

As a board member I address the importance of Corporate Social Responsibility for a sustainable business and I hope more and more organizations start seriously addressing challenges such as decarbonizing the business to name one.

How do you recognize your project is in trouble?

If people involved in the project lose faith and fun, you know you are in trouble. And you will only notice this if you spend time with these people and start listening in order to understand. This way, you will notice (potential) issues before they appear on your status dashboards.

What advice would you give to a first-time project sponsor?

There are two types of project sponsorship: casino sponsoring and project sponsoring. The second is more promising. You should only sponsor a project if you are experienced with projects and if you understand its content. Otherwise it’s just gambling.

There are very few projects you can do alone. One of the most basic success factors is to be able to work in teams or even better to love it. If you do not want to do everything by yourself, you have to win their heads, their hearts, and their hands — also known as the 3 Hs.

I am convinced that the more you know about how to work with people, how to involve people, how to lead people, how to listen to people, and how to support people, the more successful you become as a project sponsor. If you are not born with these qualities, you can cultivate them. Learn how to work with people, gain experience, and reflect on what was good and what could be done better.

What are you looking for when selecting a project manager for your project?

The fundamental task of a project manager is to deliver a project on time and at cost. So he definitely needs a good understanding of project management and must understand or at least have a judgment on the substance of the project.

But this is not enough. Project managers should also have experience and interest in the project and enough emotional intelligence to lead people. Finally, they should be passionate about the project.

What are you looking for when selecting a steering committee member for your project?

Very simple. I'm looking for E, N and K — experience, network and knowledge in a specific field. The same holds for strategic advisors.

What is/are your most important lesson(s) learned as a project sponsor?

Everything is about people. It’s about passion, persistence, knowledge and teamwork.

This is the second in a series of interviews with executive project sponsors. The interviews will be part of my upcoming book “The Art of Project Sponsorship.”

The first interview was with Urs Monstein (COO, VP Bank).

To read more about the book just click on the image.

The Art of Project Sponsorship

Read more…

Tuesday, November 05, 2019

What Are the Real Benefits of Your Technology Project?

What Are the Real Benefits of Your Technology Project?
The benefits of a project.

Everyone talks about it, and there is no lack of opinions about what they are, but when it comes to project benefits it’s often hard to reach consensus on what people actually mean.

Additional revenue, higher profit margin, happy employees, satisfied customers, new clients, less pollution, less waste, operational efficiency. They are all benefits.

Most dictionaries define a benefit as something like “a helpful or good effect.” But good effects are very hard to measure, compare, and understand.

Therefore, in my opinion ALL benefits of a technology project should be expressed in dollars (or any other currency).

Using dollars is helpful for a number of reasons. The key advantage is that, with the benefits being expressed in a uniform way, you can compare projects immediately between business areas, across delivery streams, and in the whole project portfolio.

Another reason for assigning dollars to benefits is to prevent determining the benefits of a project by measuring “who shouts the loudest,” something many organizations suffer from.

And given that most of the costs of a project are already measured in dollars, you can easily compare the costs and benefits of a project to determine it’s overall value.

While estimating a number for costs is generally easy to do, assigning a dollar value to the benefits is more of a challenge. Getting to data and assumptions that make estimating benefits in dollars possible is difficult, but it can be done.

And it pays off: You'll gain a clear idea about what the expected benefits are, and what assumptions you need to test, in a way that all project stakeholders will be able to understand.

Focus on Economic Benefits

Many projects have objectives like “delighting the customer,” “improving time to market,” “happy employees,” or “operational efficiency.” But hardly any project will be blessed with unlimited resources to invest, and there’s just no getting around the fundamental challenge that all organizations face: sustaining themselves.

At some point the projects you invest your limited resources in should help the organization to sustain itself and possibly even grow. So the question becomes “How do these benefits impact the economics of your organization?”.

The risk of placing too much focus on the economic benefits is relatively small and one can argue that lately it's largely been ignored by many organizations (just have a look at the number of Silicon Valley unicorns that are currently imploding). It is of no help having happy customers and no revenue.

The biggest risk of focusing on the economic benefits of projects is when organizations find opportunities to reduce costs that may ultimately have a negative impact on the customer experience. This is particularly problematic for service organizations, where a more efficient operation often translates to less happy customers.

To make estimating the benefits of a project easier and more realistic, I use a simple model to assess the economic benefits of a project. It consists of five benefit types (or buckets).

1) Increased Revenue
2) Protected Revenue
3) Reduced Costs
4) Avoided Costs
5) Positive Impacts

Each project must contribute to one or more of these; if it doesn't, it has no benefits at all.

Increased Revenue

Increased revenue is the revenue associated with either increasing sales to existing customers or gaining new customers. It may involve increasing share of wallet, market share or even the size of the market itself. It can be done by making changes that add value to existing products or services that customers are willing to pay for, or adding new products or services that either existing or new customers are willing to pay for.

The projects in this area are likely to be “delighting” features for either current customers or new ones. This is also where innovation occurs, enabling new business models and increasing the size of the market by serving new markets and undercutting others.

Protected Revenue

Protected revenue is the revenue that is currently being received from existing customers who are paying for the products and services you already sell. Sustaining and protecting this revenue often requires ongoing improvements to at least keep up with competitors and maintain existing market share.

The projects in this area are more defensive in nature, making processes faster and easier to use and removing any pain that might drive customers to consider switching to a competing product or service. The changes made here are usually not considered valuable enough for existing customers to pay extra, though. This is the basic maintenance of existing products and services that can be described as “sustaining innovations.”

Reduced Costs

All those ideas about how to be more efficient contribute to this bucket. The projects that add to this bucket will reduce the costs that you are currently incurring. A typical example of this can be changes that speed up or automate processes, reducing the number of people required. It can also result in savings in overhead, equipment or other costs.

Avoided Costs

Avoided costs are costs you are not currently incurring but there is some likelihood that you will in the future, unless some action is taken. Some examples of these might be the need to hire additional people to handle a new process, fines you may have to pay, or loss of reputation that impacts goodwill or brand value.

This category typically includes a lot of things that many organizations might consider to be operational or strategic risks — often with an estimate of the probability of an event occurring.

Positive Impacts

Positive impacts are benefits that cannot be translated to one of the above but are important enough for your organization to take into account when evaluating a project. For example, creating less pollution and waste, or helping the family of a deceased colleague, or working for free for a non-profit.

When such benefits are important for you and your organization you will give them a disproportionate dollar amount as value. This way projects with such benefits will always be part of the short list of projects to be selected for implementation.

When these benefits are not so important, make the dollar value of the benefits equal to the costs, and projects like these will only be part of the short list if they have additional economic benefits.

And when the importance of these benefits are just lip service just give them a zero dollar value and you will see them almost never in your shortlist.

Make Assumptions Transparent

Once you have identified the benefit types of your project, getting to a dollar figure typically requires some assumptions about the effects of the change or the cost of alternatives. The reality of technology projects is that they involve many risks, unknowns and uncertainties. In order to calculate the dollar benefits, you will often have to make assumptions or educated guesses and apply probability where necessary.

The goal of these estimations and assumptions is accuracy, not precision. Accuracy is how close an estimate is to the true value. Precision is the number of significant figures you give in your estimate. These are two different things.

The initial process of estimating the benefits of projects needs to be fast — less than one week. Over time you will get better information to improve your assumptions or replace them with facts. You can then update your estimations and re-evaluate.

By making assumptions transparent you prevent blatant gaming of the benefits and cost figures. This means all assumptions are part of the document that estimates the benefits. Transparency of assumptions also allows others to scrutinize and improve those assumptions. And better assumptions will lead to more reliable estimations.

In some cases, the same assumptions are used again and again. It can be helpful to use the same numbers for particular project benefits and even start building a repository of assumptions and key data points that are shared and built on over time.

And Now What?

Now that you've arrived at an estimate of your project benefits expressed in dollars, and you already estimated the real costs of your project in dollars you are now ready to determine the real value of your project.  This will be the topic of my next article.

In a nutshell: Estimating the benefits of your project in dollars is essential for making good decisions, comparing projects, and aligning stakeholders.

You can buy my eBook The Project Valuation Model ™ by clicking here or on the image.

Read more…