Monday, November 14, 2022

How Your Rollout in Waves Can End in a Tsunami

How Your Rollout in Waves Can End in a Tsunami

Many multinational organizations are bringing larger system implementations to a screeching halt because they misunderstand what it means to do a rollout in waves. 

We’re probably all familiar with the “phased rollout”. A phased rollout means you roll a project out to all targeted users at once but don’t deploy all of its planned functionality.

A good example of this would be rolling out a new CRM system to your organization. You go live in the first phase with Contact, Client, and Opportunity Management, and Account Management and Pipeline Management follow in the second phase.

Another popular type of rollout is the so-called staged rollout (also known as a rollout in waves). A rollout in waves or stages means that all the planned functionalities will be rolled out at once, but not for all users.

A rollout in waves gives you time to analyze the system’s quality, stability, and performance against your business goals. You can then decide if you want to roll out the system to more users, wait for more data, or stop the rollout. 

A rollout in waves is one of the core building blocks of making continuous delivery a reality. Facebook, Netflix, Microsoft, Google, and similar companies all rely heavily on staged rollouts.

One wave rollout method frequently used by multinational companies for new system implementation is the rollout by country or geographical territory. 

This is the preferred approach by companies implementing a new CRM, ERP, HCM, or some other key business application. Sometimes it’s combined with a phased approach.

Rolling out in waves is usually a good idea, especially compared to a “big bang” rollout. 

But before undertaking a rollout in waves, you have to carefully consider the following three realities:

1) The moment you switch on a new system in one country, you’ll need to address a bunch of Business As Usual (BAU) activities including Release Management, Change Management, New User Training … you name it. Your users will also discover bugs in the system and/or interfaces that weren’t discovered during testing. Many of them will be critical and need to be fixed ASAP. You’ll probably find that performance issues will be more common than not. Some companies call the first few months “Hyper Care” or some equivalent, but it is nothing else as BAU. 

2) As is always the case with a new system, it won’t work completely as expected. In addition to the bugs that need to be addressed within the BAU process, you’ll have a high number of Change Requests, because only now will users realize they need additional or different functionality to do their work. Again, a number of these requests will be critical and/or urgent. Users will probably ask for many additional reports because they don’t understand the data they see in the new system. If you combine your rollout in waves with a phased rollout, you’ll need to build and test the functionalities for the next phase.

3) At the same time, you’ll want to proceed with the next waves of your rollout, and you’ll need people to work on this. Think about discovery, migration, configuration, training, etc. for each new country that needs to be onboarded. The big idea is always to have one system for everyone, but local legislation and regulations and differences in how business is done in each country will force you to implement additional Change Requests in the system.

The number-one mistake I see is that organizations allocate a single team to accomplish all of the above tasks after the first-wave rollout. This approach always fails miserably and will bring the rollout to a screeching halt.

For a successful rollout in waves, you’ll need three different teams after the first wave:  one for BAU activities, one to deliver Change Requests, and one to onboard additional waves. Some people may work on more than one team, but this really should be the exception.

You’ll need to plan and budget for these teams, hire and train people for them, and define their organizational setup. 

And you’ll need to do all of this before you go live with the first wave – not after!

In a nutshell: You will need three teams for a successful system rollout in waves.

Read more…

Saturday, November 12, 2022

My Talk "Why Big Technology Projects Fail" @ Synergy

Why Big Technology Projects Fail@ Synergy
In September I was invited by Synergex to give a talk at their 2022 Synergy DevPartner Conference. 

The title of my talk was "Why Big Technology Projects Fail" and it covers my personal top ten reasons why this happens so often. 

If you want to watch the talk you can do this as Synergex have put the talk online.  

And if you think I might be a good match for speaking at one of your events just have a look at my speaking page

Read more…

Sunday, October 16, 2022

Case Study 16: Nike’s 100 Million Dollar Supply Chain "Speed bump"

Case Study 16 – Nike’s 100 Million Dollar Supply Chain Speed bump

“This is what you get for 400 million, huh?” 

Nike President and CEO Phil Knight famously raised the question in a conference call days before announcing the company would miss its third-quarter earnings by at least 28% due to a glitch in the new supply chain management software. The announcement would then send Nike’s stock down 19.8%. In addition, Dallas-based supply-chain vendor i2 Technologies, which Nike assigned blame, would suffer a 22.4% drop in stock price.

The relationship would ultimately cost Nike an estimated $100 million. Each company blamed the other for the failure, but the damage could have been dramatically reduced if realistic expectations had been set early on and a proper software implementation plan had been put in place. Most companies wouldn’t overcome such a disastrous supply chain glitch or “speed bump,” as Knight would call it, but Nike would recover due to its dominant position in the retail footwear and apparel market.

In 1999, two years before Knight’s famous outburst, Nike paid i2 $10 million to centralize its supply, demand, and collaboration planning system with a total estimated implementation cost of $40 million. Initially, i2 was the first phase of The Nike Supply Chain (NSC) project. The plan was to implement i2 to replace the existing system and introduce enterprise resource planning (ERP) software from SAP and customer relationship management (CRM) software from Siebel Systems.  

The goal of the NSC project was to improve Nike’s existing 9-month product cycle and fractured supply chain. As the brand experienced rapid growth and market dominance in the 1990s, it accumulated 27 separate order management systems around the globe. Each is entirely different from the next and poorly linked to Nike’s headquarters in Beaverton, Oregon.

At the time, there wasn’t a model to follow at the scale Nike required. Competitors like Reebok struggled to find a functional supply chain solution specific to the retail footwear and apparel industry. In an effort to solidify its position as the leader in sportswear, Nike decided to move forward quickly with i2’s predictive demand application and its supply chain planner software.

"Once we got into this, we quickly realized that what we originally thought was going to be a two-to-three-year effort would be more like five to seven," - Roland Wolfram, Nike’s vice president of global operations and technology.

The NCS project would be a success, and Nike would eventually accomplish all its supply chain goals. However, the process took much longer than expected, cost the company an additional $100 million, and could have been avoided had the operators or both companies taken a different approach to implementation.

"I think it will, in the long run, be a competitive advantage." – Phil Knight

In the end, Knight was right, but there are many valuable lessons to learn from the Nike i2 failure.

If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.

If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.

If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.

So, before we get into the case study, let’s look at precisely what happened...

Timeline of Events

1996 - 1999

Nike experienced incredible growth during this period but was at a crossroads. Strategic endorsement deals and groundbreaking marketing campaigns gave the company a clear edge over Adidas and Reebok, its two most substantial competitors in the 80s and 90s. However, as Nike became a world-renowned athletics brand, its supply chain became more complex and challenging to manage.

Part of Nike’s strategy that separated itself from competitors was the centralized approach. Product design, factory contracting, and order fulfillment were coordinated from headquarters in Oregon. The process resulted in some of the most iconic designs and athlete partnerships in sports history. However, manufacturing was much more disoriented.

During the 1970s and 80s, Nike battled to develop and control the emerging Asian sneaker supply chain. Eventually, the brand won the market but struggled to expand because of the nine-month manufacturing cycle.

At the time, there wasn’t an established method to outsource manufacturing from Asia, making the ordering process disorganized and inefficient across the industry. In addition, Nike’s fractured order management system contained tens of millions of product numbers with different business rules and data formats. The brand needed a new way to measure consumer demand and manage purchasing orders, but the state of the legacy system would make implementing new software difficult.

1999

At the beginning of 1999, Nike decided to implement the first stage of its NSC project with the existing system. i2 cost the company $10 million but estimated the entire project would cost upwards of $400 million. The project would be one of the most ambitious supply chain overhauls by a company of Nike’s size. 

i2 Technologies is a Dallas, Texas-based software company specializing in designing solutions that simplify supply and demand chain management while maximizing efficiency and minimizing cost. Before the Nike relationship, i2 was an emerging player in logistics software with year-over-year growth. Involvement in the Nike project would position the company as the leading name in supply chain management software.

Nike’s vision for the i2 phase of NSC was “achieving greater flexibility in planning execution and delivery processes…looking for better forecasting and more profitable order fulfillment." When successfully implemented, the manufacturing cycle would be reduced from nine months the six. This would convert the supply chain to make-to-order rather than make-to-sell, an accomplishment not yet achieved in the footwear and apparel industry.

Predicting demand required inputting historical sales numbers into i2’s software. “Crystal balling” the market had substantial support at the time among SCM companies. While the belief that entering numbers into an algorithm and spitting out a magical prediction didn’t age well, the methodology required reliable, uniform data sets to function.

Nike decided to implement the “Big Bang” ERP approach when switching to i2 for the supply chain management. The method consists of going live where the business completely changes without phasing out the old system. Nike also opted for a single instance strategy for implementation. The CIO at the time, Gordon Steele, is quoted saying, “single instance is a decision, not a discussion.” Typically, global corporations choose a multi-instance ERP solution, using separate instances in various regions or for different product categories.

2000

By June of 2000, various problems with the new system had already become apparent. According to documents filed by Nike and i2 shareholders in class-action suits, the system used different business rules and stored data in various formats, making integration difficult. In addition, the software needed customization beyond the 10-15% limit recommended by i2. Heavy customization slowed down the software. For example, entries were reportedly taking over a minute to be recorded. In addition, the SCM system frequently crashed as it struggled to handle Nike’s tens of millions of product numbers.

The issues persisted but were fixable. Unfortunately, the software was linked to core business processes, specifically factory orders, that sent a ripple effect that would result in over and under-purchasing critical products. The demand planner would also delete ordering data six to eight weeks after it was entered. As a result, planners couldn’t access purchasing orders that had been sent to factories.

Problems in the system caused far too many factory orders for the less popular shoes like the Air Garnett IIIs and not enough popular shoes like the Air Jordan to meet the market's demand. Foot Locker was forced to reduce prices for the Air Garnett to $90 instead of the projected retail price of $140 to move the product. Many shoes were also delivered late due to late production. As a result, Nike had to ship the shoes by plane at $4-$8 a pair compared to sending them across the Pacific by boat for $0.75.   

November 2000

According to Nike, all the problems with i2’s supply chain management system were resolved by the fall. Once the issues were identified, Nike built manual workarounds. For example, programmers had to download data from i2’s demand predictor and reload it into the supply chain planner on a weekly basis. While the software glitches were fixed and orders weren’t being duplicated or disappearing, the damage was done. Sales for the following quarter were dramatically affected by the purchasing order errors resulting in a loss of over $100 million in sales.

2001

Nike made the problem public on February 27, 2001. The company was forced to report quarterly earnings to stakeholders to avoid repercussions from the SEC. As a result, the stock price dove 20%, numerous class-action lawsuits were filed, and Phil Knight famously voiced his opinion on the implementation, "This is what you get for $400 million, huh?"

In the meeting, Nike told shareholders they expected profits from the quarter to decline from around $0.50 a share to about $0.35. In addition, the inventory problems would persist for the next six to nine months as the overproduced products were sold off.

As for the future of NSC, the company, including its CEO and President, expressed optimism. Knight said, "We believe that we have addressed the issues around this implementation and that over the long term, we will achieve significant financial and organizational benefit from our global supply-chain initiative."

A spokeswoman from Nike also assured stakeholders that the problems would be resolved; she said that they were working closely with i2 to solve the problems by creating “some technical and operational workarounds” and that the supply chain software was now stable.

While Nike was positive about the implementation process moving forward, they placed full blame on the SCM software and i2 Technologies.

Nike stopped using i2’s demand-planning software for short-and-medium range sneaker planning; however, it still used the application for short range and its emerging apparel business. By the Spring of 2001, Nike integrated i2 into its more extensive SAP ERP system, focusing more on orders and invoices rather than predictive modeling.

What Went Wrong?

While the failures damaged each company’s reputation in the IT industry, both companies would go on to recover from the poorly executed software implementation. Each side has assigned blame outward, but after reviewing all the events, it's safe to say each had a role in the breakdown of the supply chain management system.

Underestimating Complexity

Implementing software at this scale always has risks. Tom Harwick, Gigi Information Group’s research director for supply chain management, said, “Implementing a supply-chain management solution is like crossing a street, high risk if you don't look both ways, but if you do it right, low risk.”

One of Nike's most significant mistakes was underestimating the complexity of implementing software at such a large scale. According to Roland Wolfram, Nike’s operators had a false sense of security regarding the i2 installation because it was small compared to the larger NSC project. "This felt like something we could do a little easier since it wasn’t changing everything else [in the business]," he says. "But it turned out it was very complicated."

Part of the reason why the project was so complicated was because of Nike’s fractured legacy supply chain system and disoriented data sets. i2’s software wasn’t designed for the footwear and apparel industry, let alone Nike’s unique position in the market.  

Data Quality

Execution by both parties was also to blame. i2 Technologies is on record recommending customization not to exceed 10-15%. Nike and i2 should have recognized early on that this range would be impossible to accommodate the existing SCM system.

Choosing a Big Bang implementation strategy didn’t make sense in this scenario. Nike’s legacy system data was too disorganized to be integrated into the i2 without making dramatic changes before a full-on launch.

Poor Communication

Communication between Nike and i2 from 1999 to the summer of 2000 was poor. i2 claimed not to be aware of problems until Knight issued blame publicly. Greg Brady, the President of i2 Technologies who was directly involved with the project, reacted to the finger-pointing by saying, "If our deployment was creating a business problem for them, why were we never informed?" Brady also claimed, "There is no way that software is responsible for Nike's earnings problem." i2 blamed Nike’s failure to follow the customization limitations, which was caused by the link to Nike’s bake-end.

Rush to Market

At the time, Nike was on the verge of solidifying its position as the leader in footwear and sports apparel for decades to come. Building a solid supply chain that could adapt to market trends and reduce the manufacturing cycle was the last step toward complete market dominance. In addition, the existing supply chain solutions built for the footwear and apparel industry weren’t ready to deploy on a large scale. This gave Nike the opportunity to develop its own SCM system putting the company years ahead of competitors. Implementing functional demand-planning software would be highly valuable for Nike and its retail clients.

i2 also was experiencing market pressure to deploy a major project. Had the implementation gone smoothly, i2 would have a massive competitive advantage. The desire to please Nike likely played a factor in i2’s missteps. Failing to provide clear expectations and communication throughout the process may not have happened with a less prominent client.  

Failure to Train

After the problems became apparent in the summer of 2000, Nike had to hire consultants to create workarounds to make the SCM system operational. This clearly indicates that Nike’s internal team wasn’t trained adequately to handle the complexity of the new ERP software.

Nike’s CIO at the time reflected on the situation. "Could we have taken more time with the rollout?" he asked. "Probably. Could we have done a better job with software quality? Sure. Could the planners have been better prepared to use the system before it went live? You can never train enough."

How Nike Could Have Done Things Differently

While Nike and i2 attempted to implement software that had never been successfully deployed in the global footwear and apparel industry, many problems could have been avoided. We can learn from the mistakes and how Nike overcame their challenges with i2 to build a functioning ERP system.

Understanding and Managing Complexity

Nike’s failure to assess the complexity of the problem is at the root of the situation. Regardless if the i2 implementation was just the beginning of a larger project, it featured a significant transition from the legacy system. Nike’s leadership should have realized the scale of the project and the importance of starting NSC off on the right foot.  

i2 also is to blame for not providing its client with realistic expectations. As a software vendor, i2 is responsible for providing its client with clear limitations and the potential risks of failing to deploy successfully.

See "Understanding and Managing Your Project’s Complexity" for more insights on this topic.

Collaborate with i2 Technologies

Both companies should have realized that Nike required more than 10-15% customization. Working together during the implementation process could have prevented the ordering issues that were the reason for the lost revenue.

Collaboration before deployment and at the early stages of implementation is critical when integrating a new system with fractured data. Nike and i2 should have coordinated throughout the process to ensure a smooth rollout; instead, both parties executed poor project management resulting in significant financial and reputational blows.  

See "Solving Your Between Problems" for more insights on this topic.

Hire a 3rd Party Integration Company

Nike’s lack of understanding of the complexity of SCM implementation is difficult to understand. If i2 had been truthful in that they did not know about problems with their software, Nike could have made a coordinated decision not to involve the software company during the process.

Assuming that is the case, Nike should have hired a 3rd party to help with the integration process. Unfortunately, Nike’s internal team was not ready for the project. Outside integrators could have prevented the problems before the damage was done.

Not seeking outside help may be the most significant aspect of Nike’s failure to implement a new SCM system.   

See "Be a Responsible Buyer of Technology" for more insights on this topic.

Deploy in Stages

A “Big Bang” implementation strategy was a massive mistake by Nike. While i2 should have made it clear this was not the logical path considering the capabilities of their software and Nike’s legacy system, this was Nike’s decision.

Ego, rush to market, or failure to understand the complexities of the project could all have been a factor in the decision. Lee Geishecker, a Gartner analyst, stated that Nike chose to go live a little over a year after starting the project, while projects of this scale should take two years before deployment. In addition, the system should be rolled out in stages, not all at once.

Brent Thrill, an analyst at Credit Suisse First Boston, is on record saying he would have kept the old system running for three years while testing i2’s software. In another analysis, Larry Lapide commented on the i2 project by saying, "Whenever you put software in, you don't go big bang, and you don't go into production right away. Usually, you get these bugs worked out . . . before it goes live across the whole business."

Train Employees Sufficiently

At the time, Nike’s planners weren’t prepared for the project. While we will never know what would have happened if the team had been adequately trained, proper preparation would have put Nike in a much better position to handle the glitches and required customizations.

See "User Enablement is Critical for Project Success" for more insights on this topic.

Practice Patience in Software Implementation

At the time, a software glitch causing a ripple effect that would impact the entire supply chain was a novel idea. Nike likely made their decisions to risk the “Big Bang” strategy, deploy in a year without phases and proper testing, and not seek outside help because they assumed the repercussions of a glitch wouldn’t be as catastrophic.

Impatience resulted in avoidable errors. A more conservative implementation strategy with adequate testing would have likely caught the mistakes.

See "Going Live Too Early Can Be Worse as Going Late" for more insights on this topic.

Closing Thoughts

One of the most incredible aspects of Nike’s implementation failure is how quickly the company bounced back. While Nike undoubtedly made numerous mistakes during the process, NSC was 80% operational in 2004.

Nike turned the project around by making adjustments and learning patience. Few companies can suffer a $100 million “speed bump” without filing bankruptcy, but Nike is in that position because of its resilience. The SAP installation wasn’t rushed and resumed many aspects of its original strategy. In addition, a training culture was established due to the i2 failures. Customer service representatives receive 140 to 180 hours of training from highly skilled “super users,” All employees are locked out of the system until they complete their required training courses.

Aside from the $100 million loss, the NSC project was successful. Lead times were reduced from nine months to six (the initial goal), and Nike’s factory inventory levels were reduced from a month to a week in some cases. Implementing a new SCM system also created an integration between departments, better visibility of customer orders, and increased gross margins.

While Nike could have executed far more efficiently, Phil Knight’s early assessment of the i2 failure turned out to be true. In the long run, the process gave Nike a competitive advantage and was instrumental in building an effective SCM system. 

In a nutshell: A failure to demonstrate patience, seek outside help, and rush software implementation can have drastic consequences. 

If you want to make sure your business critical project is off to a great start instead of on its way on my list with project failures? Then a New Project Audit is what you are looking for.

If you want to know where you are standing with that large, multi-year, strategic project? Or you think one of your key projects is in trouble? Then a Project Review is what you are looking for.

If you just want to read more project failure case studies? Then have a look at the overview of all case studies I have written here.

Sources

> Nike says i2 hurt its profits

> I2 Technologies, Inc.

> How Not to Spend $400 Million

> i2-Nike fallout a cautionary tale

> Nike rebounds: How Nike recovered from its supply chain disaster

> Scm and Erp Software Implementation at Nike – from Failure to Success 

> I2 Says: "You Too, Nike"

Read more…

Saturday, October 15, 2022

White Elephant Stampede: Case Studies in Policy and Project Management Failures

White Elephant Stampede: Case Studies in Policy and Project Management Failures

This month one of the book projects I have been part of has been published by Connor Court Publishing in Australia. 

The book is titled "White Elephant Stampede: Case Studies in Policy and Project Management Failures", and it examines the seemingly endless cavalcade of projects that fail to meet their objectives, cost more than expected and become an ongoing drain on public resources. 

From public awareness campaign disasters to payroll system technology collapses, from poorly developed environmental projects to the over-optimistic benefits of proposed Olympic Games, these case studies provide lessons as to why White Elephant projects occur and how they might be avoided.

It is of value to every decision maker and to every concerned citizen. Project Managers, Senior Public Servants, Politicians, Academics and Students will gain valuable insights from this volume.

White Elephant Stampede draws on the research, industry experience, and expertise of an outstanding range of contributors: Gary Banks, Simon Dawkins, Brian Dollery, David Gration, Paul Hooper, Binoy Kampmark, Aynsley Kellow, Bruce Kingston, Justin Macdonnell, Scott Prasser, Nadeem Samnakay, Steven Schwarz and myself.

I contributed the case study on the payroll system that costed Queensland Health AU$1.25 billion. For an overview of all case studies in the book and some background on the contributors see the outline here

The book was created and edited by David Gration, Bruce Kingston, and Scott Prasser. 

Scott, who is a well-known local public policy commentator, was recently a guest on the Economics Explored podcast where he talks about these White Elephant projects. You can listen to the conversation with Scott on podcasting platforms like Apple Podcasts and Google Podcasts.

You can buy the book here

I am not making any money of it. The reason I joined this project is to spread awareness about how much public and private money is wasted on technology projects, and show better ways of defining, selecting and executing such projects.

Read more…

Sunday, August 14, 2022

Doing Something That’s Never Been Done

A Great Leading Indicator for Future Trouble - Missing Milestones

Executives, project sponsors, project managers, and steering committee members can learn a lot from how some deep technology startups approach their projects.

This isn’t true for all kinds of projects, but it is for every project that involves doing something that hasn’t been done before and has a high risk-reward profile.

This isn’t another lean startup story. Nor does it involve agile development or digital transformation. This is a story about doing technology research and development with clear goals and constraints. 

I’ve been an active angel investor for several years now. As such, I invest my own money in very early-stage startups. I try to help them with my knowledge, experience, and network, as well as my money. 

3D2cut is one of the startups I’ve invested in, and I sit on its advisory board. 3D2cut is all about winemaking. The company uses modern technology to tackle several problems associated with grapevine pruning.

The Challenge

Like winemaking, grapevine pruning is part science, part art, and getting it right determines the size of the harvest and the quality of the wine. 

In pruning, last year’s canes, which are now surrounded by a layer of wood, are removed to facilitate fresh new growth.

The work is laborious and expensive. There are around 7.5 M hectares of grapevines worldwide. Each year, approximately 1.3 M people prune for multiple weeks during the winter season.

Bad or imprecise pruning leads to wood diseases and can kill vines, which reduces yields and grape quality.

Therefore, as a vineyard owner or manager, you need well-trained staff to prune. 

However, training your staff to prune well takes a lot of resources, and the nature of the work – it’s hard! – means most pruning teams have a big turnover. Training staff is a recurring and expensive task.

Besides the cost, finding the required expertise to train pruners is a challenge.

Industrialization, extensive mechanical management of vineyards, and climate change make pruning far more complicated as your plants become more sensitive and crave for more gentle treatment.

That’s why you need an expert to correctly train your staff.

3D2cut has come up with a solution to address these challenges using modern technology. With this solution, everybody can become a Master Pruner without any training. The product guides vineyard staff through every cut.

The Solution

The solution is simple. 3D2cut will supply you with a device that uses Artificial Intelligence and Augmented Reality technologies to advise pruners on where to cut, simply and quickly.

This pruning expertise isn’t just any old advice. It represents the codified knowledge of the world’s premier pruning experts: Simonit & Sirch.

3D2cut’s solution results in:

> Better grapevine health, increased yields, and less need for chemicals.

> Less money and time spent on staff training, and reduced impacts arising from staff turnover.

> Rekindled passion for vineyard management thanks to a simplified, less stressful pruning process that’s also more sustainable.

Building It

The idea is simple, but that doesn't mean it was easy to achieve. We started by breaking down the problem and writing down its key assumptions. 

What did we need to be certain about to continue supporting the project?

> That the device could support augmented reality and work in the field. Rain, sun, snow, dust, long days. 

> That Simonit & Sirch’s knowledge could be codified in algorithms. We call this the Pruning Expert System (PES).

> That the device could recognize a vine in its natural environment and identify the vine type and its constituent parts (cane, cordon, shoot, bud, etc.) through machine learning. We call this the Vine Vision System (VVS).

> That we could create a large set of annotated images of vines to train and test the PES and VVS.

> That we could apply the PES to the results of VVS and project proposed cuts to the vine via the augmented reality device.

> That we could make this all work and produce the cutting proposals within 2 seconds.

> That we could patent the solution.

We then started testing these assumptions one by one.

After we validated the individual parts, we started building the walking skeleton and plugged in the parts. Next, we started field testing and improving the end-to-end solution.

This is where we are now. 

We have proven every assumption to be true, but we are not completely satisfied with the end-to-end product.

That said, we expect to have the first version of the product ready for a pilot client in December this year. Just in time for the northern hemisphere pruning season.

The Lessons

Start by formulating your key assumptions. Test them one by one. This process won’t tell you if the project is worth continuing, as many other factors will come into play, but it can definitely tell you if you need to abandon your project immediately. This step is excellent risk management. 

Proving individual assumptions is a sign of progress and supports discussions around funding. It also helps win support for your project from company stakeholders.

> See if you can create a win-win partnership with a company or team that complements your project team’s capabilities and assets. In our case, Simonit & Sirch are co-founders of 3D2cut. This alliance arms us with their unique expertise and gives them a vested interest in 3D2cut’s success. 

See if you can apply knowledge and working solutions from unrelated areas. In our case, we applied algorithms for detecting human movement to detecting vine structures. It worked very well and saved us a lot of effort. It also allowed us to leverage the experience and knowledge of the people who invented these algorithms. Instead of starting from scratch, we could build on a strong foundation.

If possible, collaborate with experts – universities or research institutes, for example – on certain parts of the problem where there is a common interest. We work together with IDIAP. One IDIAP employee works on problems formulated by 3D2cut; meanwhile, a 3D2cut employee is completing his Master’s degree at IDIAP.  

Build for, and test in the field. Don’t create artificial problems to test your assumptions. Make it as real as possible. Only then will you be testing the assumptions. We started working on our VVS by using vine images with an artificial background. We learned a lot, but it wasn’t the problem we had to solve. Vines in vineyards do not have an artificial background. We spent time and money on annotating these images with artificial backgrounds. In hindsight, we spent too much on these efforts.

Most startups don’t have much money to spend. You’re forced to focus and be creative. You can make that happen in your project by applying stage-gate funding, which is when you release more budget for the project only if certain milestones are achieved. This is comparable with the startup funding concept. It starts with the 3F (friends, family, and fools), then a Seed round, then a Series A, Series B, and so on. You should spend significant money to scale, not necessarily to build.

You can validate assumptions whilst knowing that another assumption is still outstanding or that the result is not yet what you need. For example, we started building PES and VVS and tested them in the field on a tablet instead of glasses that support augmented reality. By not waiting for such glasses, we bought ourselves a lot of time and discovered new requirements for the glasses.

Even if technology isn’t ready today, you can anticipate what’s coming in the near future. Based on our field tests with the tablet, we realized that the first version of the product would need both a tablet/cellphone and glasses to work. We know that within the foreseeable future that glasses alone will fulfil our needs, but today this is not the case.

Smart and motivated people learn very fast. You don’t need experts in every area. You need a small team with people that are capable and willing to learn. Since you don’t know upfront what problems you’ll encounter and what solutions will be available, it makes no sense to look for a bunch of experts. You can always hire an expert for a few consulting days if necessary.

Start building a walking skeleton as early as possible. You’ll need to work and test it in an end-to-end environment as soon as you can. Only then will you see the real results of your changes.

In a nutshell: Corporates can learn a lot from how some deep technology startups approach their projects.

Read more…

Sunday, August 07, 2022

Why Are Your Best People Not Working on This?

User Enablement is Critical for Project Success
A phenomenon I see happening again and again in larger organizations is destroying employee morale and economic value.

There’s that big internal project (think ERP, CRM, HCM/Payroll, Core Banking/Insurance) that has the potential to stop the whole organization in its tracks and will cost the organization millions of hard-earned cash to implement.

And somehow, the best people in the organization aren’t involved in the project. 

How stupid is that?

Depending on your margins, you may have to earn many more millions to pay for the project. So, if you can prevent costs from doubling and extract some benefits from it, this would seem a smart investment, right?

Profit margins vary considerably by industry, but as a general rule of thumb, a 10% net profit margin is considered average, a 20% margin is considered high (or “good”), and a 5% margin is low.

Let’s stick with the middle and take 10%. This means that for every million you spend on your internal project, you have to generate 10 million in additional revenue to make up for it.

Considering that most technology projects will cost twice what is budgeted – without taking losses in revenue and productivity into account – I think it’s worthwhile to have your best people spend some time on the project.

But you know what?

Nobody wants that. Not your executives. Not your line managers. And not your staff. 

Why is that? 

The answer is simple. Nobody makes a career with such a project. You can only lose.

If you’re staffed on the project, nine out of ten times you’ll still have to do your regular job as well. This means double the work and more stress. Your goals, bonus, and promotion probabilities are tied to your normal job, not to the project. But you have less time to achieve them. This is not conducive to career advancement.

Your line managers can try to fill the gaps in their teams but they will have additional work because their people are working on the project. This will affect their goals, chances for promotion, and bonus payments. So they will do anything to prevent their best people working on that critical project.

And you, as an executive involved in the project, can only work on spending and damage control, as everybody expects the project will be a success and within budget. The probabilities that one or the other will happen are very small. And the chances of them both happening at the same time are close to zero. 

And when you’re not a COO, CFO, or CIO, you’re expected to work on client-facing tasks, not internal projects, even when you and your teams will have to work every day with the result of that project. 

There’s a massive misalignment of incentives here. 

You have to fix this misalignment, because with critical projects it is very simple: If you want to have a higher probability of success, you need to involve your best people.

If possible, put them on it full time, and relieve them from their day jobs. It should be made clear across the organization that the project has priority and that day-to-day things may take a little longer.

Bonus payments and possible promotions should be tied to the project goals. Career goals and project goals should be aligned. If this doesn’t happen, your best people will focus on the things that improve their careers, which may not always be in your organization’s best interests.

In a nutshell: If you want to have a higher probability of success with your critical internal project, involve your best people.

Read more…

Thursday, July 07, 2022

Don't Forget SaaS Performance Testing!

Your Strategic Sweet Spot

I wrote about application performance a few years ago in my article “It's Never Too Early to Think About Performance”. Since then I have learned and witnessed a few things that warrant a new article on this topic.

As more and more companies leverage SaaS for their business applications the number of companies that suffer from serious performance problems is growing by the day,

SaaS solutions like for example Salesforce, Workday and S/4HANA can be implemented in a way that performance is decent. They can also be implemented in a way that your users will wait for a very long time after each click or keystroke they make. The latter will lead to much frustration and high opportunity costs. 

How is that? 

Well, if the solution is deemed too slow, people will find ways to not use the solution unless they absolutely have to. This means adoption will be terrible, and your original business case will be not worth the paper it was printed on. As I have written before; any system is only as good as how well it is used. 

Still many projects that implement such SaaS solutions ignore performance and do not include client side performance testing into their scope of work. And this is a very big mistake.

Companies assume that the SaaS vendor is responsible for delivering a fast enough user experience. But this is not the case. They can’t. Vendors can only be responsible for the things they have under control.

Your devices, your security stack, your browser, your internet connection, your network, your location. They all have a significant impact on the performance of your SaaS solution. None of them is under the control of your SaaS vendors.

Also your configurations, your customisations, your number of concurrent users, and your amount of data have a significant impact on the performance of the solution.

Trust me, running a revenue recognition run in SAP S/4HANA with 100 or 100.000 WBS elements makes a big difference. As there is a big difference between running a report on 50 or 50.000 opportunities in Salesforce.

And these business applications cannot live in isolation. They need to be talking with each other. They need cloud integrations. See “The Biggest Challenge of Postmodern ERP - Cloud Integrations” for some additional thoughts on this topic.

Each of these cloud integrations need to be tested individually, but even more important are the end-to-end throughput times. How long does it take before your new employee in Workday shows up in your Oracle ERP? What if you have 1000 new employees?

Most of these cloud integrations are not part of a standard SaaS solution, and therefore not under control of your SaaS vendor.

So before you even think about going live with any SaaS solution you should do the following tests:

> Load Tests: you need to understand the behaviour of the system under a specific expected load. Testing with 250 invoices does not make sense when you know you will have 250.000 invoices in the solution within a year. When you know you have 5.000 concurrent users working on the system every day you should test this before you go live. Not after… 

> Stress Tests: you need to understand the upper limits of capacity within the system. And if this capacity is lower than the capacity you expect you need then you need to deal with this before you go live. Not after…

> Soak Tests: you need to determine if the system can sustain the continuous expected load. This can be as simple as running your load tests for a number of consecutive days. 

> Spike Tests: you need to determine if the system can sustain a suddenly increasing load generated by a large number of users and/or new data in the system. Businesses are mostly seasonal and have spike periods. For example your Year End Closing in SAP/Oracle or Black Friday for your sales. You will need to know if your solution can handle this before you have to do the work. Not after…

Fixing performance issues whilst you are working with the solutions is much harder as fixing them before you go live. See Going Live Too Early Can Be Worse as Going Late for some additional thoughts on going live checks.

In a nutshell: make performance testing mandatory for every SaaS implementation project and define decent performance as a killer criterium before going live.

Read more…