Realistic Project Delivery KPI

How often have we been told that projects are late and over budget and we need to do something about it?

At the same time, it is an absolute lack of understanding that it could be a sign of project delivery maturity rather than a real issue! The real issue is when ALL projects are on time and budget.

In this post, we will discuss this challenge from different perspectives and try to answer the fundamental question: Do we want all our projects to be on time and budget, or not?

Many portfolio managers, sponsors, and project managers want their projects always to meet committed dates and budgets. An ideal scenario is when ALL projects are delivered on time and within budget, as their performance is often evaluated based on these criteria.

However, what is ideal for individual performance is not necessarily an ideal scenario for a portfolio. Let’s go deeper to a project level to understand why.

Each project ALWAYS has uncertainties and risks. So, it is not possible to predict project duration and cost precisely. It always ranges. A range for project durations and a range for project costs. Something like this famous hat:

Boundaries of these ranges may be unclear for project stakeholders, but they always exist.

There are many possibilities of how time and cost targets could be combined.

Someone (let’s say a “Sponsor”) decides which of all combinations will define project targets. The committed delivery date defines the target for the duration, and the committed project budget defines the target for the cost.

Different targets that include different types of contingencies may exist, but to keep it simple, assume there is one key set of targets. Delivery beyond these targets is going to be considered a failure.

Some sponsors choose very aggressive targets with low probabilities to meet them; others prefer a conservative approach with a higher probability of success.

When a sponsor doesn’t clearly understand ranges and distribution of possibilities (or are misled by low-quality risk analysis reports), they may assign a target outside the range. Yes, some projects are planned to fail.

If all projects in a portfolio constantly meet targets, this could mean one of two things:

• Project managers have magic wands;
• It was too easy to hit the targets;

Why could not all projects be just managed well and delivered as expected? Because Black Swan events exist.

A Black Swan event is unpredictable (Unknown Unknowns), and you won’t find a related risk in the project risk register. Also, there are predictable (Known Unknowns), unavoidable low-probability / high-impact risks that could significantly impact project delivery. If such risks materialised and the project still meets the targets, it means that this project had a sufficient contingency to cover such risk, which could be the real problem!

From the “Parkinson’s Law”, we know that work expands to fill the time available for its completion. So, projects with significant contingencies (to cover “Black Swan” events) will likely take longer and be more expensive, even if the risks don’t materialise.

Mature and non-mature portfolios have different approaches to setting up projects and controlling project delivery.

Organisations with a non-mature project delivery don’t plan projects properly or apply deterministic, critical-path-based planning. Executives assign project targets without visibility of possibilities. Their projects take longer and spend more (sometimes substantially more) budgets. However, Portfolio managers rarely understand the actual delivery rate and often have an illusion of good performance.

Once executives of one of my clients decided to increase the project delivery rate from 90% to 95%, their Project Delivery KPI report showed that they were constantly above the 90% threshold. In reality, the actual measures discovered that they were below 40%.

Organisations with the mature project and portfolio management apply probabilistic planning and set the targets based on calculated probabilities. They accurately monitor how the probabilities change during the project’s progress. Their project delivered faster and with less budget, but it is still below the 100% mark. The 80% of on-time & budget projects is usually a good result.

A project portfolio delivery methodology that includes probabilistic methods is SDPM (Success Driven Project Management). SDPM has methods for:

• full integration of scope, time, cost and risks
• development of probabilistic project delivery plan and schedule
• set and monitor probabilities of success.

Organisations that implemented SDPM understand how changes in project delivery and a risk profile impact probabilities to meet agreed targets. They control the contingency consumption.

• A project with a contingency that covers low-probability / high-impact risks and ‘Black Swan” events is likely to meet planned targets. However, the project is also likely to spend available contingency even if the risks have been avoided (Parkinson low).
• When all projects in a portfolio always meet targets, it is a sign that project KPIs don’t reflect reality, and the genuine delivery rate could be much lower.
• SDPM methods focus on building optimised probabilistic project delivery plans, setting up the achievable project and portfolio targets, and controlling probability changes.