The standard probability axioms are the foundations of probability theory introduced by Soviet Union mathematician Andrey Kolmogorov almost a century ago. These axioms remain central and have direct contributions to mathematics, the physical sciences, and real-world probability cases, except project management.

How does it happen that some of the fundamental principles of risk analysis are ignored in project management, and how does it impact the result of the Monte Carlo Simulation analysis?

Probability axioms

We assign a probability measure P(A) to an event A. This is a value between 0 and 1 that shows how likely the event is. If P(A) is close to 0, it is very unlikely that the event A occurs. On the other hand, if P(A) is close to 1, A is very likely to occur. The main subject of probability theory is to develop tools and techniques to calculate probabilities of different events. Probability theory is based on probability axioms that act as the foundation for the theory.

Axiom 1: For any event A, P(A)≥0.
Axiom 2: Probability of the sample space S is P(S)=1.
Axiom 3: If A1,A2,A3,⋯ are disjoint events, then P(A1∪A2∪A3⋯)=P(A1)+P(A2)+P(A3)+⋯

Translating Kolmogorov’s probability axioms in plain language we can say that:
• Probability of risk event is nonnegative.
• Probability of all possible risk outcomes is 100%.
• For every collection of mutually exclusive risk outcomes, the probability of their union is the sum of the individual probabilities.

Project risk management practices and popular risk simulation tools ignore the third probability axiom, making the Monte Carlo Simulation results unreliable.

There are two main reasons:
• the way project delivery models are created;
• the way risk events are analysed, probability and consequences are estimated.

Risk Analysis Challenge

According to the third axiom, a risk event may have multiple mutually exclusive outcomes. Each has a probability of occurring, and together, they have 100% likelihood. 

Traditionally, in project management, risk events are analysed with only two outcomes: the planned scenario vs all other possibilities. If a risk event has multiple scenarios, the impact of one of them (usually the worst case) is analysed. However, the probability of the analysed scenario is not sum of all ‘other scenarios’.

Example:
You can win or lose by throwing a dice. If it is 1 or 2, you win 100$. If it is 6, you lose 150$. Otherwise, draw.

In project management, such a risk event is likely to be estimated with 67% probability with a consequence minus 150$ (as the worst scenario).

Instead, it has to be:
• 33% +100$
• 50% no impact
• 16% -150$

Widely applied two-dimensional risk registers don’t allow to store of multiple probabilities outcomes in the one risk record. Such thinking creates problems with Risk Matrix and other risk management methods, not just Monte Carlo Simulation analysis.

Critical Path Method (CPM) challenge

Even if a project can evaluate probabilities correctly and find a way to store risk data, they may not be able to apply it to Monte Carlo Simulation Analysis.

The uniqueness of project risk management as it exists today is in the typical approach of creating a deterministic project delivery model and using it as the base for planning, analysis, and control. It simplifies (or oversimplifies?) project complexity and opens an opportunity to calculate planning and delivery metrics. A few to mention: Total Float, Earned Value, Team velocity, etc.

Implementing the third axiom requires ‘AND’ & ‘OR’ conditions, but applied deterministic models are based on ‘AND’ logic only. There is no way to implement ‘it depends on’ scenarios in CPM, CCPM or Scrum delivery models.

Why is this a problem?
Let’s review the following use case:

A project requires testing. It is impossible accurately predict how many testing cycles are needed. It could be one, two or three test cycles with the following probabilities:

One testing cycle: 30%
Two testing cycles: 60%
Three testing cycles: 10%

In deterministic CPM-based modelling, one of the scenarios must be used as the base, and other options are considered to be a risk event. Depending on which scenario is chosen as the base, the risk could be a threat, opportunity, or both:

• If the base scenario is ‘one testing cycle,’ it can only get worse. So, the risk event is a THREAT.

• If ‘three testing cycles’ is the base, it can only be better. The event is an OPPORTUNITY.

• However, if ‘two testing cycles’ is the base, it can be better or worse, so the event is threat and opportunity. It is a THROPPOTUNITY.

It is quite possible that the project team chose ‘two testing cycles’ be the base scenario for the CPM modelling. In that case, correct risk modelling may not be possible, as risk modelling tools only classify risk events as threats or opportunities.

I believe such tools don’t comply with ISO 31073:2022 Risk Management. ISO 31073:2022 does not recommend a term to describe a risk with positive and negative outcomes, but it states that such an event is possible.

Risk – effect of uncertainty on objectives.

Note 1 to entry: An effect is a deviation from the expected. It can be positive, negative or both, and can address, create or result opportunities and threats.

The result of the Monte Carlo Simulation analysis should not depend on which scenario is chosen as the base.

However, the fact that a base is required impacts how a dynamic delivery model is developed.

Let’s review possible options.

Option 1: ‘One testing cycle’ is the base.

In this case, the risk event has 70% (60% + 10%) probability with two possible outcomes:
2 cycles: 86%
3 cycles: 14%

Scenario probabilities:
1 cycle = 30%
2 cycles = 86% * 70% = 60%
3 cycles = 14% * 70% = 10%

Option 2: ‘Two testing cycles’ is the base.

In this case, the risk event has 40% probability with two possible outcomes:
1 cycle: 75%
3 cycles: 25%

Scenario probabilities:
1 cycle = 75% * 40% = 30%
2 cycles = 60%
3 cycles = 25% * 40% = 10%

Option 3: ‘Three testing cycles’ is the base.

In this case, the risk event has 90% probability with two possible outcomes:
1 cycle: 33%
2 cycles: 67%

Scenario probabilities:
1 cycle = 33% * 90% = 30%
2 cycles = 67% * 90% = 60%
3 cycles = 10%

Theoretically, if a project delivery model is developed correctly, the Monte Carlo Simulation analysis result will be the same regardless of which scenario was chosen as the base. Practically, such analysis is impossible as popular planning and risk management tools have two gaps:
• Planning tools don’t support ‘OR’ logic.
• Risk events have ONE outcome only.

Solutions

To be able to fully extend probability analysis to project management changes in project management methods, trainings and delivery tools are essential.

People: Theory of probability analysis need to be included in project risk management standards and trainings.

Processes: GERT (Graphical Evaluation and Review Technique) model based on conditional scheduling has to replace CPM models.

Tools: It is essential that project risks are fully embedded in project delivery model. For that risk registers and Monte Carlo Simulation Analysis must part of planning and delivery tools, not to be performed in standalone risk management systems.

Let’s back to the above use case and review possible solution.

There are multiple ways how dynamic delivery model  can be developed. This is one of them: delivery schedule includes all three scenarios and depending of team preference any of them could be the base. It may impact CPM metrics but it would not impact result of Monte Carlo Simulation.

One Test Cycle

Two Test Cycles

Three Test Cycles

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma