## Monte Carlo Simulation Challenges. Simplicity Challenge.

Before we start our review of Monte Carlo Simulation (MCS) Challenges, we have to clarify what MCS is and how it is applied in project management.

## Monte Carlo Simulation Method

A Monte Carlo Simulation is a model used to predict the probability of different outcomes when the intervention of random variables is present. Monte Carlo simulations help to explain the impact of risk and uncertainty in prediction and forecasting models.

To be useful a model has to be dynamic and represent the true relationships between inputs and outputs.

In the project management model, inputs and outputs are:

## Model

Project Delivery Plan used a model for project planning and delivery.

I don’t like the term “schedule” in the context of MCS. For many project practitioners, this term is associated with Time management and it is only one component of project delivery.

## Inputs

Apart from the Golden Triangle, which includes:

• Scope
• Time
• Cost

a project delivery plan also integrates:

• Risks (including uncertainties)
• Resources (People, Equipment & Materials)
• Benefits

Full integration means that changes in one parameter are reflected in other parameters.

## Outputs

A Monte Carlo Simulation is performed to identify:

• Project risks that require maximum attention
• Project targets that can be met with sufficient probabilities
• Contingency reserves that must be created to meet project targets
• Current probabilities of meeting project targets
• Resources required for the reliable achievement of project targets
• Project activities that require maximum attention

MCS is only one of the methods of Quantitative Risk Analysis (QRA). In this and future posts, if not specified, under QRA I mean the MCS method. If there is any interest, we can discuss other QRA methods later.

## Monte Carlo Simulation Process

In project management Monte Carlo Simulation works the following way:

1. Integrated logically driven Project Delivery Plan developed and assessed.
2. People enter three estimates of initial data that are uncertain (optimistic, most probable and pessimistic) and define what probability distribution each uncertain parameter has.
3. Risk events are included in the project risk model with their probabilities and impacts.
4. The software calculates the model and accompanying parameters over and over, each time using a different set of initial data in accordance with their probability distributions. The number of iterations is usually defined by the risk management software user. Usually, this number is measured in thousands of distributions.

As the result, we get the distributions of possible outcome values.

This process has one important missing step: “Add corresponding corrective actions”. We will cover it later in a separate post.

## Simplicity Challenge

Any process may look simple at a high level but to get the desired result detailed instructions, tools and practice are often required. Everyone, who tried to cook knows that cooking is more complex than “Prepare ingredients, mix them, apply temperature processing, and serve”. So, why do we accept the idea that the application of the scientific method in complex and complicated environments could be as simple as: “Prepare model and estimations, add risks and uncertainties, simulate, and report”? There are a few reasons for that:

1. A burned dish can damage a planned dinner and the reputation of the chef, but decisions made based on misleading project risk analysis are likely to impact someone else, not the person who runs the analysis, or even a project manager who is responsible for project delivery.

So many times we have been told that communication skill is the most important skill in project management. Now we have many project managers that could perfectly explain why their project is late and over budget, but don’t know how to develop an optimised project delivery plan and run quantitative risk analysis. Adding even a small financial incentive based on achieved results would motivate project managers to learn advanced project management methods, including MCS

Another problem is a disconnect between time, cost and benefits. Until project delivery plans will not have integrated with planned benefits, this challenge will always be there. Such integration helps to drive project decisions and understand how these decisions impact outcomes and benefits, not just time and cost. MCS also could be used to calculate the distribution of financial (NVP & ROI) and non-financial metrics.

An example of this kind of integration was discussed in this post:

https://saluteenterprises.com.au/project-success-criteria/

2. If companies that promote their MCS tool or QRA training say that MCS requires many hours of effort, no one will buy their tool or service.

There are four levels in the Conscious Competence Learning Matrix and companies that promote the MCS methods can help us with moving from ‘Unconscious Incompetent’ to ‘Conscious Incompetent’, but reaching the next, ‘Conscious Competent’ level will require much more effort to grow deep knowledge of how the MCS method works, how to develop a reliable project delivery plan and which features in risk analysis tools are important.

3. Popularity of QRA is growing. More and more companies and projects include requirements to run QRA periodically and even specify that it must be done with the MCS method. Unfortunately, well too often the QRA is performed only to “tick the box” and the result of the QRA doesn’t drive any project decisions.

There is a simple test to identify misuse of the MCS method. Ask the Project Manager about the current probabilities of meeting project targets, which is one of the outputs of the MCS process.

4. Precision creates an illusion of accurate estimation. MCS is a perfect tool to play a guesstimation game.

A knife could be used to cut cooking ingredients or as a weapon. MCS is also used to justify desired or already made decisions, not to calculate required contingencies and identify critical risks and activities. In that case, it is not critical if the MCS method was applied correctly or not, and which tool was used for the simulation.

The market is full of Monte Carlo Simulation software that primarily serves this purpose. The ability to generate colourful reports seems to be the most critical feature of this tool. If your project or portfolio runs MCS to satisfy someone’s need to say that the analysis is performed or it is performed to justify the desired decision, you may not worry about other MCS challenges.

On the other hand, for business owners and accountable project managers, the ability to run their own independent project quantitative risk analysis based on a reliable model and risk analysis tool can save money, reputation and even lives. ‘Covid-19’ projects are a great example of it.

Ones Niels Bohr, the Nobel laureate in Physics and a father of the atomic model, said: “It is difficult to make predictions, especially about the future”. Any project model is just a proxy and the result of risk analysis never could be precisely accurate. So, why are we still trying to make predictions then? Because even not 100% accurate forecasts still could be used for our decisions! However, misleading forecasts are dangerous, as they drive wrong decisions.

The challenge is to understand where to draw a line between “inaccurate” and “misleading” when MCS analysis is performed.

Probably you have already heard the famous quote of George Box: “All models are wrong, but some models are useful”. I prefer the full version:

All models are wrong, but some models are useful. So, the question you need to ask is not ‘Is the model true?’ (it never is) but ‘Is the model good enough for this particular application?’

In the application of project management, we can rephrase it as:

“Is the Project Delivery Plan good enough to explain the impact of risks and uncertainties by applying the Monte Carlo Simulation method correctly?”

In the series of future posts, we will try to find the answer to this question.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

## Monte Carlo Simulation Challenges

Risk simulation is becoming popular but most risk simulation tools and the ways how they are used miss some important functionalities that make the results of this simulation unreliable.

## Monte Carlo Simulation – Myth or Reality?

Last year, I have organised a poll on LinkedIn to understand what project practitioners think about Monte Carlo Risk Simulation:

The Monte Carlo Simulation Method is the best method for quantitative project risk analysis: Myth or Reality?

The poll had a lot of attention, and many projects and risk consultants shared their opinions.

Based on discussions, comments and the final result we can make some conclusions:

• Monte Carlo Risk Simulation (MCS) is the most recognised quantitative project risk analysis method;
• There is no common acceptance of this method across project practitioners;
• Opinion about Monte Carlo Simulation is mostly based on perception rather than knowledge;
• Majority of planners and risk consultants are mostly not aware of missing critical functionalities in Risk Simulation tools.

The popularity of MCS in recent years is primarily driven by companies that promote their own Monte Carlo Simulation software or Quantitative Risk Analysis (QRA) training. Unfortunately, well too often they measled project practitioners by telling them that it is easy to apply the method to get a reliable result. This is how one leading American consulting company that sells Quantitative Risk Analysis training attracts their clients:

“For many, Quantitative Risk Analysis (QRA) is a complex secretive technique, which relies on smoke, mirrors and mathematical trickery. The aim of this webinar is to draw back the curtain and show that QRAs are not that complex and by learning a few basic steps you can apply QRAs to any project to aid in their successful delivery.”

Demonstration of Monte Carlo principles based on the probability distribution of two dices may be good for a teenager, but projects are much more complex and deep knowledge is required to understand how to apply MCS correctly and which tool could be used to get reliable results.

Based on my research I found that different Monte Carlo Risk Simulation challenges are explained in conference presentations, blogs, White Papers and books but there is no single source where all challenges are collected or explained. I have decided to collect them and present the result at the Project Control Expo conference. 45 minutes is sufficient to cover only some key challenges at a high level. Fortunately, I am not limited by time and space on the Salute Enterprises blog and we could discuss each challenge in detail.

I am going to write a series of “Monte Carlo Simulation Challenges” posts, and discuss them on LinkedIn. If you are aware of any good source that explains such challenges please share them and join discussions on LinkedIn.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma