Monte Carlo Simulation Paradox

Let’s review the Monte Carlo Simulation Paradox, which can be described as:

Monte Carlo Simulation Analysis is very simple.
Monte Carlo Simulation Analysis is extremely difficult.

What is a Monte Carlo Simulation analysis?

A Monte Carlo Simulation analysis is a way to evaluate uncertainty by repeatedly calculating outcomes using random inputs. At its core, it is not a “method” in itself but computational mechanism used to execute other methods under uncertainty.

If we review how the mechanic works, we can find that it is indeed quite simple:

1. Develop a dynamic simulation model
2. Define the parameter of interest
3. Set the number of iterations
4. Recalculate the model multiple times using randomised inputs
5. Aggregate results into probability distributions

In project management, the parameter of interest could be:

• Overall project duration
• Duration of WBS elements
• Activity duration
• Target project delivery date
• Target activity delivery date
• Overall project cost
• Estimation to complete
• IRR
• NPV
• Payback period
• Remaining volume of work
• Resource (demand) quantity
• Total Float (criticality index!)
• Activity Drag (probabilistic Drag!)

Practically, any project parameter that has uncertainty and depends on other parameters can be used for a Monte Carlo Simulation Analysis. It creates an opportunity to apply the mechanic to other methods! A good example is the Criticality index (CI) method.

In practice, the application of Monte Carlo Simulation Analysis depends on multiple factors.

A reliable model must reflect project reality and satisfy two main conditions:

Condition 1: It must produce feasible outcomes
Each individual version of the generated schedule represents how the work may progress in real life.

Condition 2: It must be comprehensive
If the model ignores some of the possibilities, the aggregated result is incomplete and can be misleading.

Depending on the quality and maturity of the simulation model, the outcome may not be accurate. It may be hard to draw clear lines to define when it becomes inaccurate and even misleading, but retrospectively, it is possible to compare the actual result with the prediction to demonstrate that the prediction wasn’t informative.

In project management, Monte Carlo Simulation works the following way:

1. Integrated logically driven Project Delivery Plan developed and assessed
2. People enter three estimates of initial data that are uncertain (optimistic, most probable and pessimistic) and define what probability distribution each uncertain parameter has
3. Risk events are included in the project delivery model with their probabilities and outcomes
4. Corresponding corrective actions added if targets achievements are endangered
5.  The software calculates the model and accompanying parameters over and over, each time using a different set of initial data in accordance with their probability distributions. The number of iterations is usually defined by the user of the risk management software. Usually, this number is measured in thousands of distributions.

British statistician George Box once said, “The question you need to ask is not ‘Is the model true?’ (it never is) but ‘Is the model good enough for this particular application?”

Projects, even small ones, have (technological and resource) limitations, possibilities, complex causations, and human factors (biases, conflicts, politics) that directly impact the outcome. Considering these factors correctly is not an easy task. 

Rephrasing G. Box, project teams must decide: ”Is our Project Delivery model good enough to explain the impact of risks and uncertainties (on the analysed parameter) by applying the Monte Carlo Simulation mechanics correctly?” 

On my blog, I have a series of posts dedicated to lesser-known challenges in developing and evaluating reliable models. The comprehensive list of challenges is long. Here are better-known challenges that demonstrate the difficulties. 

Risk Impact
Project duration risk impact is often analysed for individual risk events, assuming other risks don’t occur, and stored in the risk register. However, in practice, each risk always exists in the context of other risks.

Risk impact is not commutative. A materialised risk in one part of the project may change the resource critical path and risk impact in another part. The impacts must be calculated, not estimated.

Resource constraints
Projects commonly have resource constraints that directly impact the overall result. For a schedule to be feasible, it must be levelled. The levelling includes adjusting activity durations (driven by reassignments) and adding critical resource dependencies.

It means that each autogenerated version must be levelled. Otherwise, we aggregate results of thousands of non-feasible scenarios. Popular MCS systems don’t provide such capabilities.

Theoretical vs Practical levelling
It is also important that the levelling method used in execution is the same as that used during analysis. Otherwise, an applied MCS system may ignore (or recognise) optimisation scenarios that will not be practically applied. Again, it will result in the aggregation of thousands of theoretical plans. As each scheduling system has its own unique levelling methods, external MCS analysis tools should not be applied. The same tool must be used for project delivery and MCS. In practice, scheduling and risk management are often fragmented. It is managed in different systems and often by different teams.

Corrective actions ignored
When the project is late, the team do not silently observe the delay. They apply corrective actions (fast tracking & crashing). It means that a simulation model requires conditional scheduling to correctly set alternative execution paths and simulate acceleration opportunities. Most of the scheduling and MCS systems still don’t support conditional scheduling.

Time vs Cost
Each project requires many time-cost balancing decisions. Additional resources or a delivery path may accelerate delivery but also increase overall cost. In practice, time and cost analysis are often performed in isolation and time-cost balancing decisions are not modelled correctly.

While the mechanics of Monte Carlo Simulation are relatively simple, applying it correctly in project management is a highly complex task.

It is not entirely impossible, but there are many recognised and lesser-known pitfalls that can undermine the reliability of the results. The real challenge lies in building and calculating a credible simulation model.

This leads to the “Monte Carlo Simulation paradox.”

Those who sell tools, training, or services, or simply misuse the method to produce desired outcomes, tend to present it as easy. In contrast, practitioners who rely on it for real project decisions quickly discover its complexity and limitations.

Remember: if you can’t demonstrate that your simulation results are reliable, there is a real risk they are misleading.