Monte Carlo Simulation Challenges. Distribution Shapes Challenge.

Risk simulation systems offer a choice of user-selected distribution shapes that can be implemented for each and every activity. Some tools offer only primary types, others have a large menu of over 50 distributions.
However, selecting a specific shape for each of 1,000+ activities (on top of generating three estimates for each item) requires many hours invested. The majority of users run the system using one of the distributions applied to all activities with uncertainties. It is mostly Triangular, Beta or Log-normal distributions.

What they may not be aware of is that different activity distribution shapes can substantially alter the result of the analysis. The result of the simulation performed with beta (PERT) and Triangular distributions at the P80 mark is likely to have a noticeable 8-15% difference. At the same time application of the same distributions at the different P-point may have a less noticeable difference.

Some construction and consulting companies may use it to get extra funds (or time).
All they need to do is just to run the risk analysis with another distribution type!
Receivers of project risk simulation reports need to review which distribution shape was applied and they need to understand how it may alter the presented result. As a minimum, there must be consistency between reports produced over time.

When projects don’t have subjective historical data and only a subjective range (2-point estimation) is available, it makes sense to calculate the Most Likely estimation as an average between the optimistic and pessimistic estimations: ML=(O+P)/2 and apply the triangular distribution.

However, if there is a reason to believe that the most likely estimation is different, the asymmetrical triangular distributions would be the choice.

Another advantage of the triangular distributions is that result of the Monte Carlo simulation would be the same regardless of the applied risk simulation tool, with an assumption that both tools don’t have other issues which we are going to discuss in future posts.

The disadvantage of the triangular distribution is that often the pessimistic and optimistic estimations represent rare extremes and it has to be reflected in the shape. In this case, projects need alternative shapes that could reflect this anomaly. The most known are the beta and Log-normal distributions.

The famous PERT method also based on Three-point estimations is able to address “extreme ends” anomalies. The PERT addresses it by taking More likely estimation 4 time often than Optimistic and Pessimistic estimations: T = (O + (4* ML) + P) / 6.
A beta distribution is often considered the best choice for project simulations. While it sounds very scientific, each risk management tool uses its own formula for beta-distribution! In fact, it is actually a family of distributions known as PERT, beta-PERT, modified PERT, etc.

This means that if a beta distribution was used as a base, an analysis is performed with the same initial data and the project delivery model but different risk simulation tools will produce different results.

The application of different distribution shapes and the way they are applied to the project delivery model could significantly alter the final result of generated distributions calculated with the Monte Carlo Simulation method. However, this challenge should not be considered as a stand-alone problem. Other challenges, like data collection, quality of project delivery model, and others, may have even more significant effects. In this case, the variability of results caused by incorrect application of distribution shapes should be considered a comparatively smaller issue.