Monte Carlo Simulation challenges. Level of details challenge.

As discussed in the previous posts, applying the Monte Carlo Simulation (MCS) CORRECTLY for a large project may take many hours. As a workaround, some risk consultants recommend running simulations based on a high-level schedule, known as a control schedule and a ‘Level 3’ schedule.

The results of MCS performed based on the detailed schedule and high-level schedule are likely to be different, sometimes significantly different! There are a few reasons for that. In this post, we review one of them: the missing level of complexity.

In this post, the term ‘Work package’ (WP) represents ‘A group of activities’. It could be the smallest unit of work or a group of units.

Let’s review a simple example: A project has two WPs. Each of them has five 10-day activities, but in 1st WP all activities are consecutive, and in 2nd WP, all activities are concurrent.

If we decide to create a high-level schedule, we could simplify it to two linked ‘50 days’ and ‘10 days’ activities. The overall duration of both schedules and the logic between WPs will be the same.

Let’s assume each activity in our project has +/-2 days of uncertainty, which is +/-20%.

Detailed schedule with uncertainties:

High-Level schedule with uncertainties:

Now, we can calculate the probability of delivering this project in 60 days (the deterministic Most Likely scenario) and use both schedules as the base for the simulation. 

We are going to use a triangular distribution. As explained in the previous posts, the Beta / PERT distributions often better represent real-life probability, but they are unique to each risk analysis tool. I want to be sure that this experiment can be replicated with other risk analysis tools.

Also, due to the randomised nature of the Monte Carlo Simulation method, the result of any new simulation is always different. The more iterations performed, the less volatile results will be. I set the Spider Project to 50,000 iterations to minimise the volatility.

The performed simulation based on the High-Level schedule gives us the following probability distribution:

The performed simulation based on the Detail schedule gives us a different result:

There is 49.8% probability that this project could be completed in 60 days if the High-level schedule was used for this analysis and 29.2% probability if it was performed based on the Detail schedule.

However, project activities in both cases are going to be executed the same way, and the probability  should be the same. To understand such an anomaly, we need to go to the next level and analyse the probability of each WP separately.

The probability distribution of the first WP based on High-Level schedule:

The probability distribution of the first WP based on Detailed schedule:

Uncertainties in a sequence with ‘Finish-to-Start’ dependencies compensate for each other, so the result of the First WP simulation is roughly the same, regardless of the applied schedule. It is around 50%.

As the second WP has only concurrent activities, the probability distribution curves are dramatically different.

Based on the Detailed schedule it is only ~3% chance that ALL activities could be completed in 10 days. If any activity is late (>10 days), the WP will be late.

In the High-level schedule, this level of complexity is lost, and the probability complete the WP in 10 days is 50%.

Probability distribution of the second WP based on the High-level schedule:  

Probability distribution of the second WP based on the Detailed schedule

There is a big difference between 3% and 50%! Depending on which result is presented to project decision-makers, it may lead to different actions.

The analysed example above may look extraordinary. Not all projects have five linked concurrent activities. However, if we analyse any schedule, we are likely to find that there are days with more than five planned activities. All these activities are logically linked via direct or indirect logic. If these activities have large Total Floats, the effect of uncertainties should be immaterial. However, if they are on a project critical path or near a critical path, the effect will likely be noticeable.

Logic in real-life schedules is much more complex. WPs have other than ‘Finish-to-Start’ types of dependencies, technical and resource constraints, activity and resource calendars, lags and leads. These factors impact the level of complexity of each WP.

Also, logic between WPs is more complex than just a ‘Finish-to-Start’ dependency between the last activity in one WP and the first activity in another WP. On top of this, there could be more than one dependency between WPs, and these dependencies could be somewhere inside each WP, not between the end and start points:

Example of complex logic between WPs

Another challenge is to demonstrate a Critical Path in a high-level schedule. A WP may or may not be on a critical path. If it is, it doesn’t mean that all activities in the WP are critical. The critical activities in the WP could be somewhere inside of WP. There also could be many critical fragments.

A scheduler may summarise critical and not critical activities in a high-level schedule. Unfortunately, there is no guarantee that the critical path is going to be stable during the life of the project. The volatility of the critical path means that a high-level schedule has to be constantly redeveloped.

The described challenge is relevant to detailed schedules, not just to Control schedules.

There are no commonly accepted rules that define the correct level of detail for each schedule. While there are some recommendations, practically the level of detail depends on portfolio standards, the experience of the master schedule, the capacity of the scheduling team, applied schedule and risk analysis tools, and some other factors.

The probability of delivering this project based on the ‘Most Likely’ durations is only 29%. This example demonstrates ‘Schedule duration paradox’:

This is one of the factors why projects are late. When project SMEs provide the ‘Most Likely’ estimations for each schedule activity, and a scheduler uses them to develop a deterministic (Critical Path Method) schedule, project stakeholders often assume that such a schedule has ‘most likely’ delivery dates. Calculated dates are used as project commitments. However, due to the complexity of logic and parallel streams, the probability of meeting these commitments is likely to be less than 50%. Sometimes significantly less.

Monte Carlo Simulation is one of the Quantitative Risk Analysis (QRA) methods that help to calculate achievable dates, but it has to be applied correctly. A high-level schedule doesn’t have the same level of complexity as it is in a detailed schedule. It impacts the project probabilities calculated based on the Monte Carlo Simulation method. The difference could be sufficient to drive the project in the wrong direction.

There are many other important MCS challenges. I will continue “unwrapping” them one by one in my future posts.