Monte Carlo Simulation Challenges

Monte Carlo Simulation Challenges

Risk simulation is becoming popular but most risk simulation tools and the ways how they are used miss some important functionalities that make the results of this simulation unreliable.

Monte Carlo Simulation – Myth or Reality?

Last year, I have organised a poll on LinkedIn to understand what project practitioners think about Monte Carlo Risk Simulation:

The Monte Carlo Simulation Method is the best method for quantitative project risk analysis: Myth or Reality?

The poll had a lot of attention, and many projects and risk consultants shared their opinions.

Based on discussions, comments and the final result we can make some conclusions:

  • Monte Carlo Risk Simulation (MCS) is the most recognised quantitative project risk analysis method;
  • There is no common acceptance of this method across project practitioners;
  • Opinion about Monte Carlo Simulation is mostly based on perception rather than knowledge;
  • Majority of planners and risk consultants are mostly not aware of missing critical functionalities in Risk Simulation tools.

The popularity of MCS in recent years is primarily driven by companies that promote their own Monte Carlo Simulation software or Quantitative Risk Analysis (QRA) training. Unfortunately, well too often they measled project practitioners by telling them that it is easy to apply the method to get a reliable result. This is how one leading American consulting company that sells Quantitative Risk Analysis training attracts their clients:

“For many, Quantitative Risk Analysis (QRA) is a complex secretive technique, which relies on smoke, mirrors and mathematical trickery. The aim of this webinar is to draw back the curtain and show that QRAs are not that complex and by learning a few basic steps you can apply QRAs to any project to aid in their successful delivery.”

 

Demonstration of Monte Carlo principles based on the probability distribution of two dices may be good for a teenager, but projects are much more complex and deep knowledge is required to understand how to apply MCS correctly and which tool could be used to get reliable results.

Based on my research I found that different Monte Carlo Risk Simulation challenges are explained in conference presentations, blogs, White Papers and books but there is no single source where all challenges are collected or explained. I have decided to collect them and present the result at the Project Control Expo conference. 45 minutes is sufficient to cover only some key challenges at a high level. Fortunately, I am not limited by time and space on the Salute Enterprises blog and we could discuss each challenge in detail.

I am going to write a series of “Monte Carlo Simulation Challenges” posts, and discuss them on LinkedIn. If you are aware of any good source that explains such challenges please share them and join discussions on LinkedIn.

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

Schedule Health Metrics: Activity Constraints

Schedule Health Metrics: Activity Constraints

In this article, we continue to review metrics for schedule health analysis. The first part, “Missing Dependencies” was covered here. Now let’s talk about activity constraints.

There are some project situations that require the use of constraints:

Delayed Start

Sometimes a project has a situation when it is impossible to start the activity even if all prerequisite work is completed.

Example: Project is ready to start testing but the test environment is only available from next week.

Deadlines

If a Project has committed dates, we need to understand how the project is performing against these commitments, also known as project deadlines.

Example: Based on the business case project committed to delivering three Outputs by 1st October, 1st January and 1st of April.

Locked Dates

Some projects have locked dates when there is no flexibility to move the start and finish dates.

Example: The Olympic opening ceremony in Brisbane is locked to 23 July 2032.

As Timely As Possible

If there is flexibility when activity could be performed without impacting key delivery dates there are three options to specify the start and finish date:

  • As Soon As Possible (ASAP)
  • As Late As Possible (ALAP)
  • As Timely As possible (ATAP)

Starting an activity ASAP is usually considered the best approach. In practice, however, starting the activity as soon as it can or leaving it ALAP can both have problems associated with them. ATAP could be achieved with constraints, lags or calendars. Constraints are more visible and often are the best approach.

Examples:

  • If the project might require delivery through the Caribbean in September, they might want to put in a constraint to delay the trip until October 15, after the height of the Atlantic hurricane season.
  • A union contract may expire on June 3, with the potential for a strike. In that case, the project might want to make sure that certain work gets performed before then.

How many constraint types are required to simulate these scenarios?

All possible scenarios in projects could be specified with only two types of constraints:

  • Start No Earlier Than (SNET)
  • Finish No Later Than (FNLT)

No Earlier Than (NET):  Prevent an activity from being scheduled to start before a specific calendar date.

No Later Than (NLT): Specify that an activity needs to be finished before a specific calendar date.

‘Deadline’ and ‘Locked dates’ events are presented with a milestone. As a milestone always has the same start and end dates there is no need to have additional Finish No Earlier Than (FNET) and Start No Later Than (SNLT) constraints. It just adds complexity to the schedule.

‘Locked dates’ and ‘As Timely As Possible’ scenarios could be represented by applying SNET and FNLT constraints simultaneously.

ASAP and ALAP are actually not constraints, even though they are listed in constraint types in Primavera and Microsoft Project. ASAP and ALAP could be presented as a switch as it is done in Spider Project.  

Activity constraints are essential to represent real-life project scenarios in schedules correctly. At the same time, incorrect application of constraints may have a negative impact on the management of project reserves or even the project delivery dates. Activity constraints may impact Free Float, Total Float and even Project Critical Path. This impact has to be analysed and accepted. So, we need Schedule Health Metrics to identify the incorrect application of activity constraints. There are some nuances that have to be discussed first.  

Constraint Terminology

Scheduling tools use different terms to describe types of constraints.

As explained above only ‘Finish No Later Than’ and ‘Start No Earlier Than’ are true types of constraints.

Primavera has 4 types of Start and 4 types of Finish constraints. Not all planners understand the difference between ‘Finish On’ vs ‘Mandatory Finish’ and ‘Start On’ vs ‘Mandatory Start’. The key difference is that ‘Mandatory’ constraints allow violation of schedule logic and ‘On’ constraints are complementary. As the result, it impacts the calculation of Total Float.

Microsoft Project offers users to decide whether the conflict is allowed:

Manage Deadlines

Also scheduling tools have totally different approaches to managing ‘Deadline’ scenarios.

  • Primavera has separate constraints types to simulate ‘Locked Dates’ and ‘Deadlines’ scenarios.
  • MS Project has an additional ‘Deadline’ field for the deadline date det up.
  • Spider Project always calculates Total Float with and without the deadline (NLT) It has additional column fields to present and compare both results.

These differences have to be reflected in the Constraints Health Check metrics.  

Hard and Soft Constraints

Some portfolios use ‘Hard’ and ‘Soft’ terms to divide constraints by softness.

There is no common agreement on what ‘Hard’ and ‘Soft’ actually mean.

Some Project Managers assume that ‘Soft’ constraints are ‘nice to have’ constraints and could be deleted if a fast-tracking is required. Others mean that ‘Soft’ constraints refer to resource (not technical) constraints. Arguably, the most popular differential amongst planners is based on a logical priority. ‘Soft’ constraints never violate network logic (but still may generate negative total float) and ‘Hard’ constraints take priority over network logic dependency relationships.

‘Hard’ constraints:

Delay of activity ‘A’ doesn’t push Activity ‘B’. The ‘No Later Than’ constraint followed. Relationship A->B is broken. The project is not logically driven anymore.

‘Soft’ constraints:

Delay of activity ‘A’ push Activity ‘B’. The ‘No Later Than’ constraint is ignored. Relationship A->B is followed. The project is logically driven.

However, some Primavera planners define ‘Hard’ constraints as ‘Mandatory Finish’ and ‘Mandatory Start’ constraints only and others also add ‘Start On’ and ‘Finish On’ constraints to the definition. Recently, I have seen scheduling standards that in one part of the document define ‘Hard’ as Mandatory Start and Finish only, and in another part of the document it also includes ‘Start on’ and ‘Finish On’. This could be driven by different definitions in international scheduling standards.

Planning & Scheduling Excellence Guide (PASEG) published by National Defence Industrial Association (NDIA) defines ‘Hard’ constraints based on Primavera terms (or did Primavera developers apply NDIA term?) and the definition includes 2 types only: ‘Must Finish On’ and ‘Must Start On’.

GAO Schedule Assessment Guide uses Microsoft Project terms (or did Microsoft developers apply GAO terms?) and the definition of the ‘Hard’ constraint includes 4 types: ‘Start No Later Than’, ‘Finish No Later Than’, ‘Must Start On’, and ‘Must Finish On’.

‘Start No Later Than’ and ‘Finish No Later Than’ constraints don’t violate the logic but are somehow still defined as ‘Hard’.  

DCMA 14 Points Schedule Assessment

DCMA 14 Points Schedule AssessmentDCMA 14 Points Schedule Assessment includes a metric to control the percentage of Hard constraints: “Number of activities with hard or two-way constraints should not exceed 5%”.

If a scheduled health check includes the terms ‘Hard’ & ‘Soft’, to avoid misunderstanding of the terms there need to be clear definitions established.

Points to Consider

  • All constraints must be explained. Ideally, the clarification has to be in the schedule but if a scheduling tool doesn’t have a feature to document and export this information a separate register should be created.
  • As each project is different by nature, criticality and management style it is not possible to define one metric that could clarify that the schedule has too many constraints. At the same time, a number of constraints in a schedule is a very good indicator that the schedule could be not logically driven.
  • Some Planners and Master Schedulers fail to appreciate the nuances of constraints and instead restrict their use altogether. Constraints are essential to represent the project work correctly. If constraints are prohibited the schedules could not be used to drive project delivery, only to satisfy someone’s need to say that the project has a schedule.
  • What type of schedule health metric to use depends on scheduling tools. Constraints in Spider Project never violate network logic, so metric is not required to control the quality of schedules. In Primavera and Microsoft Project, incorrect application of constraints may impact the calculation of Critical Path and Total Float, so users of these systems have to learn how the constraints work and decide which types of constraints have to be limited or restricted. They need a schedule quality metric to identify constraints.
  • If an activity already started it doesn’t matter if the start was constrained or not. Also, if an activity is completed it doesn’t matter if the start or finish of the activity was constrained. Such activities should be excluded from metrics.
  • Level of Effort, Hammock and WBS Summary activities should be without constraints. A separate metric should be developed to control that.
  • Primavera and Spider Project allow the setting of two constraints to the same activity while Microsoft Project allows only one. It limits Microsoft Project users to simulate all ATAP scenarios. As a workaround, all constraints (presented by milestones) could be stored under the same WBS. Then related activities are linked with the corresponding milestones. 
  • ‘Locked Dates’ and ‘ATAP’ are actually calendar constraints, not characteristics of activities and ideally have to be realised in the schedule via calendars and not via activity constraint dates. Unfortunately, Primavera is not able to calculate the schedule correctly taking both activity AND resource calendar constraints into account. Planners have to choose between activity OR resource calendars. Microsoft Project and Spider Project don’t have this problem. We will review this challenge in detail in a future post.

It also simplifies the control of constraints as all constraints outside of the WBS are considered as schedule quality issues. 

 

  • If an activity is left as late as possible, unforeseeable events can occur that make the activity take longer than anticipated, resulting in missed deadlines. So, it is strongly recommended to use pessimistic estimation as a base for all ALAP activities to have sufficient contingency and not delay the project.

Schedule Health Metrics: Constraints

Taking all these nuances into the consideration the following Schedule Health Metrics are useful:

Constrained Activities

A number and percentage of activities with constraints is a good indicator that could show that the schedule is not logically driven.

It is possible to develop filters to identify Hard and Soft constraints.

 

Primavera Hard Constraints Filter

Primavera Soft Constraints Filter

Unexplained constraints

Another Metric could help to identify constraints without an explanation. The method to identify them depends on the way the information is captured.

 

ALAP activities

Special attention has to be to all ALAP activities.

 

Technical and Resource Constraints  

Apart from DATES constraints, there are other types of activity constraints that also must be taken into the consideration:

  • Could an activity be split?
  • Does the activity need to be performed on the same date (or shift)?
  • How many minimum resources are required to complete the activity?
  • Is it technically possible to add more resources?
  • Would additional resources increase or decrease overall productivity?

 We are going to discuss these types of activity constraints in a future post.

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma