## Monte Carlo Simulation Challenges. Simulating the true source of uncertainty

Usually, duration and cost uncertainties are used as a base for the Monte Carlo Simulation (MCS). However, the duration and cost uncertainties are the results of other types of uncertainties.

Simulating the source and result of uncertainties may lead to different estimates. Sometimes substantially different. Most risk simulation tools do not work with the source data and only simulate duration and cost uncertainties.

## Uncertainty Types

Any project has Primary, Secondary and Project Level Uncertainties.

Uncertainty is a situation where multiple alternatives result in a specific outcome, but the probability of the outcome is not certain.

Ref: Risk vs Uncertainty
https://saluteenterprises.com.au/risk-vs-uncertainty

Many factors may impact the duration of the activity.

Ref: Critical path method challenge: Activity duration estimation
https://saluteenterprises.com.au/critical-path-method-challenge-activity-duration-estimation

Duration and cost of any activity are secondary uncertainties that are the result of the true primary uncertainties. The primary uncertainties include Volume of Work, Resource Productivity Rate and Cost, Skills Quantity, Resource & Activity Calendars, Material Quantity and Cost, etc.

## Does it matter?

Is it important for the MCS analysis? Why, after the probabilities to meet project objectives (time & cost) are calculated, we can’t analyse the sources?

Such analysis must be done BEFORE the simulation as the result of the MCS depends on what is simulated: primary or secondary uncertainties.

Let’s consider two simple projects consisting of a single activity:

Project 1
• The volume of work for this activity is 800 units
• Activity duration depends on the assigned resource’s productivity, with the optimistic estimate of 1.25 u/h, the most likely estimate being 1 u/h, and the pessimistic estimate being 0.8 u/p.
• The workday is 8 hours
• Uncertainty Type: Resource Productivity (-20%, +25%)

Project 2
• The volume of work for this activity is 800 units
• Activity duration depends on the volume of work, with the optimistic estimate of 640 units, the most likely estimate being 800 units, and the pessimistic estimate being 1,000 units.
• The workday is 8 hours
• Uncertainty Type: Volume of Work (-20%, +25%)

Both projects have the same duration uncertainty: 80 /100 /125 days, and the uncertainty range: -20% / +25%. If the duration uncertainty is used as the base for MCS, the probability of delivering the project in 100 days will be the same for both projects. If the true uncertainties are used instead, the duration uncertain the result will be different.

Let’s review three scenarios simulating secondary uncertainty (duration) and primary uncertainties (Resource productivity and Volume of work).

## Scenario 1: Duration uncertainty

If the work is represented by a single activity with three estimations (80d, 100d, 125d) and the activity duration is used as the base of uncertainty, as the result of MCS would be as following:
• 100d = 43%
• P75 = 108d
• P90 = 114d
• Distribution: Triangular
• Number of iterations: 50,000

## Scenario 2: Resource productivity uncertainty

If the work is represented by a single activity with three estimations (80d, 100d, 125d) and the resource productivity (0.8, 1, 1.25) is used as the basis of uncertainty, the result of MCS would be as follows:

• 100d = 55%
• P75 = 106d
• P90 = 112d
• Distribution: Triangular
• Number of iterations: 50,000

## Scenario 3: Volume of work uncertainty

If the work is represented by a single activity with three estimations (80d, 100d, 125d) and the volume of work (640, 800, 1000) is used as the basis of uncertainty, the result of MCS would be as follows:

• 100d = 44%
• P75 = 108d
• P90 = 115d
• Distribution: Triangular
• Number of iterations: 50,000

Activity Duration Formula

Why is the result of Scenario 2 different from Scenarios 1 and 3?
As mentioned, activity duration depends on many parameters. These parameters impact the probability of completion in different ways. It is impossible (at least for now) to create a complete formula that shows how all primary uncertainties impact activity duration as some parameters (resource quantity uncertainty, etc.) also depend on other activities. Still, the simplified formula that only includes reviewed scenarios is:

As we can see above, it does matter if the Primary uncertainty is the Numerator or Dominator, as it impacts the distribution curve!

In the first case (scenario 2) the probability of achieving the Most Likely duration is >50%, and in the second case (scenario 3), it is <50%.

## Complex uncertainties

We reviewed a simple example with the uncertainty of one parameter to explain the challenge. It is clear that just for one activity without specialised risk simulation software that supports the simulation of primary uncertainties, it is very hard to understand how primary uncertainties impact activity duration uncertainty.
The real-life relationship is more complex as:

Activity may have more than one source of uncertainty.

What would the Optimistic & Pessimistic duration and the distribution curve be if an activity has uncertainties in many primary parameters? Is it impossible to understand it accurately?

It is very hard, and there is a good chance that even experienced estimators make costly mistakes when they provide 3-point estimations.

Do demonstrate it before you read the result guess:
If the activity has both (Volume of work and Productivity) uncertainties,
– What would be the probability to complete the activity in 100 days?
– What would be the optimistic & pessimistic durations?

## Scenario 4: Volume of Work and Productivity uncertainties

There is 47% chance to complete this activity in 100 days.
The most likely duration is 96 days and the Optimistic – Pessimistic range is: -32% , +52% (!).
Have you noticed that the curve is starting to get a “fat tale”? Take note, I will go back to the “fat tale” anomaly in future posts.
This was a simple example, as all required information was given  (even preliminary calculations!). Do you have this right? Probably not.

It is much easier to provide reliable 3-point estimations for primary uncertainties and calculate duration uncertainties than estimate duration uncertainty correctly.

We have simulated only one activity. Projects have thousands of them. How does this challenge impact project-level uncertainties? This is not an easy question that requires separate deep analysis. However, practically we can see that:

Simulating the result of uncertainty rather than its source, the level of accuracy is substantial to say that the result is misleading, not just inaccurate.

Let’s review one more scenario to demonstrate that.

## Scenario 5: Duration uncertainty in detailed schedule

Assume the scheduler for this project decided that each activity has to be no longer than ten days and converted the schedule to 10 linked activities. Visually nothing is changed. The project has the same duration and uncertianties. Has this impact the probablity?

If the work is represented by ten 10-day activities with three estimations (8d, 10d, 12.5d) and the activity duration is used as the basis of uncertainty, the result of MCS would be as follows:
• 100d = 28% (!)
• P75 = 104d
• P90 = 105d
• Distribution: Triangular
• Number of iterations: 50,000

It does not look logical.

Probability distribution should not depend on how the work is presented in the project schedule!

The project duration uncertainty depends on uncertainties of activities. However, as they are linked in a sequence, uncertainties compensate each, and simulation shows that only 105 days are required to achieve 90% probability.
The durations of all ten activities are strongly correlated, but this correlation is different. If a resource with low productivity is available for the assignments, it will impact all ten activities similarly. If the source of uncertainty (resource productivity) is simulated instead of the result (activity duration), this problem would not exist.

## Scenario 6: Productivity uncertainty in a detailed schedule

If the work is represented by ten 10day activities and the resource productivity (0.8, 1, 1.25) is used as the basis of uncertainty, the result of MCS would be as following:

• 100d = 55%
• P75 =106d
• P90 = 112d
• Distribution: Triangular
• Number of iterations: 50,000

We received the same result as it was in scenario 2 when the simulation was based on resource productivity, but the work was represented by one 100d activity!

– I used the triangular distribution to make it possible for others to verify the analysis. If PERT or Long-Normal distributions are used instead the difference in the results will be even bigger.
– This challenge alone has a significant effect, but when correlated with other discussed in previous (and future) posts issues, it has accumulated effect and the results of MCS are likely to be misleading.

## Summary

• Activity duration and cost uncertainties are the results of other types of uncertainties. They called secondary and primary uncertainties.
• The primary uncertainties impact activity duration and probability of completion activity in planned duration. However, without Project Management software that supports the simulation of primary uncertainties, it is not easy to understand the relationship between primary and secondary uncertainties.
• An activity may have more than one source of uncertainty. The correlation between them is complex. In some cases, primary uncertainties may compensate for each other and in other situations, the impact may be accumulated.
• On the activity level simulating the source and result of activity uncertainties may lead to different estimates. Sometimes substantially different and even experienced estimators are likely to make mistakes in providing 3-point estimations. It is much easier to provide reliable 3-point estimations for primary uncertainties.
• Projects have thousands of primary uncertainties. If simulating the result of uncertainty rather than its source, the deviation from the correct result will be substantial to say that it is misleading, not just inaccurate.
• Most project management Monte Carlo solution tools don’t work with primary uncertainties, and the results produced based on these tools are unreliable.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

## Monte Carlo Simulation challenges. Level of details challenge.

As discussed in the previous posts, applying the Monte Carlo Simulation (MCS) CORRECTLY for a large project may take many hours. As a workaround, some risk consultants recommend running simulations based on a high-level schedule, known as a control schedule and a ‘Level 3’ schedule.

The results of MCS performed based on the detailed schedule and high-level schedule are likely to be different, sometimes significantly different! There are a few reasons for that. In this post, we review one of them: the missing level of complexity.

Work packages are the smallest unit of work that a project can be broken down into when creating your Work Breakdown Structure.

In this post, the term ‘Work package’ (WP) represents ‘A group of activities’. It could be the smallest unit of work or a group of units.

When WP in a high-level schedule is represented by a single activity (or a few activities), the schedule misses a level of complexity of each WP. Without this information, the result of the simulation could be dramatically different.

Let’s review a simple example: A project has two WPs. Each of them has five 10-day activities, but in 1st WP all activities are consecutive, and in 2nd WP, all activities are concurrent.

If we decide to create a high-level schedule, we could simplify it to two linked ‘50 days’ and ‘10 days’ activities. The overall duration of both schedules and the logic between WPs will be the same.

Let’s assume each activity in our project has +/-2 days of uncertainty, which is +/-20%.

Detailed schedule with uncertainties:

High-Level schedule with uncertainties:

Now, we can calculate the probability of delivering this project in 60 days (the deterministic Most Likely scenario) and use both schedules as the base for the simulation.

We are going to use a triangular distribution. As explained in the previous posts, the Beta / PERT distributions often better represent real-life probability, but they are unique to each risk analysis tool. I want to be sure that this experiment can be replicated with other risk analysis tools.

Also, due to the randomised nature of the Monte Carlo Simulation method, the result of any new simulation is always different. The more iterations performed, the less volatile results will be. I set the Spider Project to 50,000 iterations to minimise the volatility.

## Project Probability Distribution

The performed simulation based on the High-Level schedule gives us the following probability distribution:

The performed simulation based on the Detail schedule gives us a different result:

There is 49.8% probability that this project could be completed in 60 days if the High-level schedule was used for this analysis and 29.2% probability if it was performed based on the Detail schedule.

However, project activities in both cases are going to be executed the same way, and the probability  should be the same. To understand such an anomaly, we need to go to the next level and analyse the probability of each WP separately.

## 1st WP probability distribution

The probability distribution of the first WP based on High-Level schedule:

The probability distribution of the first WP based on Detailed schedule:

Uncertainties in a sequence with ‘Finish-to-Start’ dependencies compensate for each other, so the result of the First WP simulation is roughly the same, regardless of the applied schedule. It is around 50%.

## 2nd WP probability distribution

As the second WP has only concurrent activities, the probability distribution curves are dramatically different.

Based on the Detailed schedule it is only ~3% chance that ALL activities could be completed in 10 days. If any activity is late (>10 days), the WP will be late.

In the High-level schedule, this level of complexity is lost, and the probability complete the WP in 10 days is 50%.

Probability distribution of the second WP based on the High-level schedule:

Probability distribution of the second WP based on the Detailed schedule

There is a big difference between 3% and 50%! Depending on which result is presented to project decision-makers, it may lead to different actions.

## Level of complexity

The analysed example above may look extraordinary. Not all projects have five linked concurrent activities. However, if we analyse any schedule, we are likely to find that there are days with more than five planned activities. All these activities are logically linked via direct or indirect logic. If these activities have large Total Floats, the effect of uncertainties should be immaterial. However, if they are on a project critical path or near a critical path, the effect will likely be noticeable.

Without Quantitative Risk Analysis (QRA), it may not be possible to understand the effect of uncertainties on project delivery dates.

Logic in real-life schedules is much more complex. WPs have other than ‘Finish-to-Start’ types of dependencies, technical and resource constraints, activity and resource calendars, lags and leads. These factors impact the level of complexity of each WP.

Also, logic between WPs is more complex than just a ‘Finish-to-Start’ dependency between the last activity in one WP and the first activity in another WP. On top of this, there could be more than one dependency between WPs, and these dependencies could be somewhere inside each WP, not between the end and start points:

Example of complex logic between WPs

## The volatility of Critical Path

Another challenge is to demonstrate a Critical Path in a high-level schedule. A WP may or may not be on a critical path. If it is, it doesn’t mean that all activities in the WP are critical. The critical activities in the WP could be somewhere inside of WP. There also could be many critical fragments.

A scheduler may summarise critical and not critical activities in a high-level schedule. Unfortunately, there is no guarantee that the critical path is going to be stable during the life of the project. The volatility of the critical path means that a high-level schedule has to be constantly redeveloped.

## Detailed Schedule

The described challenge is relevant to detailed schedules, not just to Control schedules.

There are no commonly accepted rules that define the correct level of detail for each schedule. While there are some recommendations, practically the level of detail depends on portfolio standards, the experience of the master schedule, the capacity of the scheduling team, applied schedule and risk analysis tools, and some other factors.

Two schedules developed for the same project by two different project teams are unlikely to be identical. The difference may look immaterial but may have a significant impact to the results of Monte Carlo Simulation.

The probability of delivering this project based on the ‘Most Likely’ durations is only 29%. This example demonstrates ‘Schedule duration paradox’:

Schedule based on the most likely activity durations doesn’t have the most likely duration

This is one of the factors why projects are late. When project SMEs provide the ‘Most Likely’ estimations for each schedule activity, and a scheduler uses them to develop a deterministic (Critical Path Method) schedule, project stakeholders often assume that such a schedule has ‘most likely’ delivery dates. Calculated dates are used as project commitments. However, due to the complexity of logic and parallel streams, the probability of meeting these commitments is likely to be less than 50%. Sometimes significantly less.

Monte Carlo Simulation is one of the Quantitative Risk Analysis (QRA) methods that help to calculate achievable dates, but it has to be applied correctly. A high-level schedule doesn’t have the same level of complexity as it is in a detailed schedule. It impacts the project probabilities calculated based on the Monte Carlo Simulation method. The difference could be sufficient to drive the project in the wrong direction.

There are many other important MCS challenges. I will continue “unwrapping” them one by one in my future posts.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

## Monte Carlo Simulation Challenges. Critical Path Volatility Challenge.

The ‘3 points estimation’ known as a PERT methodology is relatively simple to use for cost, but for the schedule, it’s more complicated. The fact that some activities are on the critical path means that the volatility of those specific activities, as opposed to the set of all project’s activities, can make a big difference for Monte Carlo Simulation.

## Risks and Critical Path

It is well accepted that risks are measured based on the probability of the event (or condition) occurring and the consequences.

Risk Exposure = Probability (Event) * Consequence

The same risk may have different consequences for Time, Cost, Safety, Reputation and other project objectives.

• The time consequence is highly dependent on the criticality of impacted activities. If impacted activities are on the critical path# [including delay], the project is going to be delayed. Otherwise, it will impact some parts of the project but not the overall delivery date(s).
• Risk consequences are not always a stable parameter. Some of them depend on project conditions.
• Risk consequence has to be measured based on future conditions when the risk is likely to occur, not the current situation.
• When the risk is likely to occur project critical path may be different from the current critical path. It could be due to the impact of project uncertainties, other risks, scope changes or any other reasons.
• At that time, there will be only two possibilities: the critical path impacted or not. In other words, the probability (CP) that the risk has a time impact will be 0% or 100%. Do not mix with the Probability of the event, which is not dependent on the critical path.

However, when the risk is assessed, it is not always possible to predict if the impacted activities are going to be on the critical path or not, so the probability (CP) could be different from 0% and 100%.

## Probabilistic vs Deterministic Delivery Model

If the Monte Carlo Simulation is performed based on a dynamic PROBOBALISTIC model (aka schedule) that includes data uncertainties and “conditional branches ”, the critical path will be recalculated during each calculation, and only the event probabilities need to be assessed prior to the simulation.

PROBABILISTIC project delivery systems, like Spider Project, could additionally calculate the probability of each activity being on the critical path, which is known as ‘activity criticality index’.

However, if the Monte Carlo Simulation is performed based on a DETERMINISTIC schedule (developed in Primavera, Microsoft Project or another CPM-based scheduling tool) and risk data is collected in a separate risk register, the probability that impacted activities are going to be on the critical path (CP) has to be assessed manually. The accuracy of such an evaluation would be significantly different from calculated probabilities and it will have following effect the results of the Monte Carlo Simulation.

Some risk management tools provide users with an option to define if the critical path is going to be impacted or not with a binary option (Yes/No). Users potentially could assess it’s based on the current critical path, but, with some rare exceptions, it is impossible to say whether the future critical path is going to be impacted or not as it depends on other materialised risks.

Some project Risk consultants just assume that the Critical Path is going to be the same

(Probability (CP) =100% or 0%) and only assess event probability even if the previous similar projects had critical path volatility.

## Risk Exposure

If the (time) consequence depends on the Project Critical Path, or to be precisely the probability that impacted activities are going to be on the critical path when risk is materialised, Risk Exposure also depends on this parameter:

Consequence (Net) = Probability (CP) * Consequence (Gross)

So:

Risk Exposure = Probability (Event) * Probability (CP) * Consequence (Gross)

or

Risk Exposure = [ Probability (Event) * Probability (CP)] * Consequence (Gross)

## Cost Impact

So far, we have discussed only time exposure. What about the cost exposure? Is it also dependent on the critical path? Yes, but in a different way.

Risk cost consequence has two parts: direct cost impact and cost of delay.

Direct cost impact only depends on the Probability (Event) and the cost of delay only depends on Probability (CP).

## Summary

The volatility of the Critical path means that risk consequence is not a stable parameter and depends on future project conditions when the risk is likely to occur. It is hard to accuratelly predict which activities are going to be on the Criticla Path at that time.

The results of Time and Cost probability distributions received based on deterministic the CPM-based schedule and predefined risk data could be misleading and should be used for project decisions with caution.

Risk consequences have to be calculated in each simulation. Dynamic probabilistic model with fully integrated risk data is required for reliable Monte Carlo Simulation.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

## Risk vs Uncertainty

At project management conferences, LinkedIn discussions and even in one of the most recognised Project Risk Management book, I’ve found that uncertainty is just a risk with 100% probability. Not sure what is the origin of this statement, but a risk with 100% probability is an issue, not an uncertainty.

Managing project risks and uncertainties is essential for mature risk management and successful project delivery. Lack of understanding differences between these two concepts creates serious project planning and control issues and often results in project delays and cost overruns. In this post, we will review what is common between these two concepts and where the key differences are.

## Project Risk Management Standards

Risk and Uncertainties are interrelated. Risk arises out of uncertainty and generates uncertainty.

Project Management Institute (PMI) and Association for Project Management (APM) used these terms widely in their project management and risk management standards. Both organisations suggest dividing the term Risk into two terms: Risk Event and Overall Project Risk. PMI and APM don’t provide definitions of Uncertainty but mention that the source of uncertainties is complexity, ambiguity, and volatility.

### Project Event

Risk Event Definition by PMI:

Risk is an uncertain event or condition that, if it occurs, has a positive or negative impact on at least one project objective. (The Standard for Risk Management in Portfolio, Program, and Projects)

Risk Event Definition by APM:

A risk event is an uncertain event or set of circumstances that, should it occur, will have an effect on achievement of one or more of the project’s objectives. (Project Risk Analysis and Management Guide)

### Overall Project Risk

Overall Project Risk Definition by PMI:

Overall project risk is the effect of uncertainty on the project as a whole, arising from all sources of uncertainty. This includes individual risks and the exposure to the implications of variation in project outcome, both positive and negative. Overall risk is often a function of complexity, ambiguity, and volatility. (The Standard for Risk Management in Portfolio, Program, and Projects)

Overall Project Risk Definition by APM:

Overall project risk is joint effect of risk events and other sources of uncertainty. Project risk results largely from the accumulation of a number of individual risk events, together with other sources of uncertainty to the project as a whole, such as variability and ambiguity. (Project Risk Analysis and Management Guide)

Unfortunately, apart from the definitions, both standards don’t provide any methods and techniques to measure and control Overall Project Risk. There is no common acceptance among project and risk managers on what it is the “beast” and how to deal with it.

Success Driven Project Management (SDPM) suggest dividing risks into two major categories: uncertainties and events, where uncertainty is a property of project parameters. Based on SDPM, changes caused by risk are discreet events, while changes caused by uncertainties are continuous.

Let’s look at risks and uncertainties more closely.

## Risk

Less formally, risk is defined as the situation of winning or losing something worthy.

In projects, stakeholders are interested in understanding how such situations may impact project objectives, usually represented as the delivery of outputs (or critical deliverables) to the committed dates with the agreed budget. Depending on the type of project, objectives may also include quality, benefits, reputation and other important aspects of the business.

“Worthy” means that while risk events may impact other aspects of the project, these are the only important ones in the context of the overall delivery.

For example, a project team may raise a risk that a particular activity might be running late or next month’s forecast could not be met if a risk event materialised.
While this information still may be necessary for some stakeholders, a project risk usually is raised in a broader context:

• Does it mean that if the activity is late, the project will also be late?
• Does it mean that if the next month’s actuals are more than was forecasted, the project will be over budget?

‘Risk impact’ is a dynamic characteristic, not a stable parameter. A risk may have no impact on the project objective now, but the situation may be different tomorrow, and the risk would become the most critical risk in the project. Without Quantitative Risk Analysis methods, it is not easy to identify such changes timely.

There are many books and risk management standards with recommendations for risk control. While uncertainties are also considered in these books and standards, I think this area of project risk management is less developed and definitely less controlled.

## Primary Uncertainty

Uncertainty is a situation where multiple alternatives result in a specific outcome, but the probability of the outcome is not certain.

Any project has hundreds, if not thousands, of situations when the outcome is uncertain. However, any project has quite limited project parameters with uncertainties.
Examples of parameters that may have uncertainties:

• Volume of Work
• Resource Productivity Rate
• Resource Quantity
• Resource Calendar
• Activity Calendar
• Material Quantity
• Activity Duration
• Lag Duration
• Material Cost
• Unit Cost
• Currency rate
• Resource Rate
• Inflation Rate

There are examples of bottom-level uncertainties, also known as base, or primary uncertainties.

• Most, but not all project parameters, have primary uncertainties.

Example:
Office calendars, compared to construction calendars, don’t depend on weather conditions.

• Data collection challenges

It is easy to collect primary uncertainty data as each refers to only one project parameter. Three-point estimation is a common practical approach. Usually, the critical constraint for data collection is not the information itself but the lack of opportunity to store it in a project management system. Not many of them are designed to store and apply primary uncertainties to calculate project delivery plans. Spider Project is an excellent example of such a system when primary uncertainties are stored as corporate norms and used to calculate project delivery plans with probabilities to achieve committed project objectives: dates, costs, benefits, etc.

I believe there is a lack of recognition of the benefits of capturing and applying uncertainty data for the reasons explained below. Many organisations with such experience are in non-English-speaking countries and usually don’t publish results in recognised PM Journals.

## Secondary Uncertainties

The outcome of primary uncertainties may result in uncertainties of other project parameters, especially activity duration and cost uncertainties, which are secondary or middle-level uncertainties.

• The secondary uncertainty is the result of one or multiple primary uncertainties.

Example:

An activity has uncertainty in the volume of work and productivity of assigned resources. Uncertainty in the duration and cost of the activity is the result of both uncertainties.

• Activity duration and cost uncertainty may or may not depend on other primary uncertainties.

So, it could be a primary or secondary uncertainty. It is very confusing, but who said that project management has to be simple?

Example:

An external project review takes 24-40 hours. It means that the ‘volume of work’ of this task is measured in hours, and in the worst-case scenario, the report is going to be completed when 40 hours of effort, even if the quality of the report needs to be compromised.

• Not all primary uncertainties impact duration and cost of activities.

Example:

‘Resource Quantity’ uncertainty may impact the order of planned activities but not in their durations and cost.

• Result of Monte Carlo Simulation (MCS) highly depends on the type of uncertainties used as a base for analysis.

Example:

Results of MCS analysis performed based on secondary uncertainties may significantly vary from the analysis performed based on primary types, as important correlations between project parameters are missing. This is another big topic to discuss, and I’m planning to cover it in a separate post under the “Monte Carlo Simulation challenges” posts.

## Project Level Uncertainties

Multiple primary and secondary level uncertainties and related risks result in probability distributions of project commitments. These distributions could be considered as ‘top-level’ or tertiary uncertainties.

Examples of top-level uncertainties include:

• Delivery date of committed output
• End of the design phase
• Cost of project

• Probability distributions could be considered in the context of individual project objective or with the relationship to other objectives.

## Risk Uncertainties

Projects usually estimate risks with probability, as a percentage of the likelihood of the event or condition to occur, and the impact estimated in days and \$. However, this information is often subjective as different project stakeholders have different opinions about the probability and impact.

As a result, a risk may have uncertainties in the probability, the impact(s) or both at the same time.

## Cost of Knowledge

Both, uncertainties and risks depend on the knowledge of estimators. Sometimes it may be possible to gain this knowledge and reduce uncertainties but usually, it is associated with extra effort and additional cost.

Sometimes it is worth spending extra effort to gain missing information that could help with project decisions, sometimes not. In other situations, the required information may not exist and any effort to clarify the information is a waste of time and money. It is another separate big but interesting topic to discuss.

Portfolios with advanced project management collect historical data and apply more accurate estimations to outstanding delivery and future projects. It is important to mention that, if possible, historical data has to be collected with primary types of uncertainties. If only secondary (activity level) uncertainties are collected, the critical context would be missing, and the historical data could not be applied correctly.

Example:

An activity has uncertainty in duration due to the uncertainty in the productivity of the planned resource: new bulldozer. The uncertainty is clarified during project execution and, in theory, next projects could use this information to reduce volatility. However, if only duration uncertainty was documented and another bulldozer with a different productivity rate is assigned to perform the same type of work, the historical data would be misleading as the critical context is missing to understand that this type of work has a different duration now.

## Impacts of Risk and Uncertainties

• Not all uncertainties have an impact to project objectives.
Sometimes, under current conditions, regardless of the result of the uncertainty, it would be no impact to project commitments.

Example:

Next month, a project must pay for materials in a different currency. There is uncertainty about next month’s currency rate. This uncertainty could be an opportunity, treat or has no impact to project cost at all. It depends on:

• Which way does the currency rate go? Maybe it is an opportunity, not a threat?
• Currency rate the project forecasted. Maybe it was forecasted based on the worst rate?
• Conditions in the contract. Maybe the project has a fixed currency rate?
• A risk is always a threat or opportunity from one perspective (time, cost, etc). Uncertainties could be a threat and an opportunity to the same perspective simultaneously.

Example:

There is uncertainty in the volume of work that will impact the duration and cost of the activity. The uncertainty was estimated as a range (100 – 150 m2). If the project plan has 125 m2 as a base, it would be an opportunity to complete the activity ahead of schedule or a threat of delaying the completion.

• The same risk could be an opportunity for one perspective and a threat for another.

Example:

The bulldozer with certain productivity is planned to be assigned to work. There is a 20% chance that this bulldozer is not going to be available, and the project needs to hire another more expensive but also more productive bulldozer. If the risk is materialised, it will be a threat to the project cost, but also an opportunity to project delivery date (if planned work is on the critical path). So, this risk is a threat to cost and an opportunity to time. When a project uses a risk register and classifies all risks as a threat or opportunity, the mitigation actions may solve some potential issues but generate others.

## Creeping Normality

Primary uncertainties impact projects via very small changes, but commutatively, they may have a large effect. Such an effect could be even more severe than the impact caused by project risks.

Project control teams usually have less control over activities with sufficient contingency. Positive and negative changes caused by uncertainties slowly and in a non-linear way “eat” available contingency, and when suddenly activities start to be critical and impact project delivery dates, it is already too late to apply mitigation actions. The contingency already has been spent. It is even more applicable to project cost when a lot of minor cost increases slowly “eat” cost contingency.

I think this could be due to “Creeping Normality” cognitive bias, also known as “Landscape amnesia”.

Creeping normality is a process by which a major change can be accepted as normal and acceptable if it happens slowly through small, often unnoticeable, increments of change. The change could otherwise be regarded as remarkable and objectionable if it took place in a single step or short period.

It creates an illusion that ‘risk events’ is the biggest factor in project delay and cost overrun when, in reality, changes caused by uncertainties have a bigger effect caused by project risks.

Projects that document uncertainties and compare them with actuals can notice “patterns” and apply mitigation actions before it is too late.

## Relationship Between Risks and Uncertainties

While it is obvious that Risks and Uncertainties are interrelated, the relationship between them is not always obvious.

• Uncertainties in one part of a project may have a negative (or positive) impact on risks in another part of the project.

A risk associated with activities on the critical path is likely to severely impact project delivery dates. Changes caused by uncertainties in another part of the project may have an indirect impact on the risk activities total float and the risk mitigation actions should be changed as well. However, as projects usually don’t control uncertainties, it is hard to notice when the impact of risk is changed. In the future post, I am going to demonstrate this effect with a more detailed example.

## Summary

Risk and uncertainties are important elements of Project Risk management. They are interrelated but have critical differences that are not always understood correctly. While Project risk management standards recognise the importance of management of uncertainties and risk, projects across the Globe often don’t have mature control over uncertainties.
For project success, it is not sufficient to only control risks. Mature Risk Management also has to include the management of uncertainties.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

## Monte Carlo Simulation Challenges. Distribution Shapes Challenge.

Risk simulation systems offer a choice of user-selected distribution shapes that can be implemented for each and every activity. Some tools offer only primary types, others have a large menu of over 50 distributions.
However, selecting a specific shape for each of 1,000+ activities (on top of generating three estimates for each item) requires many hours invested. The majority of users run the system using one of the distributions applied to all activities with uncertainties. It is mostly Triangular, Beta or Log-normal distributions.

What they may not be aware of is that different activity distribution shapes can substantially alter the result of the analysis. The result of the simulation performed with beta (PERT) and Triangular distributions at the P80 mark is likely to have a noticeable 8-15% difference. At the same time application of the same distributions at the different P-point may have a less noticeable difference.

Some construction and consulting companies may use it to get extra funds (or time).
All they need to do is just to run the risk analysis with another distribution type!
Receivers of project risk simulation reports need to review which distribution shape was applied and they need to understand how it may alter the presented result. As a minimum, there must be consistency between reports produced over time.

## The Triangular Distribution

When projects don’t have subjective historical data and only a subjective range (2-point estimation) is available, it makes sense to calculate the Most Likely estimation as an average between the optimistic and pessimistic estimations: ML=(O+P)/2 and apply the triangular distribution.

Example:

A team needs to perform 10 activities. Each activity may take anything between 8 and 12 days. In this case, the most likely duration would be 10 days and symmetrical triangular distribution should be applied.

However, if there is a reason to believe that the most likely estimation is different, the asymmetrical triangular distributions would be the choice.

Another advantage of the triangular distributions is that result of the Monte Carlo simulation would be the same regardless of the applied risk simulation tool, with an assumption that both tools don’t have other issues which we are going to discuss in future posts.

The disadvantage of the triangular distribution is that often the pessimistic and optimistic estimations represent rare extremes and it has to be reflected in the shape. In this case, projects need alternative shapes that could reflect this anomaly. The most known are the beta and Log-normal distributions.

## Beta Distribution

The famous PERT method also based on Three-point estimations is able to address “extreme ends” anomalies. The PERT addresses it by taking More likely estimation 4 time often than Optimistic and Pessimistic estimations: T = (O + (4* ML) + P) / 6.
A beta distribution is often considered the best choice for project simulations. While it sounds very scientific, each risk management tool uses its own formula for beta-distribution! In fact, it is actually a family of distributions known as PERT, beta-PERT, modified PERT, etc.

This means that if a beta distribution was used as a base, an analysis is performed with the same initial data and the project delivery model but different risk simulation tools will produce different results.

## Summary

The application of different distribution shapes and the way they are applied to the project delivery model could significantly alter the final result of generated distributions calculated with the Monte Carlo Simulation method. However, this challenge should not be considered as a stand-alone problem. Other challenges, like data collection, quality of project delivery model, and others, may have even more significant effects. In this case, the variability of results caused by incorrect application of distribution shapes should be considered a comparatively smaller issue.

#### Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma