Schedule Quality Metrics. Invalid Dates.

Schedule Quality Metrics. Invalid Dates.

One of the important questions, when a schedule quality analysis is performed, is: “Is the schedule review up-to-date?”. There are a few metrics that could help find the answer to this question and identify activities with potentially invalid dates. Some of them could be performed based on the current schedule and others require a comparison to the previous schedule snapshot. Identified issues may indicate that project has gaps in a schedule maintenance process that must be addressed.

Completed, in-Progress and Not Started activities

The status of each activity is defined by Project status date (data date), Start & Finish dates and ‘% Complete’. There is a correlation between these data points.

Incomplete activities in the past.

All activities with start and/or finish dates in the past have to be actualised (100% complete) or rescheduled.

Started or Completed activities in the future.

All activities with actual dates in the future have to be unactualised (0% if not Started and <100% if In-Progress) or actualised dates updated to the past.

Activates with a start date in the past and finish in the future but with 0% or 100% of progress.

In-progress activities have to have the start date in the past, the finish in the future, and ‘% Complete’ between 0% and 100%. All In-progress activities with 100% or 0% of Completion have to be reviewed. Dates or ‘% Complete’ need to be updated.

Incorrect Status of Milestones

When a milestone indicates completion of the event, the milestone has only to be marked as “Completed” when all predecessor activities are completed and, on the opposite, if all predecessors are completed, the milestone has to have a “100% Complete” status.

Actualised milestones with open predecessors.

Completed milestones with the “Not Completed” predecessor(s).

The status of the milestone and the status of predecessor activities have to be reviewed and updated.
This issue may indicate that the milestone has a strict dependency that must be removed.

Not Completed milestones with complete predecessors.

Not Completed milestones with ALL Completed predecessors.

This issue indicates that the schedule may have missing activities or dependencies.

Historical dates

Status, Start and Finish dates of already Completed activities should not change from version to version.

Unactualised activities

Activities that previously were reported as “Completed” but the status has changed to “In-progress” or “Not Started”.

Justification for each activity previously reported as completed, but not completed now must be provided.

Historical start and dates changed

Activities that previously were reported as “Completed” with changed Start and/or Finish dates.

Justification for each activity with changed actual start and/or finish dates must be provided.

The last two metrics are particularly important when a quality analysis of a contractor schedule is performed. Constant issues are a good indication that the contractor needs to mature their schedule maintenance process.

Also, projects with such issues may have a “double counting” issue when the same activity (or deliverable) is counted as being completed a number of times.  

Schedule Guidelines

GAO Schedule Assessment Guide, Planning & Scheduling Excellence Guide (DCMA-14) and NASA Schedule Management Handbook have metrics to identify invalid dates but they include only metrics to identify invalid activities in the past and the future, the first two checks in this post.

Schedule Tools

There is a philosophical question what to do when a schedule status update is not received?
Should a project team assume that all activities in past have not been started (or completed) and automatically push the activities to the earliest possible date: Project Status Update (known as Data Date) or they should assume that some of them may have started (or Completed) and not be updated dates until the clarification is received?
Probably developers of scheduling tools have different views on what is the correct answer to this question.
Primavera and Spider Project have the “Data Date” feature to automatically push activities with dates in the past to the Data date. Microsoft Project doesn’t have a such feature. In my observation, projects managed in Microsoft Project more often have “Incomplete activities in the past” issues and projects that are managed in Primavera more often have “Historical start and dates changed” issues.

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

Implementation of the Critical Path Methods in Different Scheduling Tools

Implementation of the Critical Path Methods in Different Scheduling Tools

Would the project dates calculated based on the critical path method be the same regardless of the applied scheduling tool? Not always!!!

Some time ago one large Telco company in Australia decided to replace their existing PPM system with ‘CA Clarity’. Clarity already was implemented in the organisation but used only as a corporate timesheeting system. The previous custom-made PPM system allowed to import Microsoft Project schedules and automatically retrieve dates for project status reports.

Clarity has integration with Microsoft Project, so the tool looked like a good alternative to the old PPM system. The portfolio initiated a project, which was a big disaster. The project manager made a dangerous assumption: “As both tools are based on the Critical Path Method, the schedules calculated in these tools are going to have the same dates”.

This was not the case. Imported and recalculated schedules had different delivery dates!!! The reality was even worse: the schedule imported back to Microsoft Project had different dates compared to the original schedule developed in Microsoft Project, even if the schedule was recalculated in Microsoft Project again!

In theory, the Critical Path Method is a well-defined method and a schedule calculated in any scheduling tool that supports this method should be the same.

In practice, projects consist of components: activities, calendars, constraints, dependencies, lags, resources, etc.

Scheduling tools have different features to manage and integrate project components. The way these features are implemented impacts the Critical Path Calculation.

When a schedule is imported between the tools some information could be completely lost as the new tool doesn’t have data fields to store this information.

In the described example, at that time (not sure if it is still the case) Clarity supported only resource calendars, not activity calendars. So, during the import, the information in activity calendars was lost and when the schedule was imported back to Microsoft Project, the project dates were different. Later, when I supported one of the Australian banks to intergrade Microsoft Project and Clarity, I found other differences between systems that impact schedule calculation.

Primavera, Microsoft Project, Spider Project and other project management tools support the critical path method they have unique features. The way these features are implemented may impact the Critical Path calculation!

Some examples of such features:

  • Primavera supports more than one dependency between the activities, and Microsoft Project doesn’t;
  • ‘Deadline’ in Microsoft Project impact Total Float calculation, Primavera doesn’t store ‘Deadline’ dates;
  • Primavera and Microsoft Project have different types of activity constraints;
  • Microsoft Project supports ‘% duration lags’, and Primavera doesn’t;
  • Spider Project takes into the calculation activity AND resource calendars, Primavera activity OR resource calendars;
  • Spider Project supports ‘Volume’ and ‘Duration’ types of activity, Primavera and Microsoft Project support only ‘Duration’ type;
  • Microsoft Project allows to link activities and summary tasks (WBS), Primavera only creat links between activities;
  • Primavera and Microsoft Project support Levell of Effort / Hammack activities, but this feature is implemented in a totally different way;
  • Primavera support the ‘WBS Summary’ type of activity, and Microsoft Project doesn’t;
  • Spider Project supports ‘Conditional’ scheduling, Primavera and Microsoft Project don’t.

Principles of the Critical Path Method could be explained based on a small simple schedule and calculation of such schedule in any scheduling tool should give the same result.

Real-life projects are much more complex. They have different calendars, hard and soft constraints, lags and leads, other than ‘Finish to Start type’ types of dependencies, etc. Implementation of these features impacts the way the critical path is calculated.

Some scheduling tools have methodological issues. Project’s real-life scenarios could not be properly simulated due to technical constraints. It is important to learn these constraints, find work-arounds or maybe replace the tool.

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

Monte Carlo Simulation Challenges. Data collection Challenge.

Monte Carlo Simulation Challenges. Data collection Challenge.

Terminology

What do the terms optimistic, pessimistic, and most likely really mean?
Because they are very different for different people, when one estimator says pessimistic, that person is thinking the technology might prove to be a bit more difficult than expected and thus take 10% longer. Another person in the team may assume that ‘pessimistic’ means that the work might be interrupted by an earthquake, tsunami, and nuclear meltdown simultaneously (even if it has not happened in the past).

Data Collection

Popular scheduling tools, like Microsoft Project and Primavera, don’t have native features to capture ‘3 points estimations’, so planners usually don’t capture estimations provided by Subject Matter Experts. A risk manager or a scheduler applies a range based on a single deterministic estimation instead. This approach is based on two dangerous assumptions:

      • All provided estimations are “Most Likely” estimations.

When SMEs are forced to provide a single estimation, they have to decide on how much contingency to include in the estimation. Some estimations in schedules have “optimism bias”, others, on the opposite, are too conservative.

      •  All activities have the same level of uncertainty.

There are many reasons why some of the activities may have dramatically different uncertainties: new technology, new equipment, new contractor, etc.  

Probabilistic scheduling tools, in the opposite, give an opportunity to capture activity durations as ‘3-point estimations’. Spider Project, arguably, the most advanced project delivery tool gives the opportunity to capture any type of initial data as 3 points: durations, cost, volume of work, resource productivity, lags, etc.

 Primavera and Microsoft project have custom fields that could be used to capture at least the duration of activities. Then weighted durations could be calculated. Both tools have built-in formulas. PERT or any other method could be applied for that.

Microsoft Project used to have built-in 3 points estimation fields but it’s current version doesn’t have this feature any more.

Objective vs Subjective

The Monte Carlo method works well when a lot of historical data is available. Unfortunately, the unique nature of projects means that likely historical data is going to be limited. However, even a small volume of historical data could be beneficial and portfolios have to implement methods to collect historical project data in a systemic way.  

 When historical data is not available the only way to collect estimations is to interview Subject Matter Experts (SME) and collect subjective information. However, it needs to be done in a smart way.

 It’s always better to ask a SME to provide an optimistic estimation first. Then ask the interviewee to think about the worst-case scenario and only after that provide the most likely estimation.

The trick is when the interviewee thinks about different possibilities, the most probable estimation is likely to be more accurate.

Hidden Risks

I found it extremely useful to collect data as a range even though I am not going to use it for Quantitative Risk analysis. It actually doesn’t take much longer to do so but gives me an opportunity to identify hidden risks. After all estimations are collected, some activities may have an abnormal range, comparing to the rest of the activities. It’s worth discovering the underlying reasons as there is often a story behind each abnormal range. Some of these stories get converted into risks or even issues.

Alex Lyaschenko

PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma

Scheduling Tools Comparison Analysis. Data Identifiers.

Scheduling Tools Comparison Analysis. Data Identifiers.

This post is a part of series of posts that explains the unique features and constraints of the most popular scheduling tools: Primavera, MS Project / Project Online, Spider Project and Project For The Web.

While each of the scheduling tools provides a lot of features to manage project delivery, it is hard to imagine a tool that covers all needs for any project, program or portfolio. The ability to integrate a scheduling data with other systems, in my opinion, is a critical function that supports (or limits) the maturity of project delivery.

This post provides a detailed review of data identifiers. Data identifiers are critical for user experience (UX) and also used for data integration within a tool or to external systems. 

Systems for integration

Scheduling tools are often required to integrate with other non-scheduling tools. The most popular non-scheduling tools that utilise scheduling data are:

Excel

It is hard to find a project manager or a scheduler who never copies scheduling data to Excel for analysis or reporting.

PPM tools

Scheduling data is a critical part of any PPM (Project Portfolio Management) system. Each scheduling tool included in this review is already a part of its own PPM system and also has the capability to be integrated with some other external PPM solutions.

Primavera has integration with Unifier, Oracle PPM solutions. Project Online is a base for any PPM solution developed on the SharePoint platform. Spider Project is powerful project portfolio management optimisation software. Project For The Web is a scheduling component of the new Power Platform.  would have built-in integration with any PPM system build based on this platform.

However, it could be a requirement to integrate scheduling data with an external PPM system. For example, some programs and portfolios include technology and infrastructure projects and may have more them one scheduling system rolled out.

Reporting tools

Tableau, Power BI and some other reporting systems are extremely popular now. These tools consume scheduling data for reporting and analysis to support project and portfolio decisions. These tools are not only able to represent data in a better way but also combine scheduling snapshots for deeper analysis and governance control.

Financial systems

Some organisations need direct integration between scheduling and financial systems.

Integration types

There are three methods for data integration:
• Screen manual copy-past;
• Export via different formats (xls, csv, xml, etc);
• Direct access to data tables also known as “backend access”.

Data identifiers

Project and schedule progress is a fundamental aspect of monitoring project performance over a specified reporting period. Progress is measured by completing a schedule comparison over two points in time.  The schedule, as a system, needs to identify the same data points (projects, milestones, activities, etc) and compare data attributes (dates, status, etc) from two comparison points or snapshots.

It is not reliable to use names (eg project name, activity name, resource name, etc.) as they are likely to change during the life of a project, so all tools have coding systems that are used as an “identifier” for data integration and comparison purposes. There are three types of data identifiers that users should be aware of:

1. Codes
A Coding system gives a user the opportunity to manually set up and manage scheduling data. Data codes are unique and editable.

2. System identifier
System identifiers are automatically created and are unique but not always accessible to users. While the main function of system identifiers is to ensure data integrity, it is often useful to get access to these identifiers as they could be used for data governance control.

Example:
A user by mistake swaps two activity codes. A slippage from the previous month would not be calculated correctly as incorrect data points used for the calculation, and it is hard to notice. If activities haven’t been recreated it is possible to generate an exception report based on a system identifier (if available) that shows such a problem.

3. Screen identifiers
Screen identifiers simplify the user experience by showing activity orders on a screen. While it is convenient to use the ID within a system it is not reliable to use it outside of the system for data comparison as it is likely to change.

The lack of consistency in implementing identifiers and applied terminology between scheduling systems causes confusion and may lead to incorrect analysis & reporting.

Let’s review the capability of each scheduling system included in comparison analysis.

Primavera

1. Codes
Primavera has a built-in coding system at project (Project ID) and activity levels (Activity ID). The system automatically controls the uniqueness of data in each record.

2. System identifier
Unfortunately, Primavera doesn’t provide access to project and activity system ID via front ends (client and web). However, the system IDs are available via the backend. Also, there is a trick to access them via XER file but this is a manual and time-consuming process.

3. Screen identifier
Primavera doesn’t have screen identifiers.

Project Online

1. Codes
Unfortunately, Project Online doesn’t have a built-in coding system.

2. System identifier
MS Project provides front-end and backend access to system identifiers at the activity level (UID) and project level (Project ID)

3. Screen identifier
MS project has very convenient screen identifiers and uses them not only to display record numbers but also predecessor and successor columns. It makes MS Project more user friendly but also creates challenges to control data quality as screen identifiers are not stable.

    Workarounds

    There are two workarounds usually applied to solve the lack of a coding system. Each of them has pros and cons.

    UID: Unique but not editable.
    Some users use activity level system identifiers (UID) and project level system identifiers  (Project ID) instead of codes. However, if a user recreates an activity or uses a copy of the schedule (after what-if analysis) the IDs are going to be different.

    Custom field: editable but not unique.
    MS Project allows to set up custom fields on both, project and activity, levels. The custom fields are editable, so users are able to recover changes but the uniqueness of values must be controlled manually. This may be not an issue for small schedules and ma be quite a significant challenge for larger program and portfolios.

    Spider Project

    1. Codes
    Spider Project has a built-in coding system in all project tables including a list of projects and activities. The system automatically controls the uniqueness of all codes.

    2. System identifier
    Spider Project provides front-end and back-end access to system identifiers.

    3. Screen identifier
    Spider Project has screen identifiers and shows codes (not screen ID) in predecessor and successor columns.

    Spider Project also has a built-in version control. It is possible to save a new version of the schedule manually or it’s saved automatically when project as actual project data is added.

    Project for the Web

    1. Codes
    Project For The Web doesn’t have a built-in coding system.

    2. System identifier
    Project for the web doesn’t provide front-end access to system identifiers.

    3. Screen identifier
    Screen identifiers are viewable on screen but have no export facility.

    Recently Microsoft implemented the “custom fields” feature and now it could be used as a workaround. However, the uniqueness of values must be controlled manually.

    Summary

    The four scheduling tools examined here all have different capabilities to integrate scheduling data with other systems and each provides a different user experience.

    Primavera has a built-in coding system that controls the uniqueness of scheduling data (activities, schedules, resources) but doesn’t have screen identifiers nor does it provide access to system identifiers.  

    MS Project, on the other hand, doesn’t have a built-in coding system but does have screen identifiers and offers access to a system identifier (UID).

    Project for the web only provides a screen identifier, although does have a backend system identifier.

    Spider Project has a built-in coding system that controls the uniqueness of scheduling data (activities, schedules, resources), has screen identifiers and provides access to system IDs.

    System IDs are always accessible via the backend for portfolio solutions.

    All of this ultimately enables true integration with PPM and other tools requiring schedule data.

    Alex Lyaschenko

    PMO | Portfolio Planning & Delivery | PMP | P3O Practitioner | AgilePM Practitioner | Six Sigma