Performance-Based Earned Value
Defense AT&L Magazine/Defense Acquisition Magazine
"Integrating Systems Engineering with Earned Value Management", May 2004
EV quantitatively linked with:
Technical performance measurement (TPM)
Progress against requirements
but EVMS Standard states that EV is a measurement of the quantity, not quality, of work accomplished.
EVM can be more effective as a program management tool if it is integrated with technical performance and if the EVM processes are augmented with a rigorous systems engineering process.
"Systems Engineering (SE) and EVM Support for Performance-Based Awards," Jan. 2007 SE and EVM article atl 2007.pdf
Provides practical advice for defining the technical performance requirements and desired program outcomes in SE terms.
EV can also be a valid basis for award fee determination if it is tied to technical performance, not just to work accomplished.
"Earned Value Management Acquisition Reform", Nov. 2010 Link to PDF
DoD reported to Congress that EVM no longer serves its intended purpose as a management tool. Pending legislation addresses EVM acquisition reform. Program Managers can get more value from EVM by linking EV to technical performance. Guidance to integrate cost, schedule and technical objectives is not provided by the EVM Standard, EIA-748. Project management standards and best practices that are used by commercial companies should be considered for acquisition reform.
Major DoD acquisitions require contractor’s processes to comply with ANSI/EIA-748. However, a contractor may be compliant with the ANSI/EIA-748 guidelines but fail to link EV to technical performance or quality. The implementation and management value of EVM have been strongly criticized by DoD, the GAO, and Congress...Commercial corporations that use EVM do notbase their best practices on EVMS, EIA-748.It is time to ask whether DoD, and other federal agencies, should continue to rely on EIA-748 or should adopt the best practices of commercial companies that use EVM voluntarily, not because of a contractual mandate.
When there is no government mandate to use EVM, the Project Management Institute (PMI) Guide to the Project Management Body of Knowledge (PMBOK® Guide) is a widely used standard for project management.
Some examples of compliant practices that led to misleading management information and that would not be permitted if the Quality Gap were closed are:
· Taking EV based on percent of drawings or software modules complete even though the hardware design did not meet requirements or the software did not meet planned functionality.
· Including budget and schedule for tests and rework in Management Reserve instead of in the initial PMB, work packages, and planning packages.
· Taking EV for rework and engineering changes based on the actual vs. estimated percent of units, iterations, or problem reports instead of on the percent of requirements met.
· Taking EV for software releases based on turning over the release, even though some of its baselined functionality was deferred to the next release.
· Not taking negative EV to show the true, net percent complete when the number of drawings or other units increased from the baselined number, with no change in the technical requirements.
· Not taking EV for drawings or other units returned for rework, when rework is planned in the same work package as the initial work.
"Path to EVM Acquisition Reform," May 2011Link to PDF
DoD should consider revising its DoDI 5000.02 and DFARS to require that earned value be linked to technical performance or quality, not just to the quantity of work performed. The quality objectives should be defined in the technical baseline and linked with the Performance Measurement Baseline.
The EVMS sections of DFARS should be changed to add “product scope” to work scope, and to require that the use of TPMs to measure progress be mandatory, not optional.
"A Contract Requirement Rule for Program Managers," Nov. 2015
Recommends replacement of EVMS Standard EIA-748 with Project Management Institute Standard, PMBOK® Guide. (See PMBOK® Guide Excerpts)
EVM, based on EIA-748, is used primarily by federal contractors when contractually required. A more powerful tool is the ANSI standard that is used world-wide on a voluntary basis when there are no federal requirements. It is the PMBOK® Guide.
A Program Manager’s needs that are covered by PMBOK® Guide but are absent in EIAI-748 include:
· Technical or product baseline
· Requirements management and traceability
· Risk management
PMBOK® Guide contains many subjects that are absent from EIAI-748 including:
· Product scope description - Documents the characteristics of the product that the project will be undertaken to create. Progressively elaborates the characteristics of the product…described in the project charter and requirements documentation.
· Project scope – The work that needs to be accomplished to deliver a product..with the specified features and functions
· Requirements Documentation - Requirements baseline; unambiguous (measurable and testable), traceable, complete, consistent, and acceptable to key stakeholders. Components include, functional requirements, non-functional requirements, quality requirements, and acceptance criteria.
· Requirements Management Plan – include…product metrics that will be used
· WBS Dictionary includes quality requirements, acceptance criteria
· Scope Baseline includes product scope description, project deliverables, and defines product user acceptance criteria
· Control Scope - Process of monitoring the status of the project and product scope and managing changes to the scope baseline. Completion of the product scope is measured against the product requirements.
· Requirements Traceability Matrix (RTM) – includes requirements to project (including product) scope/WBS objectives, product design, test strategy and test scenarios
· Conduct risk management planning, identification, qualitative risk analysis, quantitative risk analysis, response planning, and controlling risk.
PMBOK® Guide also covers EVM topics including scheduling (including network diagrams), PMB, control accounts, work packages, earned value, variance analysis, estimate at completion, and management reserve.
Better Program Management Through Digital Engineering, page 32, May/June 2022
CrossTalk, the Journal of Defense Software Engineering
"Basing Earned Value on Technical Performance," Jan. 2013
Recommends contract language and project monitoring techniques to ensure that contractors integrate technical performance and quality, including software functionality, with EVM. Key enablers are a contractually-required IMP and linkage to systems engineering work products and best practices. Proposes tailored EVMS guidelines.
Require that anIMP be a contract deliverable.
Implementation of the recommended acquisition management processes and new contractual requirements will provide the following benefits:
• Close the EVMS Quality Gap
• Insightful IBRs and technical reviews
• Valid contract performance reports - Objective technical/schedule status - Credible EAC
• Early detection of problems - Program performance - EV measurement and compliance
Note: Contents of article in tutorial format: NDIA SE Conference, Oct. 2012, Tutorial, "Integrating SE with TPM"
“Practical, Performance-Based Earned Value” May 2006
DoD guidance on systems engineering (SE), basing earned value on measures of technical progress, on the progress of requirements management activities, and on the entry and exit criteria for technical reviews.
Example 1: EV based on completing drawings and meeting requirements.
Example 2: EV when the TPM for a subsystem is at a higher WBS level than its components
Example 3: Progress of systems engineering (SE) effort
The SE plan (SEP) describes the approach for controlling the overall technical effort of the program, including the technical baseline control and requirements management, traceability, and requirements verification. This example demonstrates a method for measuring progress of the SE effort to perform requirements management, traceability, and verification.
Example 4: Exit criteria event-driven technical reviews
The exit criteria in the SEP should be the completion criteria for work packages that map to the reviews. The example uses the exit criteria for a system-level detailed (critical) design review, based on the SE standard, Institute of Electrical and Electronics Engineers (IEEE) 1220-1998.
“Practical Software Measurement, Performance-Based Earned Value” Sept. 2001
Integrated, performance measurement techniques that increased Northrop Grumman Corporation’s software success enable excellent project management in the following ways:
• Defining effective, outcome-based metrics for sizing the project and measuring progress.
• Defining quality goals in terms of project milestones for needed capability and metrics.
• Planning for incremental releases and rework.
• Revising the plan for deferred functionality and requirements volatility.
• Focusing on requirements, not defects, during rework.
• Requirements traceability: a disciplined requirements traceability process and a requirements traceability data base.
• Using testable requirements as an overarching progress indicator.
To ensure the acceptance of the end product and enable consistent performance measurement, allocated requirements should be testable and traced to detailed specifications, software components, and test specifications.
College of Performance Management (CPM) Measurable News
“Integrating Systems Engineering with Earned Value Management, Part 2," 2016 Issue No. 4, page 36
Contractors are not integrating TPM with EVM and are not required to use processes that would enable a Program Manager to adhere to DoD instructions and guidance regarding TPMs and SE. Why and what is the remedy?
Although the topics and recommendations herein address the needs of DoD, they are applicable to all high tech, high risk development projects, including government and commercial, and are relevant to the College of Performance Management’s (CPM) objective to lead EVM into the next generation – Integrated Program Performance Management (IPPM).
"Integrating Technical Performance Measurement with Earned Value Management,” 2010, Issue 4, by Neil Albert and Wayne Abba
“Performance-based EV in Commercial IT Projects" , 2010 Issue No. 2, page 1
“Integrating Risk Management with Earned Value Management (Risk Management Comes Out of the Closet)", June1998
Journal of Software Management
"Agile Earned Value and the Technical Baseline ," Sept. 2009, page 9
Define Baselines for each Build (Technical, Schedule, Cost)
Once the Product Baseline is approved, establish the schedule in the IMS and the cost baseline in the EVM database… When planning incremental builds, allocate the functional requirements in the Product Baseline to each build in each block. Document each build’s technical baseline.
Deferred Functionality
For valid reporting of project status, EV should reflect the results of deferring functionality from its baselined iteration, build, or block. When functionality is deferred from the current iteration to the backlog,…the deferral has the following major impacts: If all the requirements planned for iteration are not completed, then the EV for the deferred requirements cannot be earned as part of the iteration. It is behind schedule.
“Improving the Quality of EVM Information ," July 2011, page 32.
PMs rely on DCMA to assure that the supplier’s EVM data is reliable and accurate for decision making purposes. However, even if DCMA reports that the supplier is EVMS-compliant, the EV information is often inaccurate and misleading. This reliance on DCMA’s blessing is an immaculate misconception...A PM is vulnerable because of the Quality Gaps in Intent Guidelines 7, 14, and 30.
DoD Systems and Software Technology Conference, SSTC Conference, April 2009
Tutorial 1, "Integrating SE with EVM"
•Require SE best practices in Request for Proposal (RFP)
•Confirm contractor’s proposal includes integration of SE with EVM
•Verify integration in Integrated Baseline Review
•Confirm achievement of success criteria in technical reviews
•Monitor consistency and validity of status reports and variance analyses
Tutorial 2, "Agile Methods with Performance-Based Earned Value”
Carnegie Mellon U./Software Engineering Institute
Technical Note CMU/SEI-2002-TN-016, Oct. 2002 "Using CMMI® to Improve Earned Value Management"
Includes mapping and comparison tables between CMMI and the EIA-748, the EVM System standard (EVMS) that can be used to identify practices and work products within CMMI that are not included in EVMS but, if added to an organization's processes, will close the Quality Gap in their EVM system descriptions.
The following process areas have specific practices and informative model components that address control of a project’s technical, schedule, and cost objectives or objective assessment of accomplishments but have no counterparts in EVMS:
• Requirements Management
• Measurement and Analysis
• Process and Product Quality Assurance
• Requirements Development
• Risk Management
The criteria for declaring that a work package is complete may include quality and technical criteria, not just a measure of the quantity of work accomplished.
Airforce Acquisition Reform Newsletter, Aerospace Acquisition 2000, January/February 2000, article,“B-2 Bomber Team’s EVM Innovations Make Program Soar"
Naval Postgraduate School Lecture,
Monterrey, March 11, 2020
"Integrate SE with Earned Value Management and Program Management, Contractually and Practically"
Download PDF file of lecture
NDIA Systems and Mission Engineering Conference, Tampa, Oct. 21, 2019
Tutorial, "Integrate SE with Earned Value Management and Program Management, Contractually and Practically"
You will improve your technical competencies to meet the objectives of the Program Management Improvement and Accountability Act of 2015 (PMIAA) and achieve better program outcomes. Training examples are provided to show how PMs can require contractors to integrate SE with EVM, link EV to technical performance measures (TPM), and increase focus on the product scope instead of the work scope. Recommended contract terms to make it happen will be discussed.
Download abstract for detailed information.
Download PDF file of full tutorial .
Intro: In 2009, DoD reported to Congress that Systems Engineering (SE) and Earned Value Management (EVM) should be integrated, not stove-piped. It has not happened. This tutorial provides a comparison of pertinent SE, EVM, and integrated program/project management (P/PM) standards that can be used to make it happen as well as practical examples for implementation.
Abstract: The role of SE in the PM process should have been strengthened because of the Weapon Systems Acquisition Reform Act of 2009 (WSARA), sponsored by John McCain. DoD’s report to Congress, required by WSARA, included four statements regarding SE, technical performance and EVM. DoD stated that “EVM can be an effective program management tool only if the EVM processes are augmented with a rigorous SE process and SE products are costed and included in EVM tracking.”
This tutorial will provide a path for DoD to make it happen. Training examples are provided that show how to integrate SE work products with EVM and link EV to technical performance measures (TPM). Recommended contract terms, based on provisions in SE and P/PM standards, will be discussed.
You will learn techniques to ensure that the accomplishment criteria for the key technical reviews are traceable from the technical baselines to the Integrated Master Plan (IMP) to the schedule to the work packages.Consequently, EV can be based on objective progress towards:
1. Developing and validating the requirements, including trade studies
2. Defining and allocating key TPMs,
3. Achieving TPM planned values and planned functionality.
The following topics will also be illustrated with best practices, examples, and useful templates:
- Using the IMP to communicate required technical accomplishment criteria
- Using the SE Management Plan (SEMP)
- Using the Integrated Baseline Review to ensure that the Contractor has integrated SE work products and technical performance into the IMS and work packages, Verifying that work package interim progress and completion are traceable to the status of requirements in the requirements data base,
- Discretely planning and measuring rework,
- Accounting for deferred functionality.
- Agile methods and EV