Performance-Based Earned Value
Paul Solomon, is the co-author of the book, Performance-Based Earned Value® and is an advocate of acquisition reform in the Federal Government with regard to Earned Value Management System (EVMS) and project management. He is internationally recognized as a leader and teacher of EVM. He published many articles on EVM, systems engineering (SE), software metrics, and risk management. His recommendations were included in voluntary consensus standards, in DoD policy and guides, and in federal legislation.
He retired from Northrop Grumman Corporation where he supported the use of EVM on the B-2 Stealth Bomber, Global Hawk, F-35 Joint Strike Fighter, and MP-RTIP programs. He was certified by the Project Management Institute (PMI) to be a Project Management Professional (PMP).
Paul was on the team that wrote EIA-748, the EVMS Standard, and received the DoD David Packard Excellence in Acquisition Award for that achievement. He was a core member of the team that wrote the "PMI Practice Standard for EVM" (predecessor to "PMI Standard for EVM (ANSI/PMI 19-006-2019)." He was a visiting scientist at the Carnegie Mellon University/Software Engineering Institute where he authored the 2002 paper, "Using CMMI to Improve EVM" (CMMI). His original recommendations have been included in the following PMI standards, DoD policies and guides, the GAO Cost Estimating and Assessment Guide (GAO Guide), and the College of Performance Management (CPM) Integrated Program Performance Management (IPPM™) system:
1. "PMI Project Management Body of Knowledge" (PMBOK® Guide 5th ed.) and "PMI Practice Standard for EVM" (predecessor to ANSI/PMI 19-006-2019, The Standard for EVM): product scope, risk management, management reserve, acceptance criteria (in WBS dictionary), requirements traceability matrix, and Estimate at Completion
2. "DoD Instruction 5000.02, Acquisition of the Defense Acquisition System": technical performance
3. "DoD Engineering Guidebook" CH 2.5 SE Role in Contracting
The PM should ensure that the EVMS, tied to any incentive, measures the quality and technical maturity of technical work products instead of just the quantity of work. If contracts include earned value (EV) incentives, the criteria should be stated clearly and should be based on technical performance. EV incentives should be linked quantitatively with:
Technical performance measurement (TPM)
Progress against requirements
Development maturity
Exit criteria of life-cycle phases
Significant work packages and work products
4. "NAVAIR Software EVM Toolkit," 2004: technical performance, deferred functionality, and rework
EVM has often not been successfully implemented on software development efforts. There are several reasons why this is so:
Crediting full-EV for tasks and requirements even though all tasks and requirements have not been completed.
Basing EV on metrics and measures that do not directly relate to implementation of the software requirements.
Basing EV on metrics and measures that are obsolete or inaccurate.
Utilizing EVM in isolation vice in conjunction with other software measurements and metrics to evaluate program status.
Failure to consider rework in developing the Performance Measurement Baseline (PMB).
Failure to correlate EV value with TPM.
An effective EV system must account for deferred functionality if it is to accurately reflect program status and progress. In order to do this, the system or software requirements planned to be implemented in each software development artifact or phase must be considered in determining EV.
Rework should be planned and tracked in separate work packages from the initial development of requirements, design and code. In planning incremental builds, all builds must include budget and schedule for rework of requirements, design and code to correct defects that were found in the current and previous builds. To ensure adequate budget and period of performance, the planning assumptions for rework should include the planned rate or number of defects expected and the budgeted resources to fix the defects. Failure to establish a baseline plan for rework and to objectively measure rework progress has caused many projects to get out of control.
5. "DoD EVMS Interpretation Guide" (EVMSIG):
Guideline 1, Traceability matrix from government requirements (e.g., SOW, Build Specifications) to WBS
Guideline 6, When an Integrated Master Plan (IMP) is contractually required, the schedule structure should correlate with the information in the IMP.
Incorporation and progress of risk management activities and mitigation actions.
Evidence of risk mitigation handling plan in the IMS.
Guideline 7, Objective completion criteria aligned with the accomplishment of the program’s technical requirements and goals are determined in advance, documented, and used to plan and measure the progress of program milestones and events. Objective technical performance goals and measures are incorporated throughout the schedule hierarchy based on the completion criteria developed for each increment of work, in order to limit subjective measurement of work accomplished. Objectively measured performance data that accurately reflects technical accomplishment of the work provides program management visibility into program progress and credible early indications of program problems and the need to take corrective action.
Guideline 27, The program EAC accounts for dollarized risks and opportunities that are related to the risk management process.
6. PARCA Report to Congress on Performance Assessments and Root Cause Analyses, May 2014
the PARCA EVM Division will identify, document, and publish specific methods for relating technical performance to earned value performance. The goal is to provide more accurate joint, program office, and contractor situational awareness of the program execution. PARCA believes that earned value metrics and technical metrics such as Technical Performance Metrics should be consistent with program progress. Earned Value focuses on the completion of a set of tasks to mature the design. It should be consistent with the set of metrics that indicate the actual design maturity.
7. "Section 809 Panel Report, Vol. 1, Sec. 4, Jan. 2018": Another substantial shortcoming of EVM is that it does not measure product quality. A program could perform ahead of schedule and under cost according to EVM metrics but deliver a capability that is unusable by the customer."
8. "Agile and EVM: A Program Manager’s Desk Guide,” April 16, 2018 OUSD AT&L (PARCA)
Consistent with:
a. Journal of Software Management, "Agile Earned Value and the Technical Baseline," Sept. 2009, page 9
b. DoD Systems and Software Technology Conference, April 2009
Tutorial, “Agile Methods with Performance-Based Earned Value”
9. GAO Cost Estimating and Assessment Guide, March 2020, Expert Contributor
Guidance on incorporating risk mitigation plans into budgets and using earned value to monitor performance of risk mitigation plans is consistent with 1998 article in The Measurable News, "Integrating Risk Management with EVM (Risk Management Comes Out of the Closet)." Guidance to use the EVM data...to integrate cost and schedule performance data with technical performance measures is consistent with the 2004 article in Defense AT&L Magazine, "Integrating Systems Engineering with Earned Value Management." Guidance on budgeting anticipated rework in a separate work package or planning package, not MR, is derived from the 2004 NAVAIR Software EVM Toolkit."
10. CPM's system, Integrated Program Performance Management (IPPM™) redefines EVM by building on its founding principles and by adding subject matter to fill gaps necessary to properly address Technical Performance Measurement (TPM) and integration of other systems engineering (SE) principles and disciplines. These subjects were first presented at the CPM Conference in May 2001, "Integrating SE, Risk, and EVM." The subjects were first published in the CPM Measurable News, in May and October 2008, and, most recently in “Integrating SE with EVM, Part 2," 2016 Issue No. 4, page 36.
11. DOD Best Practices for Using Systems Engineering Standards (ISO/IEC/IEEE 15288, IEEE 15288.1, and IEEE 15288.2) on Contracts for Department of Defense Acquisition Programs, April 2017
"The offeror shall, as part of its technical proposal, provide a Systems Engineering Management Plan (SEMP), Integrated Master Plan (IMP), and Integrated Master Schedule (IMS)." From PARCA-EVM ASSESSMENT STUDY, 2012. The Parca-EVM Assessment Study is summarized in the article in CrossTalk, the Journal of Defense Software Engineering, "Basing Earned Value on Technical Performance," Jan. 2013.
12. The DoD Directive 5000.01 policy, “Performance-based strategy” means a strategy that supports an acquisition approach structured around the results to be achieved as opposed to the manner by which the work is to be performed,” was first proposed in his Technical Note, CMU/SEI-2002-TN-016, Using CMMI to Improve Earned Value Management, Paul Solomon, Oct. 2002, published by Carnegie Mellon U./Software Engineering Institute. In 2012, this strategy was included in his REPORT PARCA-EVM ASSESSMENT STUDY CONTRACT #: HQ0034-12A0011-0001, within RECOMMENDED REVISIONS TO DOD ACQUISITION POLICIES, GUIDES, CONTRACT DATA REQUIREMENTS, AND DFARS.
13. GAO Agile Assessment Guide, GAO-24-105506, Nov. 28, 2023
Includes references to four articles and tutorial, especially
Solomon, Paul J. “Agile Earned Value and the Technical Baseline” Managing for Success. The Data & Analysis Center for Software. (September 2009.): requirements traceability matrix, baseline changes
Tutorial: Integrated Systems Engineering (SE) with Earned Value Management and Program
Management, Contractually and Practically. NDIA Systems Engineering Conference. Tampa, Florida: October 22, 2018.:
“Basing Earned Value on Technical Performance.” CrossTalk. (January 2013.): Rework
"It is a best practice, though, to ensure the customer and product owner are communicating on priorities and the balance between scope, schedule, and budget so that minimum viable product functionality requirements are met."
"Ultimately, EVM is effective for Agile programs when it is integrated with technical performance and EVM processes are augmented with a rigorous systems engineering process"
is from a 2004 article in Defense AT&L Magazine, "Integrating SE with Earned Value Management."
14. DODI 5000.97 DIGITAL ENGINEERING (DE)
DoD will use DE methodologies, technologies, and practices across the life cycle of defense acquisition programs,… engineering, and management activities.
activities, including, but not limited to: cost estimating, WBS, schedules
15. DOD MIL-HDBK-245E, DOD Handbook, Preparation of Statement of Work (SOW Handbook). The revision to SOW Handbook, added and defined the product scope (technical baseline) and differentiated it from the project (or work) scope.
He served on the National Defense Industrial Association Program Management Systems Committee (now called Integrated Program Management Div.) and was Vice President. His recommendations on EVM Acquisition reform were included in the Ike Skelton National Defense Authorization Act (NDAA) for FY 2011, the NDAA for FY 2024 (Sec. 815), the NDAA for FY 2025 (Sec. 804) and the 2009 DoD EVM: Performance, Oversight, and Governance- Report to Congress on EVM (in response to WSARA). His recommendation that standards for program and project management must be in accordance withh ANSI standards was included in the House version of the NDAA for FY 2021.
In his 2010 article, "EVM Acquisition Reform," he first recommended that EIA 748 be replaced with the Project Management Institute (PMI) Guide to the Project Management Body of Knowledge (PMBOK® Guide). That recommendation was incorporated in the Program Management Improvement Accountability Act of 2015 (PMIAA). PMIAA requires OMB to "Establish standards and policies for executive agencies consistent with widely accepted standards for program and project management planning and delivery." PMBOK® Guide is the only standard that qualifies. His similar recommendation to House Armed Services Committee Chairman Adam Smith was incorporated into the House version of the National Defense Authorization Act (NDAA) for FY 2021, H. R. 6395. His recommendation to discontinue the requirement to use EVMS per EIA-748 was incorporated into NDAA for FY 2024, Section 815, with regard to software contracts.
Excerpt from article: "EVM is still recognized as an international, commercial best practice, but EIA-748 has been largely ignored by commercial companies. When there is no government mandate to use EVM, PMBOK® Guide is a widely used standard for project management. One way to implement reform is to replace EIA- 748 with pertinent components of the PMBOK® Guide." Nov 2010 atlpublished.pdf
He filed a False Claims Act (whistleblower) complaint against Lockheed Martin and Northrop Grumman, alleging fraud on the F-35 Joint Strike Fighter program, in 2012.
He received a letter of appreciation for continuing efforts to improve acquisition reform from Sen. John McCain on March 5, 2015.
Education and Certifications:
B.A. degree from Dartmouth College. M.B.A. degree from Tuck School of Business Administration at Dartmouth. Honors graduate of the Air Force Institute of Technology Cost/Schedule Control Systems Criteria course, 1990.
Military Service: Vietnam War Era Veteran, U.S. Army Finance Officer.