DoD Journal of Software Technology (formerly DoD Software Tech News)
August 2011 "Improving the Quality of EVM Information"
An article in the July issue of the DoD Journal of Software Technology provides tips for “Improving the Quality of Earned Value Management Information.”
Learn how to assess the quality of contractor-reported EVM information and to detect when EV is unreliable or inaccurate and EAC is understated. DCMA’s
assessment that a contractor is EVMS-compliant can provide false assurance to a Program Manager (PM) that the EVM data is accurate (Immaculate
Misconception). There are significant loopholes and ambiguities in the EVMS guidelines (EVMS Quality Gap) that enable contractors to overstate EV. The article
describes common contractor practices that produce misleading EV information even when the contractor is compliant with ANSI/EIA-748 Guidelines 7
(Measure Progress), 14 (Management Reserve (MR)), and 30 (Retroactive Changes).
Finally, until the Quality Gap is closed by needed EVM acquisition reform (Link: EVM Acquisition Reform ), PMs should take remedial actions to see through
the Immaculate Misconception and improve the quality of EVM information that they use. These actions include:
Focus on the technical baseline and technical performance measures (TPM)
Require effective requirements traceability to the supplier's Performance Measurement Baseline (PMB)
Use the IMP to push technical success criteria and TPMs to the IMS and PMB
Verify the integration of technical objectives with cost and schedule objectives during the IBR and major technical reviews
Use IMS success criteria to define and control incremental functionality for planned software builds
Recognize when the supplier does not account for deferred functionality
Use program systems and software engineering staff to bolster EVM surveillance
Monitor supplier's reporting of rework
Detect misuse of MR
Independently assess EV based on known technical performance and achieved functionality.
Link to article: Improve Quality of EVM Info
Sept. 2009 "Agile EV and the Technical Baseline" Link: Agile EV article
April 2009 "Applying EVM to Software Intensive Programs" by Bob Hunt (Galorath Inc.) , Dan Galorath (Galorath Inc.), and Paul Solomon (Performance-Based Earned Value® ) Link: Applying EVM article
Per the article, the proper use of PBEV provides for integration of project technical scope, schedule, and cost objectives; and the establishment of a baseline plan for performance measurement. The article provides guidance for improving program performance by implementing four processes:
Develop the technical, cost, and schedule baselines
Identify critical software management metrics
Apply Performance-Based EV
Use an analytic process to project cost and schedule based on actual performance
Defense AT&L Magazine
"Integrating Systems Engineering with Earned Value Management" May/June 2004, page 42. Link: Defense AT&L, May 2004
“Integrating Technical Performance with Earned Value Management” The PMI Measurable News, Issue No. 4, page 6.
“Performance-based EV in Commercial IT Projects" The PMI Measurable News, 2010 Issue No. 2, page 1.
Link: Measurable News, 2010 Issue 2
“Integrating Systems Engineering with EVM" The PMI Measurable News, Fall 2008, page 7.
Link: PMI Meas. News, Fall 2008 “Integrating Technical Performance with Earned Value Measurement” The PMI Measurable News, Spring 2008, page 15.
“Integrating Risk Management with Earned Value Measurement (Risk Management Comes Out of the Closet)” The PMA Measurable News, June 1998, page 11.
Link: PMA Meas. News, June 1998
“From Performance-Based Earned Value to the CMMI” The Measurable News , March 2002
Defense Acquisition Research Journal
INCOSE, International Council on Systems Engineering
"Using Earned Value to Track Requirement Progress," INCOSE 2006 Link: INCOSE Using EV to Track Requirements Progress
PBEV Webinar, sponsored by DOD Data and Analysis Center for Software (DACS)
1 hr. show and tell: How PBEV overcomes deficiencies of EVMS, application, basis in Systems Engineering standards.
Also downloadable PDF file.
CrossTalk, the Journal of Defense Software Engineering "Performance-Based Earned Value” Aug. 2005
http://www.stsc.hill.af.mil/crosstalk/2005/08/index.html
Carnegie Mellon U. Software Engineering Institute (SEI)
U.S. Army Practical Software and Systems Measurement (PSM)
"Technical Measurement Guide V1.0"
Provides information on implementing technical measurement on a project, including Measures of Effectiveness (MOEs), Key Performance Parameters (KPPs), Measures of Performance (MOPs), and/or Technical Performance Measures (TPMs).
See pages 34 and 48 or guidance on linking TPMS to the WBS, schedule, and EVM. URL: PSM Technical Measurement Guide V1.0
"Systems Engineering Leading Indicators Guide" initiated by PSM, INCOSE, MIT
See page 68, Technical Measurement Trends URL: SE Leading Indicators Guide
USAF Space & Missile Systems Center
"SMC-S-001 Systems Engineering Requirements and Products" Air Force SMC Systems Engineering Primer & Handbook
https://acc.dau.mil/CommunityBrowser.aspx?id=25838&lang=en-US
USAF
PMI Project Management Journal, March 2008
"Best Project Management and Systems Engineering Practices in the Preacquisition Phase for Federal Intelligence and Defense Agencies" by Steven R. Meier
(available only to PMI members). Root causes of cost and schedule growth in large programs. Best practices during preacquisition phase include a firm program requirements baseline and strong systems engineering to bind the technical solution to the requirements and to maintain the program baseline.
Government Accountability Office (GAO)
Reports on EVM Deficiencies
GAO-10-2 INFORMATION TECHNOLOGY-Agencies Need to Improve the Implementation and Use of Earned Value Techniques to Help Manage Major System Acquisitions
GAO assessed agencies’ EVM policies to determine whether they are adequately using earned value techniques to manage key system acquisitions, and evaluate selected investments’ earned value data to determine their cost and schedule performances. For 13 of 16 investments in 8 agencies, GAO found that key practices necessary for sound EVM execution had not been implemented. These investments face an increased risk that managers cannot effectively optimize EVM as a management tool.
http://www.gao.gov/new.items/d102.pdf
GAO-09-96 HOMELAND SECURITY U.S. Visitor and Immigrant Status Indicator Technology Program (US-Visit) Planning and Execution Improvements Needed (See summary on New tab)
GAO-08-756 Air Traffic Control: FAA Uses EV Techniques to Help Manage IT Acquisitions, but Needs to Clarify Policy and Strengthen Oversight GAO-08-448 Defense Acquisitions – Progress Made in Fielding Missile Defense, but Program Is Short of Meeting Goals (See summary on New tab) GAO-08-388 Joint Strike Fighter - Recent Decisions by DOD Add to Program Risks
GAO reported deficiencies in Lockheed Martin's EVM system (See summary on New tab)
GAO-08-294 Increased Focus on Requirements and Oversight Needed to Improve DOD’s Acquisition Environment and Weapon System Quality (See summary on New tab)
GAO-08-345 Office of Personnel Management (OPM) Improvements Needed to Ensure Successful Retirement Systems Modernization (See summary on New tab)