Performance-Based Earned Value ®

PBEV = EVM + Quality
Home     New     Basic EVM     Advanced: PBEV articles     Dept of Defense     EVM Acquisition Reform     EVMS Quality Gap     ANSI 748 Refresh     PBEV the Book     Consulting     PBEV Live Training     About us     Risk Management     OMB policy, FAR, DFARS     CMMI     Contact Us     Proposed rev:DoDI5000.02      
DOD Guide Matrix
Technical Performance
Technical Baseline
DoD EVM Implementation
SE Leading Indicators

< See also sublinks to left.

 

August 2011. Article in DoD Journal of Software Technology, "Improving the Quality of EVM Information." Link: DACS Improve Qual EVM Info

 

5/1/2011:

Defense ATL Magazine, May-June 2011, published by Defense Acquisition University, includes the article "Path to Earned Value Management Acquisition Reform."
Topics:
  • Review of DoD acquisition guidance with regard to technical performance and EVM
  • Recommended changes to OMB policy, FAR, and DFARS to close the Quality Gap in ANSI/EIA-748
  • Why DoD can’t implement its own policy requirements and legislative requirements without EVM acquisition reform
  • Why compliance with some EVMS guidelines is non-value added

 

8/10/2011: DoD Letter on PARCA EVMS Performance, Oversight, and Governance Link: PARCA Authority Letter

 

6/3/2011: DoD Memo: Test and Evaluation of Defense Programs.Link: Test and Evaluation memo

addressed need for well-defined, testable requirements and the following issues:

 

1.  Requirements development must be informed by technical feasibility and rigorous trade-off analysis.

2. Define requirements in ways that are clear and testable,…should be achieved as early as possible.

3. Define requirements in ways that provide meaningful increments of operational capability.

4. Define requirements in ways that enable efficient program execution.

 

Guidance to incorporate the above needs in the EVM PMB is provided in several articles and a tutorial within this web site.

         Testable requirements CrossTalk 2001

         Incremental (and deferred) functionality, PMI Meas. News, Spring 2008, page 15  PMI Meas News, Spring 2008

        Program execution (measure progress of the SE effort to perform requirements management, traceability, and verification) CrossTalk 2006 (Example 3)

         Trade studies (discrete planning and measurement, not LOE) SSTC 2010 (Slides 115-118, 139-146)

 

 

1/7/2011: H.R. 6523 , the ‘‘Ike Skelton National Defense Authorization Act for Fiscal Year 2011 (NDAA),’’ was signed by the President.  It contains key provisions that may lead to EVM acquisition reform by linking technical performance with earned value.

 

Section 864(a) requires DoD to review acquisition guidance, including DoD Instruction 5000.02, and to consider

Sec. 864(b)(4):

  • the extent to which earned value management should be required on acquisitions not involving the acquisition of weapon systems and
  • whether
    measures of quality and technical performance should be included in any earned value management system

Sec. 864(c) requires that the Secretary of Defense submit a report to the House and Senate Armed Services Committees detailing any changes in the DoD acquisition guidance identified during the review and any actions taken, or planned to be taken, to implement such changes.

 

Note: A review of DoDI 5000.02 is provided at EVM Acquisition Reform . Tables with recommended changes to Office of Management and Budget (OMB) policy, the Federal Acquisition Regulation (FAR), and the Defense FAR Supplement (DFARS) to overcome the EVMS Quality Gap are at OMB policy, FAR, DFARS   

 

Defense AT&L Magazine, published by Defense Acquisition University, Nov-Dec. 2010 issue, includes the article, "Earned Value Management Acquisition Reform." Link: AT&L: EVM Acquisition Reform . Summary of article at tab: EVM Acquisition Reform

                                                         .

DOD Defense Support Team (DST) EVM Report to Congress, Sept. 1, 2009: 

DOD submitted its Congressionally-required report on Earned Value Management, DoD EVM: Performance, Oversight, and Governance (Report).

Report concluded that "The utility of EVM has declined to a level where it does not serve its intended purpose." It includes findings and recommendations regarding:

1. Changes in culture necessary

2. Accuracy of EVM data

3. Technical Performance Measures (TPM).

4. Systems Engineering (SE)

 

The following information is available at the DoD EVM Implementation tab.

  • Link to Report
  • Abstract of the Report regarding the above issues
  • Assessment of the Report's merits and deficiencies
  • Recommendations to improve the effectiveness of EVM sent to the Senate and House Armed Services Committees and to the Office of Management and Budget
  • Assessment of Industry's input to DoD for the Report   

The complete Report to Congress may be downloaded at link: DAU web site: DoD Report to Congress  

 

For guidance on how to link EV to TPMs and how to augment EVM processes with a rigorous SE process, see my articles and presentations on the Advanced EV: PBEV tab, especially "Integrating SE with EVM" published in the PMI Measurable News (May 2008) or the original version, published in May 2004 in Defense AT&L.

  

Defense Acquisition Guidebook (DAG) includes expanded guidance to integrate Technical Performance Measures (TPM) with EVM and enhanced Systems Engineering (SE) responsibilities for integrated planning.

DAG includes new guidance for integrating TPMs with EVM and the IMS, including contractual TPM reporting. A contractor must now have a TPM plan, defined in terms of expected performance at specific points in the program as defined in the WBS and IMS, the methods of measurement at those points, and the variation limits for corrective action.

DAG also includes expanded responsibility for SE to integrate the:

  • Technical scope of effort in the WBS
  • Corresponding event driven program implementation in the IMS and EVM
  • Technical baselines, TPMs, and EV.

 The pertinent sections of the DAG are 4.5.1, 4.5.4.1, 4.5.4.2, and 4.5.6.1. 

More specific guidance and examples for integrating technical performance, or Quality, with EVM are provided in my article, Integrating SE with EVM, in the “PMI Measurable News” and in the presentation to the DCMA.

  • A link to the PMI article is provided at the  Advanced EV: PBEV tab.
  • The presentation made to the DCMA EVM Center Conference, that covered the DAG's new guidance for EVM, is available at the  PBEV Live EV Training  tab.   
  • Detailed guidance, practical examples, and templates are in my book, Performance-Based Earned Value®.
  • A link to DAG is provided below.

 

May 22, 2009: Pres. Obama signed Weapons System Acquisition Reform Act of 2009 (WSARA), S. 454. Includes EVM improvement and Systems Engineering (SE) best practices.

Section 302, Earned Value Management, directs DoD to provide recommendations for improving EVM and its implementation within DoD including:

·         A discussion of the merits of possible alternatives

·         A plan for implementing any improvements the Secretary determines to be appropriate

The Act also requires that DoD develop policies and guidance for the

·         Use of SE principles and best practices

·         Development of SE master plans

 

The Act can be downloaded at http://www.govtrack.us/congress/bill.xpd?bill=s111-454

 

Web Editor’s note: Performance-Based Earned Value® meets the intent of the Act. PBEV augments the EVMS Standard by overcoming the Standard’s failure to incorporate Quality into the measurement of EV. PBEV provides guidance for integrating SE with EVM. It is based on SE principles and best practices from SE and Project Management standards and CMMI®. It ensures that EV is based on objective technical performance measures and on meeting the technical baseline requirements. Acquisition managers and contractors are invited to consider PBEV as a framework for process improvement.

 

DAPS:

Recent DOD guidance has finally acknowledged that the EVMS Standard has a “quality gap.” The “DOD Systems and Software Engineering Defense Acquisition Program Support Methodology”(DAPS) states that EVMS has no provision to measure quality.” To overcome the EVMS quality gap, DAPS provides excellent guidance to be used during Program Support Reviews. DAPS discusses the importance of using Technical Performance Measures (TPM), Critical Technical Parameters (CTP), and other metrics to cover other performance-related requirements/capabilities such as Developmental Test success criteria and  Operational Test entrance criteria. DAPS also provides useful focus questions that should be answered to ensure that earned value is linked to technical performance (Section 3.3.4.3). Links to DAPS and other DOD Guides for more effective program management are available below.

 

May 6, 2009: Dep.  Sec. Of Defense Lynn provided testimony on Acquisition Reform to the House Armed Services Committee.

Recommendations for Acquisition Reform include:

  1.  Improve contractor data reporting of earned value
  2. Tie incentive fees to contractor performance
  3. Maintain disciplined systems engineering integrated with testing

 The full testimony can be downloaded at http://armedservices.house.gov/pdfs/FC050609A/Lynn_Testimony050609.pdf 

 

DODI 5000.02 Operation of the Defense Acquisition System, date Dec. 2, 2008:

This revision requires a Systems Engineering Plan (SEP). The SEP shall describe the program’s overall technical approach, including key technical risks, processes, resources, metrics, and applicable performance incentives. It shall also detail the timing, conduct, and success criteria of technical reviews.

https://akss.dau.mil/Documents/Policy/DoDI%205000-02%20(Official%20Signed%20Version)%202%20Dec%202008.pdf

      Note: See DOD guides below, including guidance to

  • Integrate the entry and success criteria of technical reviews (such as PDR and CDR) in the IMP and IMS
  • Integrate SEP with Technical Performance Measures (TPM) and EVM

Dept. of the Navy memorandum:

DoD (POL) policy memoranda: 

 

 

 
Various DoD guides summarize policy and guidelines for integrating systems engineering with EVM and for basing award fee and incentives on achieving integrated plans and meeting contractual performance requirements. The guides cover the following topics:
  • Technical reviews
  • Integrated plans
  • Technical Performance Measures (TPM)
  • Technical Baselines
  • Integrated Baseline Review (IBR)
  • Award Fee and Incentives

 

The key guides are:

  • Defense Acquisition Guidebook (DAG)
  • Systems Engineering Plan (SEP) Outline, Vs. 1.0
  • Work Breakdown Structure Standard, MIL-STD-881C (WBS)
  • Integrated Master Plan (IMP) and Integrated Master Schedule (IMS) Preparation and Use Guide (IMP/IMS)
  • Guide for Integrating SE into DOD Acquisition Contracts (Integ SE)
  • Systems and Software Engineering Defense Acquisition Program Support Methodology (DAPS)
  • PDF file from DOD (AT&L), How to Prepare a SEP that Works, includes the following useful slides:

        19: Key Performance Parameters (KPP) should include

    •  Actual KPPs from draft or approved Initial Capabilities Document (ICD), Capabilities Development Document (CDD), or Capabilities Production Document  ( CPD)
    • Progress made towards KPPs

        23: Requirements Traceability

    •  Chart illustrates requirements traceability
    • Capabilities (CDD/CPD)
    • Technical Baseline
      • Functional Baseline
      • Allocated Baseline
      • Product Baseline

          35: Technical Reviews

        ·         Shows schedule of all SE technical reviews and relationships to the technical baseline

           All Guides and the PDF file, How to Prepare a SEP that Works, except WBS, may be downloaded at  http://www.acq.osd.mil/sse/pg/guidance.html within "Systems Engineering."

      WBS may be downloaded at http://www.acq.osd.mil/pm/currentpolicy/wbs/MIL_HDBK-881A/MILHDBK881A/WebHelp3/MILHDBK881A.htm

       

      Common Content of DOD Guides: (See detailed table by clicking top left corner, "DOD Guide Matrix")
             Technical Reviews:

      • Event-driven timing of technical reviews
      • Success criteria of technical reviews
      • Include entry and exit criteria for technical reviews in IMP and IMS
      • Assess technical maturity in technical reviews
        Integrated Plans:
         1. Integrate SEP with
            • IMP/IMS
            • TPMs
            • EVM
         2. Integrate WBS with
            • Requirements specification
            • Statement of work
            • IMP/IMS/EVMS
         3. Link risk management (including risk mitigation plans), technical reviews, TPMs, EVMS, IMS
         4. Include technical baselines in IMP/IMS and assess progress at event-driven technical reviews:
            • Functional Baseline: System Functional Review (SFR)
            • Allocated Baseline: Preliminary Design Review (PDR)
            • Product Baseline: Critical Design Review (CDR) and System Verification Review (SVR) 
        Integrated Baseline Review:
        During IBR, review
            • Plans for event-based technical reviews including entry and exit criteria
            • Correlation of TPMs, IMP, IMS, EVMS

        DCAA:

      • DCAA Contract Audit Manual (CAM): Chapter 11 includes 11-200 Section 2 --- Audit of Contractor Compliance with EVMS Guidelines

                            DCAA EVMS Compliance Audit 

                                            •