Summary

According to Department of Defense (DOD) officials, the Air Force Electronic Warfare Evaluation Simulator (the Simulator), located at Air Force Plant in Fort Worth, Texas, is an important asset for helping to protect U.S. and allied pilots and aircraft against the missile threats posed by adversaries. Most missiles use one of two electronic warfare technologies in order to pursue aircraft in flight and deliver an explosive warhead with the intent to inflict maximum damage. Small shoulder-launched missiles generally use infrared seekers that search for heat sources on an aircraft, while more sophisticated air-to-air and larger surface-to-air missiles can use radio waves and infrared seekers to determine an aircraft's location in flight. DOD continually develops and tests countermeasures to protect U.S. and allied aircraft from both types of missile threats. The Air Force Electronic Warfare Evaluation Simulator at Plant 4 is one of only two Air Force facilities of its kind that test countermeasures against heat-seeking missiles, and it is the only Air Force facility that currently houses the equipment necessary to test countermeasures against more sophisticated radio frequency surface-to-air missiles. The Simulator uses an array of computer hardware and software and other equipment to simulate the firing of a missile under various conditions and scenarios, precluding the need to actually fire and destroy a missile in the process. Conducting such tests provides DOD, the Department of Homeland Security, and allied governments with the necessary data to develop various countermeasures for use by military and commercial aircraft. The Test Resource Management Center within the Office of the Secretary of Defense (OSD) is tasked with reviewing Air Force and other services' proposals to change the test and evaluation infrastructure in accordance with OSD guidance and congressional direction.5 In a July 8, 2009, report, the center provisionally approved the Air Force's relocation proposal and submitted the report to congressional defense committees in response to congressional direction. Subsequently, on July 24, 2009, the House Appropriations Committee expressed concern about DOD's proposed relocation of the Air Force Electronic Warfare Evaluation Simulator. House Report 111-230 directed that funds shall not be obligated or expended to relocate the Simulator until a comprehensive cost-benefit analysis, reviewed by GAO, is provided to the congressional defense committees. Furthermore, the House report, noting that the Simulator's specialized test capabilities are a vital element of our national defense posture, directed that the study's findings should demonstrate the technical merits of any proposed relocation. In August 2009, the Test Resource Management Center submitted OSD's July 2009 report to us in response to the congressional direction in House Report 111-230 and, pending our review, has not submitted that report to the congressional defense committees. Our objectives for this review were to determine (1) to what extent OSD's report on the proposed relocation of the Air Force Electronic Warfare Evaluation Simulator includes a comprehensive cost-benefit analysis and (2) to what extent OSD has addressed the technical issues involved in the proposed relocation.

In its July 2009 report, OSD provided some limited cost and benefit information but did not include all expected costs and benefits associated with the proposal to relocate the Air Force Electronic Warfare Evaluation Simulator, and therefore the report does not constitute a comprehensive cost-benefit analysis. Since the report was issued, the Air Force provided us with additional estimated cost and benefit information intended to better support its relocation rationale. OSD's report identified the annual cost to operate, maintain, and modernize the Air Force Electronic Warfare Evaluation Simulator, but did not specifically identify any estimated onetime transition costs or other costs that may be associated with the relocation. Air Force officials subsequently provided us with an estimated total onetime transition cost of approximately $7 million that was not included in OSD's report. Additionally, during the course of our review, the Air Force identified other potential costs not included in this $7 million transition cost estimate or OSD's report, such as an estimated $3 million cost to temporarily retain Lockheed Martin personnel and an additional $200,000 to $300,000 cost to transfer its infrared test equipment to Eglin. OSD's report also identified the expected benefit of an annual recurring savings ranging from $2.8 million to $4.4 million, but did not include a detailed methodology supporting this estimate. The Air Force subsequently provided us with supporting information for this estimate as well as descriptions of the additional benefits expected. Nevertheless, the Air Force did not follow relevant guidance or best practices for completing a cost-benefit analysis. Air Force economic analysis guidance includes detailed procedures and a checklist for conducting such an analysis and obtaining a certification to ensure the reliability of cost estimates. OMB guidance also identifies several elements that should be included to promote independent analysis, and GAO-identified best practices for developing cost-benefit analyses include steps such as obtaining an independent cost estimate. Air Force officials told us they felt that the information they submitted was sufficient and indicated that conducting the kind of costbenefit analysis described in Air Force guidance would have been cost prohibitive. While Air Force guidance permits officials to secure a waiver under those and other circumstances, the Air Force office developing the cost-benefit analysis did not request such a waiver or provide us with documentation supporting its rationale that conducting such an analysis would have been cost prohibitive. Ultimately, by not following relevant guidance for conducting a comprehensive cost-benefit analysis, the Air Force may lack reasonable assurance that its proposal includes all the potential costs, benefits, and impacts associated with its relocation proposal and the proposal may also not sufficiently satisfy congressional direction to provide a comprehensive cost-benefit analysis. We are making two recommendations to improve DOD's proposal to relocate the Air Force Electronic Warfare Evaluation Simulator. First, we recommend that OSD, in consultation with the Air Force, revise the July 2009 cost-benefit analysis to adhere to internal Air Force guidance and identify all costs and benefits associated with the relocation proposal and submit it to the congressional defense committees. Second, we recommend that the Air Force finalize a transition plan that includes steps for staffing and training personnel to operate and maintain the relocated Air Force Electronic Warfare Evaluation Simulator capabilities and submit that plan to the congressional defense committees as well.

Recommendations

Our recommendations from this work are listed below with a Contact for more information. Status will change from "In process" to "Open," "Closed - implemented," or "Closed - not implemented" based on our follow up work.

Recommendations for Executive Action