DOT&E has repeatedly reported on the Lab’s shortfalls and inordinate lead times in creating these files: it takes up to 15 months to create and validate each of these files, and a minimum of six files are needed, one for each major combat theater where F-35s might be deployed plus one specifically for the operational test range environment. Because each threat country’s military operates with different equipment, more than five combat theater files may well prove necessary. Based on the estimate provided by DOT&E, the Reprogramming Lab may not even be able to provide the operational test MDL before the end of calendar year 2018, four months after the presently promised start of IOT&E. Unfortunately, despite DOT&E’s repeated warnings regarding shortfalls with the Lab, officials in the Program Office and at Lockheed Martin have not invested enough resources in it. As a result, the Lab lacks effective capability and is late in delivering the much-needed Mission Data Loads, both for the IOT&E and for actual F-35 deployments.
To be fully effective, the MDLs must be verified and tested frequently. “[T]he Department must have a reprogramming lab that is capable of rapidly creating, testing, and optimizing MDLs, as well as verifying their functionality under stressing conditions representative of real-world scenarios.” Testing the MDLs requires special electronics (called threat emitters) that generate signals mimicking the kind of radar, missile guidance, and other threat equipment the F-35 may encounter in a combat zone. DOT&E reports that the Reprogramming Lab does not have enough of these emitters to simulate arrays of anti-aircraft threats a potential adversary would have. Without the necessary equipment, the Reprogramming Lab will not be able to properly test the F-35’s electronic warfare systems to ensure “adequate performance against current and future threats.”
The Department of Defense identified these shortfalls in 2012 and appropriated $45 million in the 2014 and 2015 budgets to fill them. As of now, the Joint Program Office and Lockheed Martin have yet to complete the necessary funding or even contract all the necessary signal generators to fully flesh out the Reprogramming Lab’s capabilities. If and when they do get around to doing this, DOT&E cautions that their current plans will not get the job done. “Even after the installation and certification of the new configuration, the lab will still lack a sufficient number of signal generators to simulate a realistic, dense threat laydown with the multiple modern surface-to-air missiles, combat aircraft, and many supporting air defense radars that make up such a laydown.”
The Lab’s work creating the data files is also impacting another key component of the testing program: the Joint Simulation Environment (JSE). The JSE is intended to be an ultra-realistic flight simulation facility consisting of multiple high-fidelity F-35 cockpit simulators and manned enemy and friendly control stations so pilots can simulate “flying” in realistically large formations against pilots “flying” enemy aircraft, missile controllers operating virtual threats, and radar operators. Unlike other flight simulators that are programmed to do what the manufacturer says the aircraft can do, the JSE is required to be a validated simulator, meaning the performance of the virtual F-35 in the simulation has been verified against the measured performance of real instrumented F-35s over the same flight paths, maneuvers, and weapons launches.
Programmers for the JSE need data from approximately 100 real-world F-35 flights through test ranges equipped with signal emitters. These flights will gather data about radar performance, weapons trajectories, and how the F-35’s onboard sensors respond to ground and air threats.
But the Joint Program Office has been slow to purchase those emitters.
Accurately collecting the necessary data and properly programming the simulation is a vital part of the F-35 operational testing process, and the emitters are a critical part of that process. From the very beginning of the program, officials have known that the only way to test four- and eight-ship flights of F-35s realistically against the kind of threat arrays it was being designed to defeat is with a high-fidelity simulator. No test range can properly replicate the full numbers and types of anti-aircraft defenses a sophisticated adversary would employ to shoot down the F-35—nor can the F-35 OT&E test fleet launch enough F-35s at any one time to test large formation F-35 attacks (especially not in view of the F-35’s 26 percent full-mission availability).
The Program Office’s foot-dragging on purchasing the signal emitters is hardly the first such instance with this part of the operational test program. The JSE is the program’s second high-fidelity simulation facility design. The Program Office cancelled the first, called the Verification Simulator (VSim), after that project had fallen hopelessly behind. Amazingly enough, the Program Office had contracted Lockheed Martin to build the VSim. That meant the prime contractor of the F-35 would have built and manned the facility that would produce the data decision-makers would use to determine the combat suitability and contractual future of the program. The students would have literally written their own final exams. Yet despite having 14 years to build the facility, Lockheed Martin fell hopelessly behind and then asked for overrun funding to fix their failure to deliver. Finally, the Program Office cancelled Lockheed Martin’s contract and made a fresh start of the simulation project by contracting with a Navy facility that had no prior experience with such large-scale simulations.
While the decision to shift responsibility for the simulator appears to be the correct one, starting over again greatly exacerbated the already disastrous schedule slips. The Program Office originally expected to have the Joint Simulation Environment completed by the end of 2017, but DOT&E reports it will likely not be fully accredited before late 2019—the currently promised end of the initial operational testing schedule. These delays are largely due to delays caused by Lockheed Martin. While the government’s physical facility with cockpits, computer servers, and visuals is reportedly nearing completion, the virtual environment with the terrain, threat, and targets is not. This is especially true with the basic software simulation model for F-35 performance because Lockheed Martin has not yet provided the data the JSE programmers need to complete the simulation for the virtual terrain, threat, and targets. The delay is allegedly because of “contractual difficulties.”
Defenders of the F-35 program usually cite the plane’s supposed unique ability to handle complex threats as the main justification for the massive costs. Without the ability to properly realistically test these complex threat capabilities, DOT&E states, we will be putting pilots' lives in danger when we send F-35s into combat. Right now, it is extremely difficult to believe the full F-35 system will be ready to start a valid, realistic IOT&E process as early as is currently scheduled.
The new DOT&E is undoubtedly under great pressure from F-35 program advocates to start IOT&E quickly and to compromise the scope and realism of the F-35’s operational test in whatever way necessary to avoid delays ramping up the F-35 buy. If he does stand firm, will “business as usual” prevail anyway? The new DOT&E may find himself overruled by high-ranking defenders of continuing concurrency and malpractice in the Pentagon’s acquisition bureaucracy.
If the defenders of the status quo prevail and force premature operational tests that are far less realistic than was agreed to in the TEMP, it will be interesting to see whether the new DOT&E’s future reports certify that the testing was adequate to reliably assess F-35 combat suitability. If his reports do certify the adequacy of the testing, it will then be interesting to see whether they go on to confirm the F-35’s suitability for combat.
The new annual DOT&E report reveals details about a dominant component of the F-35’s 17 years of acquisition malpractice: its high level of concurrency. Concurrency is a term for the deliberate overlap of development, testing, and production in an acquisition program. The Government Accountability Office has identified this as one of the single biggest drivers of cost and schedule growth in the F-35 program. The GAO also identifies concurrency as a root cause of many of the F-35’s performance shortfalls.
While the problems with concurrency have been well understood in broad terms, we are now beginning to see the details of how the rush to buy F-35s impacts the development and testing process.
One of the biggest dangers of rushing hundreds of aircraft into production with an immature design is that they will later have to be retrofitted and retested with the revised design fixes that overcome discovered problems. This is an expensive and time-consuming process, especially considering that the aircraft being fixed were already purchased at full price and that reworking them will result in additional costs that would otherwise not have been incurred. Concerns over these very large concurrency costs prompted Air Force leaders to float the idea of leaving 108 F-35s purchased early in the program in their immature state, which could have left taxpayers with $21 billion to $40 billion worth of “concurrency orphans”—aircraft that were paid for but are unsuitable for combat. The Air Force has since backed off from this embarrassing stance.