Nothing Like an F-35: How America's Next Stealth Fighter Could Go 'Virtual'

May 14, 2019 Topic: Security Blog Brand: The Buzz Tags: BAE SystemsMilitaryTechnologyWorldF-35

Nothing Like an F-35: How America's Next Stealth Fighter Could Go 'Virtual'

"A future fighter might feature only a virtual cockpit projected directly onto the helmet display of the pilot."

If that sounds similar to the F-35 helmet-mounted display system, it is because that is where the idea came from.

Next generation fighters of the future might have cockpits and flight control systems that are radically different from the cockpits of today.

(This first appeared last year.)

While the Lockheed Martin F-35 Joint Strike Fighter and the forthcoming Block III version of the Boeing F/A-18E/F Super Hornet have state-of-the-art wide screen displays and feature advanced helmet-mounted displays, a next-generation fighter might dispense with traditional cockpit displays entirely. A future fighter might feature only a virtual cockpit projected directly onto the helmet display of the pilot.

“In terms of future concepts, we are looking at what we are calling a ‘wearable cockpit,’” Jean Page, a lead technologist with BAE Systems’ Human Factors team, said. “Here, you remove many of the physical elements of the cockpit, and replace it with a virtual display, projected through the helmet. Essentially, it’s a software-only cockpit that’s upgradeable, adaptable and reconfigurable.”

But dispensing with traditional cockpit displays would also require novel approaches to the aircraft’s pilot vehicle interface (PVI). One avenue that BAE Systems is exploring is tracking a pilot’s eye movements in order to interface with the virtual displays.

“In such a world, we need to think about what controls are critical to the pilot and then make them easier to manage,” Page said. “Eye-tracking gives you the option of looking at something to highlight it and then making a gesture to ‘press’ a button, rather than having a series of physical buttons on the aircraft.”

Using technologies such as eye tracking in combination with artificial intelligence might allow the jet’s systems to anticipate the pilot’s intentions and allow the machine to act proactively to help the operator. “The really clever bit will be that based on where the pilot is looking, we can infer the pilot’s goal and use intelligent systems to support task performance and reduce the pilot’s workload,” Page said. “We want to do it in a way that doesn’t always ask for permission, because that would get very annoying very quickly but equally, it is essential that it is always evident to the pilot what task the intelligent system is performing.”

BAE Systems demonstrated the idea on its Tempest concept at the Farnborough air show earlier this year, but the idea is not a new one. The United States Air Force had studied and even demonstrated such concepts during the 1980s with its so-called “Super Cockpit” program, which sought to transform the entire concept of PVI for a next-generation aircraft. The Lockheed Martin F-22 Raptor air superiority fighter and the F-35 Joint Strike Fighter adopted many of the concepts that the Air Force had explored, but many of the more radical concepts were shelved since they were so far ahead of their time.

Washington Post reporter Steven L. Thompson saw the Super Cockpit demonstrator first hand in 1987.  “The pilot wears a custom-molded helmet incorporating three key elements: the ‘helmet position sensor,’ which enables the master computer to determine the position and orientation of the pilot's head; the ‘3D binaural sound system’ with ultra-high-fidelity speakers whose computer-generated sound cues allow the pilot to pinpoint objects spatially; and the ‘cognitive port’ itself, an optical system that uses small cathode-ray tubes mounted above each of the pilot's eyes to project the virtual-world components onto variably-reflective mirrored surfaces inside the helmet visor,” Thompson wrote. “The result is an image like the one above, within a ‘port’ 120 degrees wide and 60 degrees high. The pilot can either blank out the ‘real’ world entirely and rely on the projected image (shown to him at 1:1 scale), or superimpose the two, resulting in an image overlay like what you see when your car's dashboard lights reflect on your windshield at night.”

Recommended: What Will the Sixth-Generation Jet Fighter Look Like?

Recommended: Imagine a U.S. Air Force That Never Built the B-52 Bomber

Recommended: Russia's Next Big Military Sale - To Mexico?

If that sounds similar to the F-35 helmet-mounted display system, it is because that is where the idea came from. Similarly, the Air Force had looked at virtual switches and eye position trackers even during the 1980s. “His weapons are displayed at bottom center in a panel of ‘virtual’ switches. To select any of them, he uses a combination of voice commands and ‘touch,’” Thompson wrote. “Fingertip sensors are sewn into his gloves. When he moves a sensor into one of the outlined switch areas, it produces a click in the headset and the weapon is armed. If time or G-loading is too critical to allow manual selection, he can do it all by voice, aiming the weapon by commanding a tiny laser collocated with the CRT/display system. The laser tracks his eyeball motion, ‘locking-on’ the target at his verbal command and firing the weapon.”

Thus, what is old is new again. However, the difference is that computer technology has made quantum leaps over the course of the past 35 years. In today’s world, the technology envisioned for the Super Cockpit in 1980s could be made into a practical reality onboard the next-generation of fighters with relative ease.

Dave Majumdar is the former defense editor for The National Interest. You can follow him on Twitter: @davemajumdar.