The United States Navy’s recent shoot down of a Syrian Arab Air Force Sukhoi Su-22 Fitter near the town of Tabqah over Syria is illustrative of a truth in modern warfare: Weapons do not always work as advertised.

During the engagement between a pair of Boeing F/A-18E Super Hornets—flying off the aircraft carrier USS George H.W. Bush (CVN-77)—and the Fitter, advanced U.S. air-to-air missiles were decoyed at short-range. Indeed, as was reported by CNN, the Super Hornets first attacked the antiquated early-1970s vintage Su-22 strike aircraft with an infrared-guided Raytheon AIM-9 Sidewinder.

Though the Pentagon has not released any details, the Sidewinder was likely an AIM-9X, the latest iteration of the long-serving weapon that features high off-boresight capability. Though the F/A-18E pilot fired the Sidewinder from about half a mile away—very short range even for an AIM-9 shot—the weapon was decoyed by the Russian-built Su-22’s flares. The Navy pilot reengaged with a Raytheon AIM-120C AMRAAM—a considerably more expensive and much longer-ranged active radar guided weapon—to dispatch the antiquated Fitter.

The question one might ask is: How did the Su-22 pilot decoy the AIM-9 with flares given the modern Sidewinder’s advanced imaging focal plane array infrared seeker?

Indeed, the AIM-9X is specifically designed to avoid being spoofed by decoys. All Sidewinders have had such features since the early 1980s—but the 9X variant is exponentially more capable than its predecessors. The problem is that technology is never perfect, and often, the enemy will have their own tricks up their sleeves.

In any case, American-made air-to-air missiles have historically demonstrated excellent Probability of Kill (Pk) during testing and at live-fire Weapon System Evaluation Program exercises like Combat Archer. However, those same air-to-air weapons often fail to meet those performance levels during actual combat.

The air war over Vietnam is perhaps the best illustration of this phenomenon. During its development, the AIM-7 Sparrow achieved a probability of kill of roughly 80 to 90 percent. That figure dropped to between 50 and 60 percent during operational tests. However, when the weapon was deployed into combat, its performance was nothing short of disastrous.

Early in the Vietnam War, during Operation Rolling Thunder, initial versions of the AIM-9 only managed to achieve a dismal 16 percent success rate—only 29 missiles out of the 187 launched hit their targets. For the longer range AIM-7 Sparrow semi-active radar-guided missile, it was even worse—it hit only eight percent of its intended targets, with only 27 hits out of 340 launches. Even during the late phases of the Vietnam War, during Operations Linebacker I & II in 1972 and 1973, the AIM-7 achieved a dismal 11 percent success rate while the AIM-9 wallowed at a 19 percent hit rate. By the time of the 1991 Gulf War, newer version of the AIM-7 achieved a 51 percent hit rate while the AIM-9 achieved a 67 percent hit rate—according to data compiled by the Center for Strategic and Budgetary Analysis.

The Probability of Kill numbers for the modern day AIM-120 AMRAAM and the AIM-9X are classified, but weapons are known to perform exceptionally well during live fire exercises. However, as with previous generation weapons, the AIM-120 has not lived up to its statistics during actual combat. Since its debut during Desert Storm and through Iraqi Freedom, the AMRAAM has defeated six targets in combat for 13 shots taken at beyond visual ranges—roughly a 46 percent success rate. The weapon also recently destroyed the aforementioned Su-22 over Syria—but that was at close range.

It is not clear if the AIM-9X has been used in combat previously—the recent Su-22 engagement is probably its debut against a manned warplane. However, during the engagement, the weapon was decoyed by Soviet-built flares onboard the Su-22. While this might seem shocking given the advanced technology on the AIM-9X, the ancient Soviet flares have proven to be a vexing problem to overcome for years.

Indeed, as the U.S. Air Force discovered, much to its horror during the 1980s, the AIM-9P—which was designed to reject flares—showed that it would preferentially track Soviet flares over the real target. As John Manclark—commander of the U.S. Air Force’s elite 4477th Test & Evaluation Squadron from 1985-87—recounted to my former colleague, legendary aviation reporter Bill Sweetman, in 2012, testing the weapon against actual Soviet countermeasures proved eye opening.

“The CIA gave us a flare dispenser from a Frogfoot [Su-25] that had been shot down in Afghanistan. We gave it to maintenance – it was just a thing with wires coming out of it. Four hours later they had it operational on a MiG-21...In 1987 we had the AIM-9P, which was designed to reject flares, and when we used U.S. flares against it would ignore them and go straight for the target. We had the Soviet flares – they were dirty, and none of them looked the same – and the AIM-9P said 'I love that flare,’” Manclark told Sweetman, who was at Aviation Week at the time.

"Why’d that happen? We had designed it to reject American flares. The Soviet flares had different burn time, intensity and separation. The same way, every time we tried to build a SAM simulator, when we got the real thing it wasn’t the same.  I use the AIM-9P because it is out of the system and I can talk about it. The same thing happened to a lot of things that are still in the system and that I can’t talk about."

Indeed, the AIM-9X might have been one of those weapons. The best-laid plans of mice and men often go awry definition as the saying goes.

Dave Majumdar is the defense editor of The National Interest. You can follow him on Twitter @DaveMajumdar.