As the novel coronavirus disease 2019 deepens its impact on people across the globe, citizens are desperate for a magic bullet that can kill the pandemic before it claims more lives and imposes months of social distancing and isolation on society.

The antimalarial drugs chloroquine and hydroxychloroquine, it has been suggested, might be that bullet. Seemingly in support of that possibility, on March 29 the Food and Drug Administration authorized the Strategic National Stockpile to begin building up stores of hydroxychloroquine so that it “be distributed and prescribed by doctors to hospitalized teen and adult patients with COVID-19, as appropriate, when a clinical trial is not available or feasible.”

But statements by senior health officials tell a different story.

Dr. Anthony Fauci, the top U.S. expert on infectious diseases, told ABC News in an interview on March 31 there “no evidence at all” that chloroquine treatment was effective. He earlier characterized evidence in favor of its use as “anecdotal. It was not done in a controlled clinical trial.”

A former FDA Commissioner Scott Gottlieb told The Hill “I’m not that optimistic about hydroxychloroquine. It's not really clear it's having a clinical effect. I hope it works, but I think a lot of the enthusiasm is because it's available, so people are over-interpreting the data, because it's on the shelf.”

And Dr. Joshua Sharfstein, Vice Dean of the Bloomberg School of Public Health at John Hopkins’ University told Science magazine “Efforts to widely distribute unproven treatments are misguided at best and dangerous at worst.”

Let’s look at why the experts aren’t convinced. 

What are chloroquine and hydroxychloroquine?

Chloroquine has been used for decades to suppress certain non-resistant strains of malaria. Chloroquine can have side effects including nausea, diarrhea, mental confusion, and vision problems.

Hydroxychloroquine also sold commercially as Plaquenil, is similar but has milder side effects. In addition to anti-malarial applications, it is used to treat lupus and rheumatoid arthritis.

Because these drugs are already relatively extensively used and manufactured, it would be good news if they proved effective against coronavirus. The possible use of chloroquine for anti-viral purposes has been studied multiple times in the past. It appears to have proven effective in animals, but not so much in humans.

What about the French studies?

Since February, reports in Chinese media have suggested some experts believe chloroquine could be an effective treatment. On March 9 a Chinese team published a study reporting that hydroxychloroquine had been demonstrated to reduce COVID-2019 in vitro—that is, under artificial lab conditions. Could that finding be replicated in people?

On March 16, Dr. Didier Raoult—a prominent but controversial French infectious disease specialist—released a study (read here) involving forty-two patients infected with COVID-19. Twenty-six patients in Marseilles were administered either hydroxychloroquine or a combination of hydroxychloroquine and azithromycin. They were compared to sixteen patients at multiple hospitals that did not receive these treatments.

Based on samples collected using a nasal swab, Raoult’s study found that the quantity of the virus diminished more rapidly in those that received the medications. These results were widely disseminated, eventually leading to their endorsement by media and political figures in the United States.

But many scientists were not as convinced by these findings.

A paper by Darrehn Dahly et al. looks in-depth into numerous problems with the study, starting with a lack of randomization of test subjects, inappropriate use of control patients (a “red flag” according to Dahly), poor choice of a six-day study period, overstating extant supporting evidence for both hydroxychloroquine and azithromycin and inappropriate removal of data on the six patients that dropped out of the study.

Those six patients were the individuals suffering the most adverse health outcomes—three went to an ICU, one died, and another suffered strong side effects from the hydroxychloroquine. Of course, removing failures from the data set may contribute to a more positive net result. Another issue is that the metric used (viral loads) was not as relevant as, say, survival over a twenty-eight day period.

Dr. Elise Biek, the founder of Microbiome Digest, noted in a blog post the paper was formally published within twenty-four hours of submission—which suggests it may not have received much peer review. One of the study’s authors was also the editor of the journal the study was published in, suggesting a conflict of interest. Biek’s post also calls attention to numerous inconsistencies in the study’s conduct, such as different age ranges in the control and test groups.

The Second French Study

On March 29, Raoult reported findings from a second study this time with eighty patients who were administered 200 milligrams hydroxychloroquine and 500 milligrams azithromycin. He reported that 80 percent demonstrated a favorable outcome, but three were taken to Intensive Care Units, and one died.

There was no control group this time. One of the study’s authors, Dr. Philippe Gautret, stated that the observational groups were selected from patients with less severe cases of COVID-19. 

In a new critique, Biek notes additional inconsistencies, particularly by focus the study mostly on patients who were likely to recover anyway: 

“In short, most of these participants were not patients fighting for their lives breathing through a ventilator in a hospital ICU bed. I am just baffled by how these patients were admitted to a hospital ward to be treated, if they were not feeling too bad… A group of 80 relatively mild patients who have hardly any fever and who have already been sick for 5 days is very likely to already been recovering in the next 5 days. So what would have happened if these 80 people had not been treated? It is easy to imagine that they might have recovered by themselves.” 

Another expert, Dr. Francois Balloux of University College, London, similarly told France24 that the outcomes of the patients receiving the treatment were similar to those found in COVID-19 patients who weren’t being so treated.

The Chinese Study 

On March 24, the Zhejiang University Journal published findings from a study conducted involving thirty randomized subjects in Beijing over a two-week period in February. Half were treated with Kaletra (lopinavir and ritonavir), flu drug Arbidol and four hundred milligrams daily of hydroxychloroquine.

This study found over a two-week period that patients receiving the medicines did not recover any faster than those not given treatment. This is considered the first controlled study of hydroxychloroquine’s effectiveness in treating COVID-19. However, it does have flaws due to its small sample size and the involvement of additional drugs muddying the results.

It’s worth noting that another study in China focused on antiviral Kaletra with a large sample size of 199 patients who also failed to find any improved outcomes. 

What’s the harm in taking chloroquine, even if we aren’t sure whether it works or not?

Chloroquine can have potentially severe mental and physical side effects in some individuals. French doctors have reported that at least three persons infected by COVID-19 have died due to cardiac arrest possibly linked to hydroxychloroquine treatment. Thirty persons have also reported severe side effects. Persons in Nigeria and the United States have poisoned themselves out of ignorance attempting to self-medicate with chloroquine.

Another issue is that hydroxychloroquine’s is known to be an effective treatment for lupus and rheumatoid arthritis. Unfortunately, hoarding and panic-buying of the drug is causing a major shortage, limiting availability for people who definitely can benefit from the medicine. 

Will we ever get better evidence whether there are effective treatments for COVID-19?

There are currently several additional studies in progress testing the effectiveness of chloroquine and other mediations in mitigating the symptoms of COVID-2019. The large-scale Solidarity study underway by the World Health Organization of four medications (hydroxychloroquine, antiviral remdesivir, HIV drugs lopinavir/ritonavir again; and the last two combined with immune system messenger interferon-beta) should be especially interesting due to a sample size planned to be in the thousands.

If any studies do find a promising treatment, that should be reflected in how their findings are evaluated by peer reviewers and senior medical experts for major health organizations.

One must also keep in mind that even if an effective medication is identified, it would likely primarily serve to diminish the fatality rate of the disease. Ultimately defeating COVID-2019 will still require the development of a vaccine.

Sébastien Roblin holds a master’s degree in conflict resolution from Georgetown University and served as a university instructor for the Peace Corps in China. He has also worked in education, editing, and refugee resettlement in France and the United States. He currently writes on security and military history for War Is Boring.