In the past week, both the FDA and the CDC have collapsed to the pavement. Scientific rigor has been cast as Deep State conspiracy. Its subjugation for political expediency has already cost human lives.
The (Pre-Convention) Miracle
Consider the recent, much ballyhooed announcement by Trump about the miracle of convalescent plasma. He boasted of battling obstructionists to give the public access to a therapy that, according to him, reduces mortality by 37%. “The deep state, or whoever, over at the FDA is making it very difficult for drug companies to get people in order to test the vaccines and therapeutics,” Trump had complained a day earlier. “Obviously, they are hoping to delay the answer until after November 3rd.” Let’s break this down.
This assertion of a 37% reduction in mortality was based on an analysis conducted at the Mayo Clinic using data from hospitals all over the country. Patients were recruited to participate and those who agreed to the experimental therapy were given plasma that had been extracted from patients who had recovered from the disease. The plasma contained antibodies, but the investigators had no way to control the concentration of antibodies, so it varied widely among patients. The study compared survival rates based on the concentration of antibodies in the plasma they received. These are the results on the FDA website.
The PITCH
This figure represents a stunning misrepresentation of the data. The investigation considered two outcomes, mortality at 7 days and mortality at 30 days. They also considered three levels of exposure in two different time frames, within 3 days of admission and after 3 days post-admission. That’s twelve different comparisons. They only report one of them. In a comparison of the highest and lowest doses given within three days of admission, seven-day mortality shows a significant improvement. None of the other 11 results was statistically significant. This is marketing, not science.
Two problems. First, an improvement in seven-day mortality doesn’t seem so great if there is no significant improvement in 30-day mortality. Second, conducting twelve tests, getting a single significant result, and presenting that as representative as the FDA has done here violates the most basic protocols of science and statistics. Probability dictates that, in twenty comparisons of a random event, one expects one to be significant just by chance. So, finding a significant effect in this case is a bit like shooting at a target 12 times, hitting it once, and claiming you are a sharpshooter. I would not let a first-year medical student get away with that, much less an FDA scientist.
And that is not the worst of it.
The old normal
Normally, for approval of a drug, the FDA requires a randomized control trial (RCT). That means that a group of willing participants with a particular disease is identified and randomly assigned to either the experimental treatment or, in most cases, a placebo that looks like the experimental treatment. In this way the two groups should be roughly the same with respect to characteristics that can influence the course of disease. That is not at all what happened here.
This was conducted as part of an “Open-label, Expanded Access Program (EAP) for the treatment of COVID-19 patients with human convalescent plasma”. There was no placebo control group for comparison. They could only compare groups based on levels of antibodies in the plasma. More importantly, treatment was not randomly assigned. As a result, there was no guarantee that the treatment groups were similar. In fact, they were not at all similar.
Why Clinical trials matter
As shown in the Table below, the treatment groups differed in critical ways. These data should have been prominently featured but were buried in the Appendix. They show that those receiving lower doses were significantly sicker prior to treatment than those receiving high doses. Before receiving convalescent plasma, low dose patients were more likely to have signs of severe lung infection, less able to oxygenate their blood, more likely to have multi-organ failure, and 34% more likely to have five or more severe symptoms. In addition, patients in the low dose group were also much more likely to have been sick early in the course of the pandemic when hospitals were more crowded and treatment was less effective in general. In a testament to the President’s last effort to push a treatment with inadequate science, low dose subjects were 61% more likely to have received hydroxychloroquine.
All of these differences could contribute to the low dose patients doing worse than high dose. The reasons for these differences were not fully examined. They could reflect regional, hospital specific, or temporal differences in antibody levels in plasma collected that was correlated with poor outcomes. Bottom Line: we have no way to be sure that the difference in dose explained the difference in outcome.
A Comparison of patient characteristics between those receiving high and low concentrations of antibodies in convalescent plasma.
| Low Dose | High Dose | % Increase from High to Low Dose | |
| Lung Infiltrates | 51% | 43% | 18% |
| Poor Oxygenation | 36% | 27% | 32% |
| Multi-organ Failure | 17% | 14% | 21% |
| Septic Shock | 15% | 13% | 14% |
| Five or more severe symptoms | 26% | 19% | 34% |
| Treatment with Hydroxychloroquine | 31% | 19% | 61% |
| Treatment Early in Pandemic | 26% | 17% | 56% |
Making Good science harder to do
The investigators attempt to account for these differences in treatment groups mathematically, but there is no way to know if the mathematical correction is adequate. RCT’s do not require this sort of correction because treatment is randomly assigned. That’s why we rely on them. That’s why the investigators spend half of their discussion suggesting their study is as good as an RCT. It’s not.
There have been RCT’s for this treatment, but they have been small and inconclusive. They suggest there may be a benefit and larger RCT’s could demonstrate that conclusively, but conducting them will now be difficult to do. By pushing this treatment before having all the evidence, it will be hard to find patients willing to be randomized to possibly not getting it. As a result of the FDA authorization, we are likely to collect plasma from tens of thousands of recovered patients and inject it into patients who are ill without knowing for sure that it works.
CDC Testing Change with no scientific justification
It is bad to push a treatment without adequate data. It is even worse to change public health measures with no data at all. That appears to have just happened with a major change in CDC guidelines for testing. With no warning or public announcement, the CDC modified their official guidance to state that testing people who are asymptomatic is no longer warranted, even when someone has been in close contact with someone known to be infected. They provide absolutely no scientific justification for this. We know that pre-symptomatic cases can spread the disease. If we want to stop the spread of disease, we need to determine if people are infected as early as possible. This change in protocol would stop us from doing just that. It is a recipe for disaster.
Epidemiologists are dumbfounded. Medical organizations from the American Medical Association to the Infectious Disease Society of America have decried this change. The only possible reason for doing this is to artificially reduce the number of cases by testing less. The President has been complaining since June that testing is creating cases. Evidence suggests that he has been trying to limit testing from very early on. He has complained that testing is making the numbers high and damaging his chances for re-election. He is willing to subvert science and public health, putting American lives at risk, solely to improve his chances in November.
The war of fog
I will be the first to admit it. The scientific process is a pain in the ass. But a necessary one. Its rigor makes it slow and plodding. But it has no politics. It is not about left or right. It is about correct.
Trump’s overt and covert efforts to steamroll scientific protocols create confusion and conflict where none should exist. His intentional efforts to suppress science makes it far harder for researchers to conduct good research and for the public to make good decisions. But that seems intentional. His obfuscation allows him to convince his followers that science is part of some “Deep State” conspiracy to prevent his reelection. His is a War of Fog. In that war, science (and Americans) are dying.
