Three recent studies seem to be delivering startling information about the prevalence of antibodies to SARS-CoV-2 and the percent of infections that are asymptomatic, but they may be telling us more about the flaws inherent in the release of studies in the absence of peer review.
The first of these is a study from a homeless shelter in Boston. After 15 cases were identified and removed from the shelter over a five day period, investigators tested those still in the shelter over a two day period for presence of the virus and found the virus in 147 out of 408 residents tested and there was no statistically significant difference between those who tested positive and those who didn’t. This was picked up in at least one news outlet to suggest that the majority of those with the infection don’t have symptoms, but there are two problems. One, they had already removed the 15 people with symptoms, so they had guaranteed there would be no difference and two, they did no follow-up. The long incubation period without symptoms is one of the hallmarks of the disease. The only thing this study tells us is the ease with which the virus spread in a homeless shelter. It is useless as any sort of assessment of the prevalence of asymptomatic disease, because we don’t know if they stayed asymptomatic.
The second is an antibody seroprevalence survey in Santa Clara County, California. This study used Facebook to recruit participants and made a big deal about their efforts to ensure that they had a representative demographic mix. They extrapolate from this sample to the county. But they are making a huge assumption that they don’t address. People who want to drive to a testing center to have a giant Q-tip jammed up their nose or have a needle jammed into their finger are probably not representative. In particular, they may have some reason to think they have been infected. Either they think they might have been exposed or they have had some sort of symptoms. The authors seem to have some awareness of this second possibility. They state, “bias favoring those with prior COVID-like illnesses seeking antibody confirmation are also possible”. They add that the “overall effect of such biases is hard to ascertain”. Actually, it’s not that hard. It’s done all the time. It’s called asking. In fact, it does appear they asked some questions about symptoms, but, for some reason, they didn’t report the answers. And they asked nothing about motivation for participating. So it is impossible to generalize to population sero-prevalence from their results, but the biases I mention here would lead to an overestimate of sero-prevalence.
The first two studies were released online without peer review, but as complete articles. The third study, a collaboration between USC, Stanford, and the LA County Health Department, doesn’t even do that. The only information is contained in a press release from the county health department and it gives absolutely no information on who was tested. None. Nada. Zippo. Unbelievable. Hard to peer review a research paper without the paper. This is research grandstanding of the worst kind. I will have more to say about this study in another post.
Ultimately, seroprevalence studies will be critically important to our understanding the epidemiology of this disease and controlling its spread. But two of these studies are presented in a way that is misleading and likely to lead to profoundly wrong interpretation. The third is impossible to evaluate, but the way in which it was released is disturbing and raises serious red flags about the validity of the results.
The problem is that all of these studies are making the news. One comes from a Harvard researcher, one from a Stanford researcher, and one from a Dean at USC. That means they will be taken seriously. But it doesn’t mean they aren’t flawed and doesn’t mean they can be properly interpreted without an understanding of the potential limitations.
What do these studies really mean? I don’t know. If they show up on a standardized test, choose option e. Not enough information given.
The battle against misinformation goes on.
