The gun-injury estimate released by the Centers for Disease Control and Prevention this week is less reliable than it has been over the last 15 years.
This year's estimate has a 39.1 percent coefficient of variation, which measures the amount of uncertainty in the estimate's calculation. Given that variation, the CDC's measure puts the number of gun injuries in 2017 at 133,895. But the number could actually be anywhere between 31,000 and 236,000. That gap is so large that many academics are now refusing to use the estimate in their work.
"When I looked at the 2017 numbers, I went, 'Oh, my god,'" David Hemenway, the director of the Harvard Injury Control Research Center, told FiveThirtyEight and The Trace. "You just can't use those numbers."
"I would not cite these estimates," Guohua Li, editor in chief of the medical journal Injury Epidemiology and director of Columbia University's Center for Injury Epidemiology and Prevention, told the publications. "As an editor, I would not publish any manuscript that is based on these estimates."
The data have only gotten worse over time. In 2016, the coefficient of variation was 30.6 percent. Back in 2001, it was at 22.1 percent. The variation may be due to the small sample size the CDC relies on. In 2017 they only surveyed 60 hospitals to determine the number of gun injuries in the United States.
The CDC said they recognize the problems with their gun-injury estimate but are exploring ways to improve it.
"We accept that there are some unstable estimates," Courtney Lenard, a CDC spokesperson, told the publications. "CDC continues to look into various ways to strengthen the estimates for nonfatal firearm injuries."
The CDC's measure has shown a steady increase in gun injuries over the past 15 years but other measures that rely on larger samples and have less variation, like the Healthcare Cost and Utilization Project (HCUP) that surveys more than 950 hospitals, have found the exact opposite is true.
The CDC told FiveThirtyEight that HCUP has its own limitations, and additional congressional funding would be necessary to fix the problems with the agency's own estimates. Researchers question the idea, however, that the CDC's efforts wouldn't be improved by incorporating HCUP's data. Guohua Li said it's paramount the CDC come up with some kind of fix.
"The data quality has become more important than ever, so they should really pay immediate attention to this issue and get it improved," he told FiveThirtyEight and The Trace.