Another Sham E-Cigarette Study is Retracted; Can We Trust Any of this Literature?

If you missed it, an article published in 2023 in the Journal of Investigative Medicine sent shock waves among researchers and policy makers by concluding that e-cigarette use causes chronic lung disease (both asthma and COPD). The study was cited in an influential meta-analysis concluding the same and was the basis of an advertising campaign in California touting the lung disease risks associated with e-cigarette use. 

The investigators based their conclusion on a cross-sectional study of the prevalence of e-cigarette use, smoking, dual use, and nonsmoking among a sample of 178,300 adults interviewed as part of the 2015-2016 and 2017-2018 National Health and Nutrition Examination Survey (NHANES) survey. 

According to the study authors: "The survey examines a nationally representative sample of about 5000 persons each year."  

The Rest of the Story

You don't have to know any more about the study than what I wrote above in order to figure out that it was a complete sham. And it doesn't take any knowledge of epidemiology or biostatistics. All it takes is some simple math.

The study combined four years of NHANES data. According to the authors themselves, the NHANES sample size is approximately 5,000 per year. So with four years of data, you would have a total of about 20,000 respondents in your overall sample. However, the authors claim to have a sample size of 178,300 adults. That doesn't add up.

Now, if we go to the NHANES web site, it turns out that the total number of people interviewed in the 2015-2016 and 2017-2018 cycles was 31,538. So there could not have been 178,300 respondents in the sample. 

This alone should get you to question the results of the study. If the authors cannot even get the sample size correct within an error margin of 140,000, does it give you any confidence that the results can be trusted?

But that's just the beginning of the sham. 

The study claimed that it compared the prevalence of COPD and asthma among a sample of e-cigarette users, smokers, dual users, and nonsmokers (two separate analyses). But it turns out that they used the following definitions:

"Smoking: The current smoking status of the participants was assessed by the following questions:
SMQ020: ‘‘Smoked at least 100 cigarettes in life,’’ SMQ040: ‘‘Do you now smoke cigarettes?’’

"E-cigarette: The e-cigarette use was assessed by the question; SMQ900 ‘‘Have {you/SP} EVER used an
e-cigarette?"   

Since smoking was defined as current smoking, the study apparently considered ex-smokers to be nonsmokers. In addition, it apparently classified as an e-cigarette user anyone who had ever used an e-cigarette. So if a 80 pack-year smoker tried an e-cigarette once, they would be classified as an e-cigarette user. Obviously, this is problematic because if that person gets COPD, it is certainly due to the smoking rather than the vaping. But the analysis is considering this as a death due to dual use. And if that individual was able to quit smoking, then they would be classified as a pure e-cigarette user. Clearly, that is going to artificially inflate the estimated risk for COPD among e-cigarette users.

This, too, would be enough to discount the study results. But it doesn't end there.

According to the data presented in Table 2 in the paper, the crude odds ratio for COPD associated with exclusive vaping is 0.18. This, in and of itself, would actually suggest that vaping is protective for COPD. But the paper presents an adjusted odds ratio of 11.3 for the risk of COPD among vapers compared to smokers. This is essentially impossible. It indicates that there is a major problem with the data being reported or the data being analyzed. The study results are meaningless.

We don't need to go any further, but it bears mentioning that even if the study did find an increased prevalence of COPD among e-cigarette users, it would most likely be due to the fact that when a smoker is diagnosed with COPD, that it likely to spur a quit attempt, possibly with e-cigarettes. So it would most likely be that the COPD is "causing" the vaping rather than that the vaping is causing the COPD.

Fortunately, the efforts of two of my colleagues were successful in getting the journal to retract the paper. 

Interestingly, another paper by some of the same authors, which concluded that vaping causes stroke, was also retracted recently. That study, which also used the 2015-2018 NHANES data, claimed to have a sample size of 266,058 respondents, which is 8 times higher than the actual total number of NHANES respondents. I don't know where the additional 234,520 respondents came from but it appears that many of these imposters also intruded themselves into the more recent study as well.

The reporting of inaccurate science, reported widely through the media and by tobacco control groups, has contributed to a widespread misunderstanding of the relative risks of vaping compared to smoking and to irrational policy decisions regarding the regulation of electronic cigarettes. In turn, this misunderstanding has harmed the public's health by deterring many smokers from quitting and has caused many ex-smokers to return to smoking. 

But the adverse implications of this go beyond that. Unfortunately, they cast doubt on the validity of the entire field of research claiming that e-cigarette use causes lung disease, stroke, and cancer. Even worse, these continuing retractions risk the loss of public trust for scientific research more broadly.