During the 2025 edition of the Global Forum on Nicotine (GFN), Dr. Arielle Selya presented the results of her research on the most common flaws found in studies on e-cigarettes and their health effects.
Once again, GFN 2025 served as an opportunity to showcase to both professionals and the general public the latest findings from scientists and researchers in the field of research on the harm caused by smoking, nicotine use and modified-risk products.
As in previous years, CoEHAR took part in the event to present a summary of the research conducted in its Catania-based laboratories, as well as those of its international partners. The projects span a wide range of topics, from oral health and respiratory well-being to the impact of new technologies.
Among the most noteworthy presentations was that of Dr. Arielle Selya, from Pinney Associates INC. and CoEHAR consultant, who shared the results of her collaborative work with the Italian center’s founder, Prof. Riccardo Polosa, focusing on the most common flaws in e-cigarette studies.
To better understand the importance of this topic and its implications for the development of research in this field, we sat down with Dr. Selya for an interview.
- What are the common flaws in e-cigarette studies?
There are different types of flaws for different types of research studies. In my GFN 5 with Prof. Polosa, we focus on the flaws that are common to studies that analyze survey data to look at health outcomes associated with e-cigarette use. We focus on three main ones:
- Confounding by prior cigarette smoking: since most people who vape currently or formerly smoke, any negative health effects they are having could be a lingering effect of cigarette smoking rather than of vaping.
- Uncertain or even reverse directionality: in many studies, it’s not clear which came first: e-cigarette use or the health outcome. In other studies, there are known to be data points where the health condition developed before the participant ever used e-cigarettes – in which case, e-cigarettes can’t have been the cause.
- Poor definitions: This is about defining comparison groups or variables in an overly broad way. For example, nearly all of these studies define “e-cigarette use” as using at least once in the past month, but this combines people who vape daily with people who happen to have experimented once in the month before the survey. But for health outcomes, it’s duration and intensity of use that matter.
- What is your estimate of the percentage of methodologically flawed studies that are included in statistics about the harms of e-cigarettes or the behaviors of smokers who switch to reduced-risk products?
This has actually been calculated by a few different groups of researchers in the context of pointing out flaws in a 2024 systematic review of these types of studies by Glantz et al., which reported that e-cigarette use is associated with several health risks. Gal Cohen and Steve Cook showed that, just within the 10 studies examining cardiovascular disease, half had the first flaw (Confounding, i.e. duration or intensity of cigarette smoking was not accounted for). Peter Lee and Konstantinos Farsalinos shows that of 18 studies in the meta-analysis that compared switching to vaping vs. continued smoking, 16 had the second flaw (Temporality, i.e. did not exclude participants who developed the health condition before they ever started vaping). Nearly all studies I’ve seen have the third flaw (Poor Definitions, i.e. defining e-cigarette use as any past-month use, and not looking at duration or intensity).
- What is the practical impact of these flaws?
All of these flaws essentially overestimate the direct health risks of e-cigarette use. In the first two flaws, the result is that health harms from cigarette smoking are misattributed to e-cigarette use instead. In the third flaw, this could go in either direction: by combining experimental/light e-cigarette use with prolonged and heavy use, this could mask any true health effect; but on the other hand, overly broad definitions of health outcomes (e.g. considering “any wheezing in the past year from any cause” as a respiratory harm) would inflate any apparent risks. Either way, imprecise definitions can make the results uninterpretable.
Even worse, meta-analyses of studies with these flaws is perceived as being higher-quality evidence, but it’s not if it simply combines the underlying studies and perpetuates their flaws. This point was made in a different field in letter to the editor titled “A perspective or review based entirely on low-quality evidence is simply an opinion”.
- One of the most debated factors is the smoking history of study participants. How can we effectively assess the impact of past smoking in subjects who switch, and how should a study be designed to avoid biases that could compromise its validity?
There are two approaches, and unfortunately neither is perfect, but that’s the nature of observational data. The first way is to include in the analysis any and all variables that capture duration and intensity of prior cigarette smoking (including among people who used to smoke but don’t anymore—and then the time since they quit becomes very important). This will be imperfect, but there are several studies showing that when using this approach, associations that initially appeared to show a possible health effect of e-cigarette use became non-significant . The second way is to avoid the confounding issue altogether by analyzing only people who never smoked – but this is a small group so data are limited.
- What are the best practices for researchers in this field?
On the data collection side, researchers should ask detailed questions about both cigarette smoking history (cigarettes per day, how long they’ve been smoking, if they quit, how long ago, etc.) and e-cigarette use (frequency, duration, puffs per day, etc.), as well as information about timing (i.e. when their heath condition or symptoms developed and when they started using e-cigarettes).
On the analysis side, researchers should focus on participants who developed the health condition only after using e-cigarettes for some time. Researchers should also be careful about accounting for any prior cigarette smoking, either by focusing the analysis only on people who never smoked (which avoids the confounding issue) or adjust for smoking history as thoroughly as possible. Looking at duration and intensity of e-cigarette use is also important, because if there’s a direct health effect then we’d expect the risk to increase with heavier and prolonged use.
- What developments do you foresee for research in the harm reduction sector?
In following the academic literature, it seems like it’s generally accepted now that switching completely to e-cigarettes is less risky than continuing to smoke cigarettes. I think the harm reduction debate has moved now to dual use, with some concerns over whether dual use increases overall health risks or prolongs smoking, and others arguing that dual use is part of the transition away from cigarettes and often involves reducing cigarette consumption over time. So I think understanding dual use is an important topic for current research.
