Slow-release L-cysteine Lozenges in Smoking Cessation: Meta-analysis of Two Randomized Controlled Trials

Discussion

The present study was undertaken to provide additional support to the results reported in two original RCTs, suggesting that slow-release L-cysteine is an effective new tool in assisting the smokers to quit (12-14). The results in both studies were very similar, implicating that among the smokers who completed the trial according to the PP compliance, the OR for smoking cessation (point prevalence of abstinence) varied between 1.51 to 1.65. The higher OR was obtained in the first RCT which was smaller in its sample size (n=423) and unfortunately proved to be inadequately powered to reach statistical significance to this OR (12). In the larger study (n=1,998 smokers), however, the OR for smoking cessation was 1.51 (95% CI=1.12-2.02) and was statistically significant (p =0.006) (13). It was of interest to see, whether the meta-analytical approach would yield a significant summary effect size (Acetium^® efficacy) when both studies are included in a RE meta-analysis (19). Indeed, this proved to be the case, when the summary estimates of Acetium^® efficacy were statistically significant in both the ITT (n =2,421) and PP (n=863) groups: OR=1.29 (95% CI=1.04-1.60, p=0.018) and OR=1.53 (95 CI%=1.16-2.01, p=0.0025), respectively. This data translates to risk ratios (RR) and risk difference as follows: in the ITT group, RR=1.24 (95%CI=1.03-1.48, p=0.019), and the risk difference is 3.5% (18.1%-14.6%), with the number needed to treat, NNT=28 (95% CI=15-168). In the PP group, the respective numbers are RR=1.29 (95% CI=1.09-1.52, p=0.0024), and the risk difference is 10.2% (45.0%-34.8%), with the NNT=10 (95% CI=6-27). In other words, in the ITT group one needs to treat 28 smokers to get one extra quitter, whereas in the PP group, this number is only 10 smokers.

The question arises: how to interpret these data based on a meta-analysis of two studies only? In a setting where only two studies are included in a meta-analysis, several methodological aspects need to be considered, including feasibility, statistical limitations and interpretation of the data (20-22). Regarding feasibility, it is statistically permissible to perform a meta-analysis with only two studies (19, 21, 22). Modern meta-analytical techniques (both fixed- and random-effects models) (19) can compute a pooled estimate from as few as two data points, provided that: 1) the studies are comparable in their i) design, ii) population, iii) intervention, iv) control, and v) outcomes, and 2) the effect size and its variance (95% CI) are available for each study (20-23).

All these conditions are fulfilled in the present setting, where the two studies of interest (12, 13) are practically identical in their design, i.e., double-blind, randomized placebo-controlled trials using a population-derived cohort of active cigarette smoker volunteers (12, 13). The only difference is the study execution; while the first one was based on personal, on-site monitoring (12), the latter was conducted fully electronically (13). Yet, the key results of both studies were practically identical, including the results of the multivariate analysis for the significant co-variates of smoking quit (12, 13). As to the study population, both RCTs included only cigarette smokers who volunteered to participate in the trial and committed themselves to adhere to the study protocol (12, 13). The only difference between the two studies was in how the study subjects were recruited. In the first study (12), the invitations included different media (mostly newspaper announcements), whereas in the latter (13) only electronic media were used. As to the intervention, this was exactly the same in both studies, using the same preparations for Acetium^® lozenge with the same mode of administration, i.e., one lozenge used in context with every smoked cigarette (12,13). As to the control, the same placebo preparation (a lozenge) was used in both studies, specially manufactured for this purpose. Importantly, the control group was randomized, double-blinded to the study subjects and study monitors and only disclosed after the completion of the trial (12, 13). Thus, there were no dissimilarities in the controls between the two studies. Finally, the outcomes of both studies were recorded similarly, using the same primary endpoints (PA and PPA), recorded separately for all randomized subjects (ITT group) and for those completing the study per protocol (PP group) (12-14).

As to the prerequisite for effect size and variance, this condition was also perfectly matched in these two RCTs, reporting the key results in an identical manner in the original reports (12, 13), and supplemented later by an addendum to report the ITT group (14). The effect size in both RCTs was very similar, including the variance (95% CI) within narrow limits. Accordingly, these critical points are in good control, and this two-study meta-analysis can be considered valid, as it was done transparently and the results interpreted with caution (20-22). In this meta-analysis including two studies only, also the use of Knapp-Hartung (also known as Sidik-Jonkman) method (KH) could be considered to provide adjusted CIs for the pooled effect (19). KH method is considered to improve the accuracy of the CI when the i) number of studies is small or ii) between-study heterogeneity (τ²) is substantial. In this analysis, only the first is true (two studies), whereas the study heterogeneity is non-existent (see below). Because KH adjusts the standard error (SE) and uses a t-distribution instead of a normal distribution, it will produce wider CIs than those estimated by the traditional DerSimonian–Laird (DL) RE method (19). Indeed, this was the case in the present analysis, for both the ITT and PP groups, and in fact, the effect size in the ITT group lost its significance (OR=1.30; 95% CI=0.59-2.82, p=0.147). This result needs to be interpreted with caution, however, because the KH is known to perform poorly in settings with <5 studies, and its use is questionable when there is no documented heterogeneity between the studies (19).

Referring to statistical limitations, these are inevitable in a meta-analysis with two studies as compared with a similar meta-analysis with a larger number of studies (19-23). Most importantly, a meta-analysis of two studies is underpowered for assessing certain aspects of larger meta-analyses (20-23). First of those is the evaluation of study heterogeneity. More precisely, the I² and Cochran’s Q statistic have very low power, and a non-significant heterogeneity with these tests does not mean that the studies are truly homogeneous (20-22). In the present analysis, I² reached the value 0.00 in both ITT and PP analysis (Figure 1 and Figure 2), and Q=0.31 (p=0.58) and 0.05 (p=0.83), respectively. Similarly, the variance component estimator of heterogeneity (τ²) gives the value 0.00 in both ITT and PP analysis, indicating that RE model and fixed effects (FE) model give identical results. Indeed, this was confirmed by running both RE and FE models for these data. In a larger meta-analysis (23), this would certainly indicate lack of any heterogeneity at all, and indeed, indicates that the studies included in the meta-analysis are homogeneous. In this respect, however, the present meta-analysis can be concluded to represent a special setting in that the two studies included in the analysis are practically identical in most key aspects, including the design, population, intervention and outcome measures. Accordingly, it seems warranted to conclude that there is no heterogeneity between the two studies, as also confirmed by the conventional statistics (I² and Cochran’s Q), formally suggesting the lack of heterogeneity, as did the Galbraith plots (Figure 3 and Figure 4). The latter, however, can be ignored because these are generally considered meaningless with only two data points (20-22).

The second shortcoming of the two-study meta-analysis is the formal evaluation of the publication bias. As recently discussed in expert reviews, the funnel plots as well as the Egger’s and Begg’s test are meaningless with only two studies (20-22). When these tests were done with the present data, the results indicated no publication bias. The same was true with the funnel plots (Figure 5 and Figure 6). This is, however, consistent with the reality, because these two studies (12, 13) are the only ones ever conducted and published, and no additional studies are underway or even planned. With these two studies giving practically identical results, it is hard to imagine any “publication bias” with the common meaning of the term. An additional hypothetical study failing to report a significant effect for Acetium^® might generate a publication bias, if subjected to the same meta-analysis with the existing studies.

Yet the third point of concern (a potential limitation) in two-study meta-analysis is the precision of the pooled (summary) estimates (20-22). If the included studies are small, there is a danger that the confidence intervals may be broad, in which case even small changes in either study can substantially shift the combined effect. In the present meta-analysis, however, this was not an issue (Figure 1 and Figure 2). In both ITT and PP analysis for the summary effect size, the 95% CI was quite narrow (less than 1.0 unit), being narrower in the ITT analysis, with substantially more cases included. Thus, the combined effect was not significantly shifted from the values of the individual studies (20-22).

Following the stringent statistical principles (19-23), the results of the present meta-analysis clearly permit an estimation of the average (summary) effect, but one cannot robustly evaluate heterogeneity or publication bias. Accordingly, the pooled estimates for Acetium^® efficacy in smoking intervention are statistically significant both in the ITT and PP analysis (Figure 1 and Figure 2). This result is important as such, because the first RCT (12) failed to demonstrate a statistically significant effect size even in the PP analysis, simply due to an insufficient statistical power. In the second trial, the effect size reached statistical significance in both ITT and PP analysis (13, 14). The summary estimates of the present meta-analysis indicate a significant efficacy for Acetium^® intervention in both the PP and ITT analysis. In all RCTs, the ITT group is considered equally important as the PP analysis, both having different scopes and interpretations depending on the context (18). In the current context, we can state that smoking intervention by Acetium^® lozenge seems to be more effective than placebo among all smokers randomized in two trials, irrespective of whether they complete the study following the protocol or with some degree of violations (12-14).

As to the value of such a small meta-analysis like this, there are several points that support undertaking of a two-study meta-analysis. At a general level, even though small, such a meta-analysis can 1) summarize evidence from geographically or temporally distinct populations, 2) provide a more stable overall estimate than either study alone (if both estimates point to the same direction), 3) serve as an evidence bridge until further studies become available, and 4) support regulatory or marketing documentation, especially when both trials meet good clinical practice standards. In the case of Acetium^® lozenge in smoking intervention, most of these points are valid. Two independent, double-blind RCTs showing nearly identical results in two different cohorts of smoker volunteers in Finland substantially strengthen the external validity of the original RCTs, even if some of the formal meta-analytic tools have limited statistical power (heterogeneity and publication bias). A discussion about the significance of these results in comparison with the other (conventional) smoking intervention methods falls outside the scope of this communication; this is comprehensively addressed in the original reports (12, 13).