Original articleNode-making process in network meta-analysis of nonpharmacological treatment are poorly reported
Introduction
Clinicians should base their daily health-related decisions on the best evidence. The medical literature production is constantly growing; therefore methods to summarize evidence are strongly needed. Systematic review with meta-analysis has been developed to play this role [1], but this design can assess only two interventions in a pairwise comparison. However, several interventions are often available for a single indication, and clinicians need to assess the comparative effectiveness of each of them [2], [3]. Network meta-analysis (NMA) allows for combining both direct and indirect comparisons to estimate all possible pairwise comparisons between interventions for a single indication and ranking them according to their estimated treatment effect [4], [5]. This method has recently experienced incredibly fast use [6].
Nonpharmacological treatments include interventions such as surgery, technical procedures, devices, rehabilitation, psychotherapy, behavioral interventions, and alternative medicine [7]. They are widely used in clinical practice and represent 24% of assessed interventions in published randomized controlled trials [8], [9], [10], [11]. Nonpharmacological treatments are usually complex interventions, involving several components, each component possibly affecting the success of the intervention [12]. The complexity of these interventions raises important issues when performing NMAs. One major issue is to gather nonpharmacological treatments together in a homogenous group to allow for comparison. This process, called “lumping”, is known to feature limitations for further interpreting results [13], [14], [15], [16], [17], [18], [19], [20]. Indeed, for a single clinical indication, similar but not identical interventions can be lumped in different ways, which might impact the result of the NMA [18], [21]. For example, Del Giovane et al. proposed models to evaluate the assumption of similarity of dose effect [18] for situations of doubt as to whether the various doses of the same treatment can be lumped or not.
In 2015, the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) group developed a guideline dedicated to NMAs [22]. This guideline recommends specifying inclusion and exclusion criteria for each node and providing a justification when interventions are merged to form a single comparator (i.e., lumping of interventions). Indeed, different ways of lumping interventions could affect the results of the study [18], [21]. However, no method is available to support the node-making process, and to our knowledge, guidelines on this issue are lacking.
We aimed to identify methods for building nodes in NMAs assessing nonpharmacological interventions. To achieve this, we planned a literature review of methodological articles of NMAs, and a systematic review to describe methods currently in use and how they were developed.
Section snippets
Literature review of guidelines and methodological articles about NMAs
We performed a literature review of English reports to identify potential methods for the node-making process in NMAs. We searched (1) MEDLINE via PubMed (equation search provided in Appendix 1); (2) Google Scholar (the first 100 references from each equation search presented in Appendix 2); and (3) the references of major articles such as the PRISMA or International Society For Pharmacoeconomics and Outcomes Research recommendations [22], [23]. One author (A.J.) screened titles and abstracts.
Methods proposed to lump interventions into nodes: literature review
In our literature review, we included 116 articles (Fig. 1) published between 2005 and 2016. Among the included articles, 12 (10%) discussed the concept of lumping in NMAs. Lumping was reported as “frequently used” [26], “challenging to manage” [18], [20], and a “determinant for the further interpretation of the results” [18], [27]. Expert consensus [28], [29] based on “clinical grounds” [18], [20] was the only recurrent solution proposed to support lumping. Some authors indicated that this
Discussion
Our study highlighted (1) the lack of recommendations for lumping interventions into nodes in NMAs and (2) the poor reporting of the node-making process for NMAs assessing nonpharmacological interventions. In our systematic review, less than 10% of the NMA reports gave a method or at least references to explain the lumping of interventions. Among the nine NMA reports giving a justification, five cited a previous study to justify their nodes, and four reported a consensus method to build nodes.
Conclusions
These methodological and systematic reviews confirm that recommendations on reporting the node-making process are lacking and that the node-making process is poorly described in NMAs of nonpharmacological treatments. The methods used to decide inclusion and exclusion criteria for each node is a key element to be reported. Using panels of experts or previous references are the most frequent solutions for the node-making process in published reports of NMAs. Enhancing recommendations on the
Acknowledgments
The authors acknowledge David Moher and Perrine Créquit for critical revision of the manuscript. They also thank Laura Smales for English proofreading.
References (62)
- et al.
Bibliographic study showed improving statistical methodology of network meta-analyses published between 1999 and 2015
J Clin Epidemiol
(2017) - et al.
Epidemiology and reporting of randomised trials published in PubMed journals
Lancet
(2005) - et al.
Introducing a series of methodological articles on considering complexity in systematic reviews of interventions
J Clin Epidemiol
(2013) - et al.
Mixed treatment comparison analysis provides internally coherent treatment effect estimates based on overviews of reviews and can reveal inconsistency
J Clin Epidemiol
(2010) - et al.
Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision Making : an ISPOR-AMCP-NPC Good Practice Task Force report
Value Health
(2014) - et al.
Agency for healthcare research and quality evidence-based practice center methods for systematically reviewing complex multicomponent health care interventions
J Clin Epidemiol
(2014) - et al.
Intra-articular corticosteroids may be a cost-effective strategy for short-term management of knee osteoarthritis
Osteoarthritis Cartilage
(2015) - et al.
Percutaneous coronary interventions for non-acute coronary artery disease: a quantitative 20-year synopsis and a network meta-analysis
Lancet
(2009) - et al.
Mixed treatment comparison meta-analysis of altered fractionated radiotherapy and chemotherapy in head and neck cancer
J Clin Epidemiol
(2011) - et al.
Comparison among drug-eluting balloon, drug-eluting stent, and plain balloon angioplasty for the treatment of in-stent restenosis: a network meta-analysis of 11 randomized, controlled trials
JACC Cardiovasc Interv
(2015)
Clinical outcomes with bioabsorbable polymer- versus durable polymer-based drug-eluting and bare-metal stents: evidence from a comprehensive network meta-analysis
J Am Coll Cardiol
Percutaneous coronary interventional strategies for treatment of in-stent restenosis: a network meta-analysis
Lancet
AHRQ series on complex intervention systematic reviews—paper 6: PRISMA-CI extension statement and checklist
J Clin Epidemiol
Systematic reviews: synthesis of best evidence for clinical decisions
Ann Intern Med
Validity of indirect comparison for estimating efficacy of competing interventions: empirical evidence from published meta-analyses
BMJ
Lack of head-to-head trials and fair control arms
Arch Intern Med
Demystifying trial networks and network meta-analysis
BMJ
Combination of direct and indirect evidence in mixed treatment comparisons
Stat Med
Methods and processes of the CONSORT group: example of an extension for trials assessing nonpharmacologic treatments
Ann Intern Med
Can we systematically review studies that evaluate complex interventions?
PLoS Med
Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials
BMJ
Intervention synthesis: a missing link between a systematic review and practical treatment(s)
PLoS Med
Indirect comparisons for evaluation healthcare interventions: Review of published systematic reviews and discussion of methodological problems
Simultaneous comparison of multiple treatments: combining direct and indirect evidence
BMJ
Evaluation of networks of randomized trials
Stat Methods Med Res
Extending methods for investigating the relationship between treatment effect and baseline risk from pairwise meta-analysis to network meta-analysis
Stat Med
Network meta-analysis models to account for variability in treatment definitions: application to dose effects
Stat Med
Evidence synthesis for decision making 4: inconsistency in networks of evidence based on randomized controlled trials
Med Decis Making
GetReal in network meta-analysis: a review of the methodology
Eur J Cancer
Fluid type and the use of renal replacement therapy in sepsis: a systematic review and network meta-analysis
Intensive Care Med
The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations
Ann Intern Med
Cited by (26)
Treatment classifications and interventions for neck pain: a scoping review
2023, Journal of Clinical EpidemiologySome conservative interventions are more effective than others for people with chronic non-specific neck pain: a systematic review and network meta-analysis
2022, Journal of PhysiotherapyCitation Excerpt :When the I2 value was > 90% (ie, high degree of heterogeneity), we did not perform meta-analysis.36,37 According to the PRISMA-NMA guideline,24 the process leading to node grouping38 and rationale for node adopted are displayed in Appendix 3 on the eAddenda. We first evaluated the transitivity assumption,39-41 defined as the balance between the distribution of potential effect modifiers (age, sex, mean pain duration, presence of widespread pain, presence of somatisation, length of treatment, number of randomised participants, and baseline pain intensity)42 across pairwise comparisons.
Critical evaluation of reporting quality of network meta-analyses assessing the effectiveness of acupuncture
2021, Complementary Therapies in Clinical PracticeCitation Excerpt :Only 33.3 % of acupuncture-NMAs specified treatment nodes included in the network and 21.4 % included justification for lumping or splitting nodes. Future researchers should prespecify eligibility criteria of studies and those for each node, including different components of acupuncture [41]. Based on clinical argument whether to investigate clinical effectiveness of acupuncture at a broad level or estimate treatment effects of specific acupuncture interventions, researchers should decide to lump or split and report the node-making process.
School-based interventions to prevent anxiety and depression in children and young people: a systematic review and network meta-analysis
2019, The Lancet PsychiatryCitation Excerpt :Our typology of interventions and control conditions was based on previous literature,29–32,35 piloting, and discussion in our team. Nevertheless, we acknowledge that the process of categorising complex interventions is subjective40 and future work should agree on a process for node-making as well as agreeing on a taxonomy for classifying psychological interventions. Reporting of experimental interventions was adequate for classification.
Node-making processes in network meta-analysis of nonpharmacological interventions should be well planned and reported
2018, Journal of Clinical Epidemiology
Conflict of interest: The authors have declared that no conflict of interest exists.
Funding: A.J. was funded by the French Society of Anesthesia and Intensive Care Medicine (SFAR) and the Fondation pour la Recherche Médicale (FRM). Neither funding sources had any influence at any stage of the study, writing of the manuscript or decision to publish.