Abstract
To assess how common it is for a published network meta-analysis (NMA) to have other published overlapping NMAs, and to evaluate these overlaps.
A total of 88 NMAs of randomized controlled trials evaluating the comparative effectiveness of health interventions were randomly selected. For each of these, we searched for NMAs on the same topic. A random sample of 40 pairs (an index NMA and one of its overlapping NMAs) was selected to assess the overlap in terms of nodes, treatments and references. The topic with the largest number of overlapping NMAs was described in depth.
In all, 68 of the 88 index NMAs had at least one overlapping NMA: 77% [95% confidence interval (CI), 69–86%]. We identified 515 pairs of overlapping NMAs. Among the 40 randomly selected pairs, 73% (95% CI, 58–88%) of nodes, 79% (95% CI, 72–86%) of treatments and 48% (95% CI, 37–59%) of references included in the index NMAs were also found in the respective overlapping NMAs. Efficacy of biologics in rheumatoid arthritis had the largest number of overlapping NMAs, with 28 NMAs published between 2003 and 2014. Differences in selection and definition of nodes of treatments resulted in different network geometries. There were also differences in both the direction and the statistical significance of effects.
Published NMAs exhibit extensive overlap and potential redundancy. Erratic retrieval of eligible trials, and lack of consensus on the range of interventions to be considered and how they might be merged or split in different nodes, may cause confusion.
Published network meta-analyses (NMAs) exhibit extensive overlap and potential redundancy.
Our data suggests a lot of fragmentation of effort and also fragmentation of the evidence, even though NMAs are supposed to examine the big picture of all interventions.
Lack of consensus on the range of interventions to be considered and how they might be merged or split in different nodes may result in different network configurations and may add variability in effect estimates across NMAs. Further studies are needed to explore this point.
Introduction
Network meta-analysis (NMA) is a very influential evidence synthesis technique that has become increasingly popular among public health decision makers, the industry and clinicians who have to choose the best therapeutic option for a given patient. This technique is useful to explore the comparative effectiveness of interventions that may or may not have been evaluated directly against each other.1
The development of automated statistical packages2 that facilitate NMA enables the broader application of this complex evidence synthesis technique. Since the number of NMAs is rapidly expanding,3–4 there is chance of overlap between different NMAs. Some overlap may be desirable, e.g. when an NMA is updated with new evidence that may change previous conclusions. Also, some independent corroboration of the same topic by different NMAs may be useful. However, large-scale redundancy may indicate waste of research efforts.
Such redundancy has been documented for pairwise meta-analyses.5 A similar phenomenon might thus be expected for NMAs, although some possible differences may be expected due to the additional degree of complexity of the methods used and variability on which interventions are considered in the nodes of a network.
Here, we assessed how common it is for a published NMA of randomized controlled trials to have other published overlapping NMAs, and investigated the characteristics of these overlaps.
Methods
Identification of index NMA: search strategy and eligibility criteria
We used the 822 NMAs identified previously in a survey.6 In that survey, PubMed was searched from inception until 6 May 2015 for NMAs of randomized controlled trials published without any limitation of date and without language restrictions, using the following search terms: ‘(network[tiab] AND (meta analys*[tiab])) OR indirect comparison*[tiab] OR indirect treatment comparison* [tiab] OR multiple treatment comparison*[tiab] OR mixed treatment comparison*[tiab]’. Eligible studies were NMAs of randomized trials evaluating the comparative effectiveness of preventive or therapeutic health interventions for any condition. NMAs pertaining to safety of interventions exclusively, prognostic associations, NMAs about diagnostic accuracy and methodological studies were excluded. Eligibility was assessed by two independent reviewers (F.N. and E.S.).
Instead of assessing overlap for all 822 identified NMAs (a formidable task), we drew a random sample. Based on a survey of overlapping pairwise meta-analyses5 and on a pilot on 15 NMAs, we anticipated that nearly 65% of the index NMAs would have at least one overlapping NMA. Therefore, we calculated that a random sample of 88 NMAs was sufficient to achieve an expected precision of 10% for the percentage of NMAs with overlapping NMAs.
Overlapping NMAs: search strategy and eligibility criteria
For each of the 88 index NMAs included, two reviewers (F.N. and E.S.) independently searched for any overlapping publication within the 822 NMAs with the same eligibility criteria. To be considered overlapping, the NMAs had to come from the same topic by addressing the same disease with the same indication of treatment. Whenever one NMA of the pair included a subgroup of patients included in the other NMA, but with the same therapeutic indication, these two NMAs were considered as pertaining to the same topic. In addition, three nodes (i.e. treatments or combinations of treatments compared within the NMA) of the index NMA had to be included in the second NMA (either as independent nodes or lumped in one or two nodes) to consider the pair as overlapping.
Data collection and data analysis
Apart from investigating the overall frequency of overlapping NMAs and the extent of overlap, we decided to assess in depth two specific subsets of NMAs: a random sample of overlapping pairs of NMAs, and the topic with the largest number of overlapping NMAs.
Random sample of pairs
To explore in depth characteristics of overlaps, a random sample of 40 pairs of overlapping NMAs (an index NMA and one of its overlapping NMAs) was selected from all the different pairs that we have identified. Two independent reviewers (F.N. and E.S.) extracted characteristics of both NMAs within each pair, including the publication year, the type of condition (and when applicable, any differences in terms of study population, e.g. diabetics vs general population) and the numbers of nodes, treatments and references, and the overlap of these items between NMAs. Following a comment formulated during the peer review process, one reviewer (F.N.) extracted the number of outcomes and their overlap between NMAs within each pair.
Efficacy of biologics for rheumatoid arthritis (topic with largest number of NMAs)
For each NMA on the efficacy of biologics for the treatment of rheumatoid arthritis, two independent reviewers (F.N. and E.S.) extracted data on year of publication, type of funding, patient characteristics, network geometry (i.e. indirect star-shaped network or network with at least one direct comparison between treatments), treatments considered and numbers of included studies, patients and nodes. One reviewer (F.N.) extracted the references of the primary publications of individual randomized controlled trials included in each NMA, and searched PubMed until the date of publication of the most recently published NMA on the topic for all randomized controlled trials (RCTs) comparing two or more interventions investigated in this topic. If applicable, the choices authors of NMAs made to lump together different therapeutic strategies within one node within the network of evidence, potentially resulting in different network geometries,7 were extracted. Network geometry or evidence structure means the configuration of the network, i.e. the nodes considered and comparisons that have been made.
Concerning efficacy, the two reviewers extracted results of comparisons of the treatment most frequently included in the NMAs within the topic with other treatments, to explore whether the findings varied between different NMAs.
For each of these steps (searches and extraction), any disagreement between the two reviewers (F.N. and E.S.) was resolved by consensus or in consultation with a third reviewer (J.P.A.I.).
Data analysis
Quantitative data are presented with median and interquartile range (IQR), and categorical data with number and percentages. Estimations of overlap are presented as percentage or mean with their 95% confidence intervals (95% CIs). All analyses were performed with R, version 3.2.1 (R Development Core Team, Vienna). Data and code are publicly accessible on the Open Science Framework (https://osf.io/289kj/).
Patient involvement
No patients were involved in setting the research question or the outcome measures, nor were any involved in the design or implementation of the study. There are no plans to involve patients in the dissemination of results, nor will we disseminate results directly to patients. Therefore, ethics approval is not required.
Results
Random sample of index NMAs
To achieve our random sample of 88 NMAs, we screened 100 index NMAs and excluded 12 because they did not meet eligibility criteria upon scrutiny of the full text, i.e. safety of interventions only (n = 8), or diagnostic accuracy NMAs (n = 4). Details of this process are presented in Supplementary Figure 1 (available as Supplementary data at IJE online). There were 68 out of the 88 index NMAs with at least one overlapping NMA [77% (95% CI, 69%; 86%)]. These 68 NMAs belonged to 49 different topics. We identified 515 pairs of overlapping NMAs, corresponding to a median number of overlapping NMAs of 4 (IQR 2–10). Figure 1 summarizes the number of overlapping NMAs per index NMA; 15 (17%) of index NMAs had only one overlapping NMA, 33 (38%) had five or more overlapping NMAs and 19 (22%) index NMAs had 10 or more overlapping NMAs. Because sometimes different index NMAs pertained to the same topic, the 19 NMAs with 10 or more overlapping NMAs were from six distinct topics: biologics for rheumatoid arthritis (n = 28 overlapping NMA), stents for coronary artery disease (n = 23), anticoagulants for atrial fibrillation (n = 20), drugs for chronic obstructive pulmonary disease exacerbation prevention (n = 13), drugs for osteoporosis treatment (n = 12) and antidiabetic agents in type 2 diabetes (n = 11).
Distribution of the number of overlapping network meta-analyses per index network meta-analysis.
Random sample of pairs
Among the 515 pairs of overlapping NMAs, 40 were randomly selected to explore in depth the degree of overlap. The 40 pairs belonged to 15 different topics from various medical disciplines (see Supplementary Table 1, available as Supplementary data at IJE online). The median number of years between overlapping NMAs was 1.5 (IQR 1–4). In 33 (83%) pairs, both NMAs were performed on the same population, and in the other seven one of the NMAs focused on a subgroup of patients (e.g. diabetics only) as compared with the other NMA in the pair.
For 23/40 (58%) pairs, the exact overlap in terms of nodes was confounded by differences between NMAs in defining the nodes: in 16/40 (40%) pairs, treatments were lumped by class in one of the NMAs; in 4/40 (10%) pairs, different doses or treatment durations of the same treatment were split in separate nodes in one of the NMAs; and in 3/40 (8%) pairs, authors of one NMA concluded that NMAs were not possible (e.g. because of lack of transitivity due to different effect modifier distributions across included studies) or failed to present the nodes in a meaningful way. Totals of 73% (95% CI, 58; 88%) of nodes (for the 17 pairs where applicable), 79% (95% CI, 72: 86%) of treatments and 48% (95% CI, 37; 59%) of references included in the index NMAs were also found in the respective overlapping NMAs. Across these NMAs, the percentage of outcomes of the index NMA also used in the second NMA was high (with a median overlap of 82%). Only 6/40 pairs had no outcome in common [15% (95% CI, 4–26%)].
Biologics in rheumatoid arthritis
Efficacy of biologics in rheumatoid arthritis had the largest number of identified NMAs, with 28 overlapping NMAs published between 2003 and 2014 (see Supplementary Figure 2, available as Supplementary data at IJE online). Supplementary Table 2 (available as Supplementary data at IJE online) details each individual NMA of this topic, including the outcomes that were studied. Table 1 summarizes their principal characteristics. All NMAs considered inadequate responders to disease-modifying anti-rheumatic drugs (DMARD), but some additionally considered naïve patients (in two separate analyses or in the same analysis). The treatments included in these NMAs are presented in Figure 2A. In terms of treatments included, no NMA was exhaustive. Only one NMA considered combinations of different conventional DMARDs (double or triple). When biologic drugs only were considered, only three NMAs included all biologics that had at least one published trial at the time the NMA was published (PMIDs 21182357, 21848493 and 23092868). The number of nodes per NMA increased with the years, but this increase was less evident after 2011 (see Supplementary Figure 3, available as Supplementary data at IJE online).
Characteristics of network meta-analyses on biologic agents in rheumatoid arthritis
| Characteristics | Values (n = 28) |
|---|---|
| Funding | Missinga |
| Non-profit funding sources | 9 (43%) |
| Company/Biotech | 8 (38%) |
| No specific grant | 4 (19%) |
| Year | 2012 (2010– 2013) |
| Type of MA | |
| Star shaped network (centred on placebo) | 20 (71%) |
| At least one direct comparison between active drugs | 7 (25%) |
| Not possible to perform the indirect meta-analysis | 1 (4%) |
| Population included | |
| Not naïve to DMARDs | 18 (64%) |
| Naïve and not naïve, pooled in analyses | 8 (29%) |
| Naïve and not naïve, analysed separately | 2 (7%) |
| Number of nodes | 7 (5–9)b |
| Number of patients included | 7076 (5756– 10810)b |
| Number of studies included | 18 (13–27) |
| Anti-TNFs are lumped together | 6 (21%) |
| Regimens with and without DMARDs are lumped together | 12 (44%) |
| Characteristics | Values (n = 28) |
|---|---|
| Funding | Missinga |
| Non-profit funding sources | 9 (43%) |
| Company/Biotech | 8 (38%) |
| No specific grant | 4 (19%) |
| Year | 2012 (2010– 2013) |
| Type of MA | |
| Star shaped network (centred on placebo) | 20 (71%) |
| At least one direct comparison between active drugs | 7 (25%) |
| Not possible to perform the indirect meta-analysis | 1 (4%) |
| Population included | |
| Not naïve to DMARDs | 18 (64%) |
| Naïve and not naïve, pooled in analyses | 8 (29%) |
| Naïve and not naïve, analysed separately | 2 (7%) |
| Number of nodes | 7 (5–9)b |
| Number of patients included | 7076 (5756– 10810)b |
| Number of studies included | 18 (13–27) |
| Anti-TNFs are lumped together | 6 (21%) |
| Regimens with and without DMARDs are lumped together | 12 (44%) |
Quantitative data are presented with median and interquartile range, and categorical data as number and percentages.
aSeven missing data.
bIn one case it was not applicable because authors concluded that indirect comparisons were not possible.
Characteristics of network meta-analyses on biologic agents in rheumatoid arthritis
| Characteristics | Values (n = 28) |
|---|---|
| Funding | Missinga |
| Non-profit funding sources | 9 (43%) |
| Company/Biotech | 8 (38%) |
| No specific grant | 4 (19%) |
| Year | 2012 (2010– 2013) |
| Type of MA | |
| Star shaped network (centred on placebo) | 20 (71%) |
| At least one direct comparison between active drugs | 7 (25%) |
| Not possible to perform the indirect meta-analysis | 1 (4%) |
| Population included | |
| Not naïve to DMARDs | 18 (64%) |
| Naïve and not naïve, pooled in analyses | 8 (29%) |
| Naïve and not naïve, analysed separately | 2 (7%) |
| Number of nodes | 7 (5–9)b |
| Number of patients included | 7076 (5756– 10810)b |
| Number of studies included | 18 (13–27) |
| Anti-TNFs are lumped together | 6 (21%) |
| Regimens with and without DMARDs are lumped together | 12 (44%) |
| Characteristics | Values (n = 28) |
|---|---|
| Funding | Missinga |
| Non-profit funding sources | 9 (43%) |
| Company/Biotech | 8 (38%) |
| No specific grant | 4 (19%) |
| Year | 2012 (2010– 2013) |
| Type of MA | |
| Star shaped network (centred on placebo) | 20 (71%) |
| At least one direct comparison between active drugs | 7 (25%) |
| Not possible to perform the indirect meta-analysis | 1 (4%) |
| Population included | |
| Not naïve to DMARDs | 18 (64%) |
| Naïve and not naïve, pooled in analyses | 8 (29%) |
| Naïve and not naïve, analysed separately | 2 (7%) |
| Number of nodes | 7 (5–9)b |
| Number of patients included | 7076 (5756– 10810)b |
| Number of studies included | 18 (13–27) |
| Anti-TNFs are lumped together | 6 (21%) |
| Regimens with and without DMARDs are lumped together | 12 (44%) |
Quantitative data are presented with median and interquartile range, and categorical data as number and percentages.
aSeven missing data.
bIn one case it was not applicable because authors concluded that indirect comparisons were not possible.
Descriptive analysis of the topic of biologic agents in rheumatoid arthritis. A: Treatment arms included in the different NMAs. Network meta-analyses are identified by their PMID (PubMed-Indexed for MEDLINE) numbers and presented in chronological order; green colour indicates inclusion of the treatment in the network meta-analysis; black colour indicates non-inclusion of the treatment; white colour indicates non-availability of randomized evidence for the treatment at the time the NMA was conducted. B: Number of individual randomized controlled trials (biologics and conventional DMARDs) included in each NMA (black points) in comparison with their cumulative number included in all NMAs of the topic (black line) as a function of publication year. C: Number of individual randomized controlled trials of biologics included in each NMA (black points) in comparison with their cumulative number included in all NMAs of the topic (black line) as a function of publication year. The top line presents all RCTs comparing the interventions used in this topic identified in a PubMed search (see Appendix, available as Supplementary data at IJE online).
Figure 2B presents the number of trials referenced per NMA in comparison with the total number of randomized clinical trials included in at least one NMA of the topic. The biggest NMA included 44 studies out of the total of 101 individual studies (44%) included in at least one NMA of the topic (82/101 on biologics). Since combinations of conventional DMARDs were only considered in one NMA, Figure 2C only focuses on biologics. In addition, Figure 2C presents all 129 RCTs available at the time the NMAs were performed, regardless of whether they were included in any of the NMAs (red line). Both figures show that no NMA included all the evidence that was published at the time of publication of the NMA. Typically, only a minority of the available trials were included in the NMA. For biologics, the most inclusive NMA included 55% of available references (Supplementary Figure 4, available as Supplementary data at IJE online). The search date for evidence for the 28 NMAs was a median of 13.5 months (IQR 8.8–24.5) before publication. The number of published trials by the date of the search always exceeded substantially the number of trials included in each NMA. For biologics, the most inclusive NMA included 66% of references published 2 years before the publication of the NMA (Supplementary Figure 4, available as Supplementary data at IJE online).
Similar to the random sample of 40 pairs, network geometries differed in terms of nodes and connections between nodes. Differences originated primarily from two main methodological choices: lumping treatments together [anti-tumour necrosis factor (TNF) agents] or not, and lumping together treatments with or without co-administration of DMARDs. The resulting four variants of possible network geometries are illustrated by examples in Figure 3.
Illustration of differences in definitions of network geometry in the topic of biologic agents in rheumatoid arthritis All examples come from NMAs that are identified with their PMID. Anti-TNF drugs might be grouped by class (left panels) or not (right panels). Treatments with and without co-administration of DMARDs might be lumped together (lower panels) or not (top panels).
In all, 22 NMAs (79%) used the ACR (American College of Rheumatology) criteria as an outcome (21 studies used at least ACR 50 and one was only focused on ACR 20). Efficacy results (expressed as log odds ratios) observed for etanercept, the most frequently investigated treatment, as compared with each of the other treatments are presented in Figure 4. It shows differences between NMAs, in both the direction of the effect and the statistical significance of the effect (i.e. whether the reported 95% confidence intervals or credible intervals were entirely on the same side of the null).
Etanercept comparative effectiveness across network meta-analyses in the topic of biologic agents in rheumatoid arthritis. All comparisons between etanercept and comparators are presented here except in NMAs where etanercept was lumped with other anti-TNFs (n = 6), in NMAs where no binary outcome was reported (n = 4) and in one NMA with missing data. All but one explore the effect on ACR 50 (PMID 23092868 explores withdrawal for lack of efficacy). NMAs are identified by their PMID; odds ratios were extracted from binary outcomes and their logarithm is presented. In one publication (PMID 23269860), two different NMAs (one for combinations and one for monotherapies) were performed and thus two point estimates are presented for relevant comparisons. Errors bars represent 95% confidence or 95% credible intervals (as selected by the original network meta-analysts).
Discussion
Statement of principal findings
In this empirical study, we found that three-quarters of NMAs of randomized trials have at least one overlapping NMA. There are many topics where there are already 10 or more overlapping NMAs. Generally, overlapping NMAs were performed in the same population (and, less commonly, in more specific subgroups), and published in close time sequence. Treatment of rheumatoid arthritis is the most popular topic of NMA that we have identified, with 28 NMAs published on this topic. Different lumping or splitting of treatments in nodes often resulted in different network geometries. No NMA explored all available treatments for rheumatoid arthritis, and the ‘biggest’ NMA in terms of included studies examined 55% of the trials that had been published by the time the NMA appeared and 66% of the studies that had already been published 2 years before.
Altogether, these data suggest that many NMAs exhibit extensive overlap and potential redundancy. As suggested by the rheumatoid arthritis topic, each NMA may present a fragmented and incomplete picture of the total available evidence, and inferences about comparisons of drugs may vary across overlapping NMAs. Of course, the result observed in one specific topic may not be generalizable to NMAs performed in other topics. However, the divergent results we observed in this topic empirically suggest that NMAs can be prone to vibration of effects (VoE). VoE describes the extent to which treatment effects may change under multiple distinct analytical choices.8–9 Further work would be useful to explore VoE, by exploring how study selection and effects estimates are affected by different specific methodological choices.
Strengths and weaknesses of the study
Our survey of PubMed-indexed NMAs is comprehensive, but we may have missed some NMAs not listed in PubMed. Furthermore, we may have missed also some unpublished NMAs. A recent survey suggests that many NMAs are performed by consultancy firms and the large majority remain unpublished.6 Therefore, overlap and redundancy are potentially even larger than we describe.
We did not include the outcomes (type and time frame) in our basic definition of overlapping NMAs, since all studies were evaluating efficacy. We found a large overlap in terms of type of outcome, but some NMAs differed on what exact efficacy outcomes they considered, and 15% of the pairs of compared NMAs had no outcomes in common. The absence of widely adopted core outcomes in many diseases exacerbates this problem.10–12
Our study selection was based on clinical judgment. To limit the impact of such a subjective assessment, two reviewers assessed separately all the studies and had to agree. For this reason, we do not claim that the overlapping NMAs were necessarily redundant and unnecessary, because such a case-by-case qualitative assessment of utility might have been even more subjective. However, our data suggest a lot of fragmentation of effort and also fragmentation of the evidence, even though NMAs are supposed to examine the big picture of all interventions. It is clear that, in the topic of rheumatoid arthritis, NMAs were far from complete. It is difficult to know the exact extent to which studies were non-included due to narrow eligibility criteria vs search failure. Pragmatically though, no matter what the exact reason for not including studies, NMAs are still incomplete.
Strengths and weaknesses in relation to other studies, discussing important differences in results
Our study extends the results observed for overlap in traditional pairwise meta-analysis5,13 to NMAs. In NMAs, some methodological issues, such as lack of consensus on the range of interventions to be considered and how they might be merged or split in different nodes, might add another level of complexity and confusion. These eligibility and network design/geometry choices may affect estimated treatment effects in NMAs.14–15 Also, it is surprising to see how frequently treatments were lumped together. Collapsing interventions into broader classes might increase precision in the effect estimates, but might also add complexity in terms of interpretation and practical decision making.16 In addition, there are techniques available that allow for exploring nodes (as class) and subnodes (as treatments) simultaneously.17
It is not so surprising to find the rheumatoid arthritis topic as an emblematic example of redundancies across NMAs. In this field, there are numerous blockbuster drugs and the market is worth many billions of dollars.18–20 These drugs generate high profits, and they are mostly evaluated against placebos. Therefore, the paucity of head-to-head comparisons between drugs19,21 increases the interest in NMA for evidence of comparative effectiveness. An empirical evaluation performed in 201222 had already identified 13 multiple treatment comparisons on rheumatoid arthritis treatment, of which nine were published between 2009 and 2012. At that time, investigators had proposed methodological changes in order to achieve a more consistent synthesis of the available evidence. Since then the number of overlapping NMAs has grown, and we found 28 NMAs by the end of 2014. An updated search as of 1 November 2016 yielded another five NMAs about efficacy of biologics, a total of 28 + 5 = 33. This seems like a prolific factory of NMAs.13 We documented persistent methodological inconsistencies and lack of exhaustiveness, there has been no increase in the number of nodes considered per NMA after 2011 and no network considered all the potentially available randomized trials.
Meaning of the study: possible explanations and implications for clinicians and policy makers
There is considerable debate about the meaning of overlap, even for pairwise meta-analyses.5 Some potential overlap can be justified and the redundancy should sometimes be welcomed.23 For instance, updating is necessary when new influential trials appear. In NMAs, some new therapeutic options may appear and other older treatments may be abandoned, justifying a progressive evolution of network geometries to take into account the practical clinical context. There can be debate on whether a treatment or a trial can be considered ‘old’ and thus should be discarded.24 Finally, one or two independent replications of an NMA may be useful. Our study, however, suggests that the overlapping NMAs were generally published very closely in time and often there are numerous overlapping NMAs. This suggests potentially substantial wasted effort in this redundancy.
Ideally, NMAs should include all alternative treatments for a given condition.25 In rheumatoid arthritis, we identified a maximum of 101 randomized controlled trials that were synthesized in 28 different NMAs–among which 82 were about biologics–and a total of 129 relevant trials about biologics in PubMed. However, no NMA included more than 44 of these trials. Probably it would be preferable to aim for fewer, more inclusive NMAs.
Unanswered questions and future research
The extent of redundancy and wasted effort may be reduced with pre-registration of systematic reviews in general, and this may also apply to NMAs. At the level of pre-registration, PROSPERO26 allows researchers to check whether an NMA on their topic of interest is registered, and if so, if it is ongoing or completed. However, it has been shown that the vast majority of NMAs, especially those conducted by contractors, are currently not registered; and if they are registered, their status on PROSPERO is not up to date.6 NMA publications may also need to routinely record which other NMAs have been done on the same topic, justify why a new NMA is necessary, and indicate how the new NMA differs, agrees or disagrees with previous NMAs. The drivers behind the conduct of multiple NMAs also need better study. Meta-analyses (including NMAs) have become so influential that they can be powerful in shaping guidelines and clinical practice, and they can also be used as a marketing tool.27 Future work may compare the results and inferences of overlapping NMAs to see whether manufacturer sponsorship is important in shaping their design, results and interpretation.28 Cumulative network meta-analyses providing a broad, complete and updated presentation of the evidence regarding all the available management options, may also help decrease redundancy and confusion due to multiple NMAs with sometimes divergent results.29–30
Supplementary Data
Supplementary data are available at IJE online.
Funding
This work was supported by the Meta-Research Innovation Center at Stanford (METRICS). METRICS is funded by the Laura and John Arnold Foundation. In addition, F.N. received grants from La Fondation Pierre Deniker and Rennes University Hospital, France (CORECT: COmité de la Recherche Clinique et Translationelle). E.S. was supported by the Netherlands Organisation for Scientific Research [grant number 825.14.001]. The work of J.I. is supported by an unrestricted gift from Sue and Bob O’Donnell. The sponsors had no role concerning preparation, review or approval of the manuscript.
Author Contributions
Conceived and designed the experiments: F.N., E.S., J.P.A.I. Performed the experiments: F.N., E.S. Analysed the data: F.N. Interpreted the results: F.N., E.S., J.P.A.I. Wrote the first draft of the manuscript: F.N. Contributed to the writing of the manuscript: E.S., J.P.A.I. Agreed with the results and conclusions of the manuscript: F.N., E.S., J.P.A.I.
Conflict of interest: F.N. has relationships (travel/accommodations expenses covered/reimbursed) with Servier, BMS, Lundbeck and Janssen, who might have an interest in the work submitted in the previous 3 years. E.S. and J.P.A.I. have no relationship with any company who might have an interest in the work submitted. None of the authors has any non-financial interest that could be relevant to the submitted work.
