Cumulating evidence from randomized trials: utilizing sequential monitoring boundaries for cumulative meta-analysis. Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care. Cumulative meta-analysis of therapeutic trials for myocardial infarction. Identifying null meta-analyses that are ripe for updating. A survey of clinical trials of antibiotic prophylaxis in colon surgery: evidence against further use of no-treatment controls. Uncertainty of the time of first significance in random effects cumulative meta-analysis. Recursive cumulative meta-analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data. Preparing and updating systematic reviews of randomized controlled trials of health care. Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses.
We propose the adaptation of classical monitoring boundaries for use in cumulative meta-analysis as guidelines for deciding when accumulating evidence is statistically significant and medically convincing. The interpretation of information from a randomized controlled trial is compared with that from a meta-analysis. The concept of optimal information size for a meta-analysis is developed and used to adapt monitoring boundaries to cumulative meta-analysis. Cumulative meta-analysis of clinical trials (a Bayesian interpretation for accumulating evidence) will profoundly affect medical care by summarizing evidence in the assessment of technology innovations. Application of the technique to the randomized control trials (RCTs) of streptokinase treatment of acute myocardial infarction, reduction of peri-operative mortality by antibiotic prophylaxis, and prevention of death from bleeding peptic ulcers has revealed efficacy years before it was suspected by any other means. Arrangement of the trials according to event rate in the controls, effect sizes, quality of the trials or according to covariables of interest has supplied unique information. If carried out prospectively the technique supplies invaluable information regarding indications for another trial, the number of patients necessary to determine the validity of past trends, and the type of patients who might be benefitted. Careful examination in a cumulative manner of the prior trials can reduce the need for future large trials. The large volume of published randomized, controlled trials has led to a need for meta-analyses to track therapeutic advances. Performing a new meta-analysis whenever the results of a new trial of a particular therapy are published permits the study of trends in efficacy and makes it possible to determine when a new treatment appears to be significantly effective or deleterious. We describe the use of such a procedure, cumulative meta-analysis, to assess therapeutic trials among patients with myocardial infarction.
                We performed cumulative meta-analyses of clinical trials that evaluated 15 treatments and preventive measures for acute myocardial infarction.
                An example of this method is its application to the use of intravenous streptokinase as thrombolytic therapy for acute infarction. Thirty-three trials evaluating this therapy were performed between 1959 and 1988. We found that a consistent, statistically significant reduction in total mortality (odds ratios, 0.74; 95 percent confidence interval, 0.59 to 0.92) was achieved in 1973, after only eight trials involving 2432 patients had been completed. The results of the 25 subsequent trials, which enrolled an additional 34,542 patients through 1988, had little or no effect on the odds ratio establishing efficacy, but simply narrowed the 95 percent confidence interval. In particular, two very large trials, the Gruppo Italiano per lo Studio della Streptochinasi nell'Infarto Miocardico trial in 1986 (11,712 patients) and the Second International Study of Infarct Survival trial in 1988 (17,187 patients) did not modify the already established evidence of efficacy. We used a similar approach to study the accumulating evidence of efficacy (or lack of efficacy) of 14 other therapies and preventive measures for myocardial infarction.
                Cumulative meta-analysis of therapeutic trials facilitates the determination of clinical efficacy and harm and may be helpful in tracking trials, planning future trials, and making clinical recommendations for therapy. As an increasingly large number of meta-analyses are published, quantitative methods are needed to help clinicians and systematic review teams determine when meta-analyses are not up to date.
                We propose new methods for determining when non-significant meta-analytic results might be overturned, based on a prediction of the number of participants required in new studies. To guide decision making, we introduce the "new participant ratio", the ratio of the actual number of participants in new studies to the predicted number required to obtain statistical significance. A simulation study was conducted to study the performance of our methods and a real meta-analysis provides further evidence.
                In our three simulation configurations, our diagnostic test for determining whether a meta-analysis is out of date had sensitivity of 55%, 62%, and 49% with corresponding specificity of 85%, 80%, and 90% respectively.
                Simulations suggest that our methods are able to detect out-of-date meta-analyses. These quick and approximate methods show promise for use by systematic review teams to help decide whether to commit the considerable resources required to update a meta-analysis. Further investigation and evaluation of the methods is required before they can be recommended for general use. To evaluate the use of antibiotics given prophylactically of colon surgery, we examined 26 trials published from 1965 to 1980 in which patients given various antibiotic regiments were compared with controls given no antibiotic treatment. In 22 (85 per cent of these trials) antibiotics reduced postoperative wound infection (p less than 0.05 in 14). Combining the results of the trials published from 1965 to 1975 reveals a 95 per cent confidence interval from the true difference in infection rates of 14 +/- 6 per cent (36 per cent for control group vs. 22 per cent for treatment group) and the true difference in death rates of 6.7 +/- 4.4 per cent (11.2 per cent for control group vs 4.5 per cent for treatment group). Yet trials employing control groups given no treatment continue to be reported. Since the use of such controls is justified only when no effective alternative therapy exists, we believe that any further trials of antibiotic prophylaxis in colon surgery should employ a previously proved standard. However, steadily increasing efficacy of treatment means that comparisons of new therapies with standard therapies will become prohibitively expensive because of the large number of patients required. We propose a process for evaluating quantities of clinical and statistical interest in cumulative meta-analysis. We use Monte Carlo simulation studies to assess the error variance of the time to significance in cumulative meta-analysis. For the specific cumulative meta-analyses that we simulated, the 95% confidence interval of the treatment effect, estimated by the random effects method at the time of earliest significance, appears to be approximately appropriate except when the hypothesized treatment effect is near the null. The Monte Carlo approach that we used can also estimate the power of a cumulative meta-analysis when two treatments differ in efficacy. We illustrate these issues in the context of five cumulative meta-analyses, each comparing two treatments for preventing mortality from myocardial infarction. These simulated meta-analyses demonstrate our main point, which is that the time of first significance, however parameterized, is itself a random variable with error variance. Meta-analyses of randomized evidence may include published, unpublished, and updated data in an ongoing estimation process that continuously accommodates more data. Synthesis may be performed either with group data or with meta-analysis of individual patient data (MIPD). Although MIPD with updated data is considered the gold standard of evidence, there is a need for a careful study of the impact different sources of data have on a meta-analysis and of the change in the treatment effect estimates over sequential information steps. Unpublished data and late-appearing data may be different from early-appearing data. Updated information after the end of the main study follow-up may be affected by cross-overs, missing information, and unblinding. The estimated treatment effect may thus depend on the completeness and updating of the available evidence. To address these issues, we present recursive cumulative meta-analysis (RCM) as an extension of cumulative metaanalysis. Recursive cumulative meta-analysis is based on the principle of recalculating the results of a cumulative meta-analysis with each new or updated piece of information and focuses on the evolution of the treatment effect as a more complete and updated picture of the evidence becomes available. An examination of the perturbations of the cumulative treatment effect over sequential information steps may signal the presence of bias or heterogeneity in a meta-analysis. Recursive cumulative meta-analysis may suggest whether there is a true underlying treatment effect to which the meta-analysis is converging and how treatment effects are sequentially altered by new or modified evidence. The method is illustrated with an example from the conduct of an MIPD on acyclovir in human immunodeficiency virus infection. The relative strengths and limitations of both metaanalysis of group data and MIPD are discussed through the RCM perspective. People providing health care tend to turn to review articles rather than reports of primary research when seeking information to guide their practices. For this reason, the process of reviewing the results of primary research must respect scientific principles. Reviews must be kept up to date and disseminated in appropriate ways to the people who make decisions about health care, including policy makers, practitioners, and users of the health services. This article describes the methods developed and used in an attempt to address these challenges for care during pregnancy and childbirth. Evidence on how much medical interventions work may change over time. It is important to determine what fluctuations in the treatment effect reported by randomized trials and their metaanalyses may be expected and whether extreme fluctuations signal future major changes. We applied recursive cumulative metaanalysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 interventions in two medical fields (pregnancy/perinatal medicine, n = 45 interventions; myocardial infarction, n = 15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes greater than 2.5 standard deviations of the expected. With 500 accumulated patients, the pooled odds ratio may change by 0.6- to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74- and 1.35-fold for pregnancy/perinatal medicine and between 0.83- and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of preeclampsia), where recent mega-trials have contradicted prior metaanalyses, as well as in four other examples where early large treatment effects were dissipated when more data appeared. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their metaanalyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may sometimes signal further major changes in the future.
Little research has been conducted on when and how to update systematic reviews and the feasibility and efficiency of the identified approaches is uncertain. These shortcomings should be addressed in future research.