Treatment success in cancer: new cancer treatment successes identified in phase 3 randomized controlled trials conducted by the National Cancer Institute-sponsored cooperative oncology groups, 1955 to 2006. Effect of a US National Institutes of Health programme of clinical trials on public health and costs. Treatment success in pragmatic randomised controlled trials: a review of trials funded by the UK Health Technology Assessment programme. Thirty years of Medical Research Council randomized trials in solid tumours.
The evaluation of research output, such as estimation of the proportion of treatment successes, is of ethical, scientific, and public importance but has rarely been evaluated systematically. We assessed how often experimental cancer treatments that undergo testing in randomized clinical trials (RCTs) result in discovery of successful new interventions.
                We extracted data from all completed (published and unpublished) phase 3 RCTs conducted by the National Cancer Institute cooperative groups since their inception in 1955. Therapeutic successes were determined by (1) assessing the proportion of statistically significant trials favoring new or standard treatments, (2) determining the proportion of the trials in which new treatments were considered superior to standard treatments according to the original researchers, and (3) quantitatively synthesizing data for main clinical outcomes (overall and event-free survival).
                Data from 624 trials (781 randomized comparisons) involving 216 451 patients were analyzed. In all, 30% of trials had statistically significant results, of which new interventions were superior to established treatments in 80% of trials. The original researchers judged that the risk-benefit profile favored new treatments in 41% of comparisons (316 of 766). Hazard ratios for overall and event-free survival, available for 614 comparisons, were 0.95 (99% confidence interval [CI], 0.93-0.98) and 0.90 (99% CI, 0.87- 0.93), respectively, slightly favoring new treatments. Breakthrough interventions were discovered in 15% of trials.
                Approximately 25% to 50% of new cancer treatments that reach the stage of assessment in RCTs will prove successful. The pattern of successes has become more stable over time. The results are consistent with the hypothesis that the ethical principle of equipoise defines limits of discoverability in clinical research and ultimately drives therapeutic advances in clinical medicine. Few attempts have been made to estimate the public return on investment in medical research. The total costs and benefits to society of a clinical trial, the final step in testing an intervention, can be estimated by evaluating the effect of trial results on medical care and health.
                All phase III randomised trials funded by the US National Institute of Neurological Disorders and Stroke before Jan 1, 2000, were included. Pertinent publications on use, cost to society, and health effects for each studied intervention were identified by systematic review, supplemented with data from other public and proprietary sources. Regardless of whether a trial was positive or negative, information on use of tested therapies was integrated with published per-use data on costs and health effect (converted to 2004 US dollars) to generate 10-year projections for the US population.
                28 trials with a total cost of 335 million dollars were included. Six trials (21%) resulted in measurable improvements in health, and four (14%) resulted in cost savings to society. At 10 years, the programme of trials resulted in an estimated additional 470,000 quality-adjusted life years at a total cost of 3.6 billion dollars (including costs of all trials and additional health-care and other expenditures). Valuing a quality-adjusted life year at per-head gross domestic product, the projected net benefit to society at 10-years was 15.2 billion dollars. 95% CIs did not include a net loss at 10 years.
                For this institute, the public return on investment in clinical trials has been substantial. Although results led to increases in health-care expenditures, health gains were large and valuable. Previous research reviewed treatment success and whether the collective uncertainty principle is met in RCTs in the US National Cancer Institute portfolio. This paper classifies clinical trials funded by the UK HTA programme by results using the method applied to the US Cancer Institute trials, and compares the two portfolios.
                Data on all completed randomised controlled trials funded by the HTA programme 1993-2008 were extracted. Each trial's primary results was classified into six categories; 1) statistically significant in favour of the new treatment, 2) statistically significant in favour of the control treatment 3) true negative, 4) truly inconclusive, 5) inconclusive in favour of new treatment or 6) inconclusive in favour of control treatment. Trials were classified by comparing the 95% confidence interval for the difference in primary outcome to the difference specified in the sample size calculation. The results were compared with Djulbegovic's analysis of NCI trials.
                Data from 51 superiority trials were included, involving over 48,000 participants and a range of diseases and interventions. 85 primary comparisons were available because some trials had more than two randomised arms or had several primary outcomes. The new treatment had superior results (whether significant or not) in 61% of the comparisons (52/85 95% CI 49.9% to 71.6%). The results were conclusive in 46% of the comparisons (19% statistically significant in favour of the new treatment, 5% statistically significant in favour of the control and 22% true negative). The results were classified as truly inconclusive (i.e. failed to answer the question asked) for 24% of comparisons (20/85). HTA trials included fewer truly inconclusive and statistically significant results and more results rated as true negative than NCI trials.
                The pattern of results in HTA trials is similar to that of the National Cancer Institute portfolio. Differences that existed were plausible given the differences in the types of trials -HTA trials are more pragmatic. The results indicate HTA trials are compatible with equipoise. This classification usefully summarises the results from clinical trials and enables comparisons of different portfolios of trials. This paper reviews the survival outcome from the randomized Phase III trials in solid tumours published on behalf of, or in collaboration with, the Cancer Therapy Committee (CTC) of the British Medical Research Council over a 30-year period to 31 December 1995. We review briefly the innovations in statistical methodology that have occurred over the period. We also note the ways in which standards of reporting the trials have improved, with more recent publications including, for example, estimates of the size of effect and confidence intervals. In all, 32 trials, involving over 5000 deaths in more than 8000 patients, have been published. Tumour types have included bladder, bone, brain, cervix, colon and rectum, head and neck, kidney, lung, ovary, prostate and skin. This paper presents a bibliography of these trials and gives details of the treatment comparisons made, the numbers of patients randomized and included in the analysis for each treatment arm, the observed numbers of deaths, and an estimate of the hazard ratio with associated 95% confidence intervals. The bibliography also indicates the main endpoint of each trial, whether recurrence-free survival or survival, and whether the trial was aimed at finding a difference or showing equivalence. The MRC trials have made an impact on both clinical practice and research activities. For example, the lung cancer programme has helped to establish the role of chemotherapy in small cell lung cancer and has developed better palliative treatment for non-small cell lung cancer. Trials of the radiosensitizer misonidazole have demonstrated that it has no role in the treatment of a number of cancers, trials of hyperbaric oxygen have defined the biological activity of this approach, and the appropriate dose of radiotherapy in patients with brain tumours has been found. The individual trials recruited between 44 and 824 patients (median 213). A better measure of the information in a trial is the number of deaths reported, which varied from 28 to 661 (median 145). A large proportion of the comparisons (8/29 or 28%) anticipating a survival difference, demonstrated such a difference at the 5% level of significance. Despite this, it is concluded that some of the trials should have been larger. In such cases, hindsight suggests either that an overoptimistic view of the anticipated survival benefit was taken at the design stage, or, for equivalence trials, the planned confidence interval was too wide for definitive statements to be made. As a consequence, the current CTC profolio of ongoing randomized trials open to patient accrual at 1 January 1996 have a projected median size of 600 and range from 120 to 2000 patients.
Society can expect that slightly more than half of new experimental treatments will prove to be better than established treatments when tested in RCTs, but few will be substantially better. This is an important finding for patients (as they contemplate participation in RCTs), researchers (as they plan design of the new trials), and funders (as they assess the 'return on investment'). Although we provide the current best evidence on the question of expected 'success rate' of new versus established treatments consistent with a priori theoretical predictions reflective of 'uncertainty or equipoise hypothesis', it should be noted that our sample represents less than 1% of all available randomized trials; therefore, one should exercise the appropriate caution in interpretation of our findings. In addition, our conclusion applies to publicly funded trials only, as we did not include studies funded by commercial sponsors in our analysis.