Discussion
ADVERSE EVENTS, RECRUITMENT
Two-stage single-arm trials with the option to stop for futility are often planned according to Simon’s design. However, if the primary outcome takes a long time to be observed, such as 12-month survival, Simon’s design implies a long interruption of patient recruitment to conduct the interim analysis. Kunz et al.The two proposed designs are both allowing for completely flexible timing of the interim analysis. In our simulations, the break between the two stages was of the same length as in Kunz’s design, and therefore considerably shorter than in Simon’s design, but that is not a requirement and it could also be shorter (or longer). The longer the break, the more patients will have complete follow-ups in the interim, so the more accurate the predictions will be. To correctly assess the operating characteristics of a specific trial, the length of the break must be prespecified in the planning stage. A (rather weak) assumption of the methods is that for each endpoint there should be at least one observation. If that is not the case, the survival probabilities need to be imputed, for example, by interpolating the assumed underlying survival curve. Our simulations have shown that assuming constant hazards over time work reliably also under non-constant hazards.The simulations showed that STE–CP and STE–PoS closely control the type I error rate in all considered scenarios with slow patient recruitment and substantially improve power compared to Kunz’s design in many situations. The price is a loss of power in the narrow window where the true parameters of short-term and long-term survival are exactly or very close to as postulated in the design stage. If the assumptions about the short-term survival from the design stage were correct, STE–CP and STE–PoS had lower power than Kunz’s design (between 0.75 and 0.9 compared to 0.95 under Kunz’s design). The more one is off with the assumptions in the design phase about the short-term survival, the greater the advantage of the new methods over Kunz’s design, because the power of Kunz’s design will drop drastically (down to zero) while the power of STE–PoS and STE–CP is barely affected. We emphasize that this is only true for slow patient recruitment and that for trials with fast recruitment, the methods would not be applicable, because then none or only little information on the long-term endpoint would be available for the interim analysis. The extent to which a trial could potentially benefit from STE–CP and STE–PoS depends also on the hypothesis to be tested: if the long-term survival rate is assumed to be rather low, the parameter space of the short-term survival rate is rather large, and hence the benefit in terms of power and type I error rate could be quite large (as seen from scenario 1 in our simulations). If the long-term survival rate is greater, the parameter space of the short-term survival rates gets smaller, hence less power can be gained (as seen from scenario 2 and particularly scenario 3).If prior information is available, one might want to express this by informative priors. We have presented a simple way of how this can be done with STE–PoS in principle. Adjusting the prior distributions to express certain prior information is easily done, and as expected optimistic informative priors led to decreased PET, and increased type I error rate and power. Interestingly, the stability across different short-term survival rates diminished compared to weakly informative priors, which is a sign of a lack of robustness against deviations from the assumed relations between short-term and long-term survival rates. We would like to emphasize that the origin of this work was the situation of extremely scarce information, so weakly informative priors will be more appropriate in the intended applications, and the formulation of informative priors will require additional research before applicable in practice.The differences between the operating characteristics of STE–CP and STE–PoS (with weakly informative priors) were small, so they are more relevant on a philosophical level. The distribution of CP values under STE–CP showed a strong tendency toward extreme values close to or equal to 0 or 1 compared to the distribution of PoS values under STE–PoS, which can be explained by the fact that STE–CP uses a single-point estimate of the probability of survival to predict new data, while STE–PoS uses the whole posterior distribution.One should note that we have considered binding stopping rules, so our results are only valid if one follows the stopping rule. There are good arguments to have a non-binding stopping rule, the most important one being able to react flexibly to information not considered in the stopping rule, such as adverse events.Another aspect is that for Kunz’s design, computationally efficient closed-form solutions exist, which implies an important practical advantage. For STE–PoS and STE–CP, calculations are much more time-consuming. Although it would be possible to calculate exact operating characteristics, the number of possible outcomes is too high to be computationally feasible even for the small sample size and the few measurement time points that were considered here. Therefore, we relied on simulations. For the same reason, searching for suitable designs in the whole parameter space does not seem to be feasible at the moment. Therefore, we suggest first finding the optimal Kunz design with a somewhat higher power than desired and then use this design as starting point for simulating the operating characteristics of STE–PoS or STE–CP.When planning a trial, one should carefully consider whether the advantage of a faster conduct of the trial is worth the more complicated trial design compared to the relatively simple Simon’s two-stage design, which also has a completely stable type I error rate and power regardless of any other endpoints. In our discussions with clinical trial teams, the time advantage appeared often to be desirable, but the trade-off may be different for each situation. With this study, we hope to provide the required tools when facing the dilemma between an undesirably long interruption under Simon’s design and the risk of an underpowered study under Kunz’s design due to uncertainty about the short-term endpoint.In principle, the presented methods are not limited to two-stage clinical trials with stopping for futility but could be applied to any setting in which one or more interim analyses are conducted and a survival endpoint with discrete measurement times is estimated. As they do not include a decision rule for the final analysis, such final decision rules need to be developed for the specific situation. Also, we have not considered the use of baseline covariates, as proposed recently in the context of randomized clinical trials.