Discussion
Non-proportionality
In this article, we have compared three population-level summary measures for clinical trials with time-to-event outcomes when analysed under a correct proportional hazards assumption.When the hazards are proportional, analysing the data with a model that reflects their proportionality leads to big improvement in power, compared to the standard approach of estimating DRMST and DS non-parametrically.When using standard methods for estimation, that is, Cox models for HR and non-parametric methods for DRMST and DS, the relative performance of summary measures varies substantially depending on the design parameters, as previously observed.For superiority trials, the null hypotheses correspond for all three summary measures, and therefore testing on any summary measure leads to similar power levels.For non-inferiority trials, DRMST assuming proportional hazards is almost always at least as powerful as – and often more powerful than – HR, except in rare cases, for example, in scenario 21, where survival probability at τ approaches zero and the non-inferiority margin is large. The relative performance for DS depends on design parameters.Thus, some of the apparent differences between HR and DRMST observed in Freidlin et al.In this article, we have assumed proportional hazards throughout. Non-proportionality of hazards may potentially have huge impact in terms of power and/or type I error. Therefore, future work will investigate this and possibly derive specific methods to address it. For example, weighting could be used to get an unbiased estimate of the average HRThe delta method that we used for estimating standard errors for DRMST and DS from the flexible parametric model relies on asymptotic arguments and can hence be poorly calibrated with smaller sample sizes. Preliminary simulations suggest that a non-parametric bootstrap strategy might lead to better control of type I error; however, while this might be preferable for a single analysis, it was too computationally intensive for this simulation study. Further, the non-parametric bootstrap itself relies on asymptotic arguments so would need to be studied further. Importantly, while type 1 error might be wrongly controlled in some of our scenarios, this did not appear to have the potential to have any impact on the conclusions of our study in terms of power.While most trials that target the HR use Cox models, we used flexible parametric models here for comparability of models across summary measures, since we could not find an analytic way to estimate standard errors around DRMST based on Cox model estimates without resorting to bootstrap. Nevertheless, results of separate simulations (not shown) confirm