Model development, validation and testing
overfitted
DTS
The median of each feature was used to fill in the missing values, preserving the distribution of data to the greatest extent. However, underestimation of the weight of features using this method is possible. All patients and their specific characteristics in this cohort were randomly divided into the training and testing cohort (70% and 30% of the dataset, respectively). Because most of the hyperparameters are overfitted by default, a validation set, as an internal testing cohort, was needed to optimize the hyperparameters to ensure the better generalization of the ML model. This method enabled the loss reduction, model modification and determination of the weight of every selected feature. Therefore, the training cohort was randomly split into a training set and validation set in the ratio of 9:1. A tenfold cross validation method was used to evaluate the accuracy of the model in the training cohort to optimize the hyperparameters. The split of the datasets is shown in Fig. The optimal hyperparameters based on accuracy were determined step by step: first, learning curves were plotted to select the appropriate number of DTs, which may effectively reduce calculation time and avoid introducing too many systematic errors; second, repeated grid search was employed to determine the partial optimal hyperparameters, which were overfitted by default; then, adjusted feature numbers were included in the ML model according to the fitting situation of model; finally, according to the above results, grid search was executed to find out the global optimal hyperparameters.